Hierarchical Intent Labeling for Consistent ML Training Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing intent labeling systems face challenges with large and complex intent spaces, leading to ambiguity and choice overload, where data labelers struggle to uniquely identify intent labels, resulting in the same data being labeled differently.
Innovation Solution
An intent ontology is used to generate intent labels composed of multiple actions or entities subject to a labeling flow hierarchy, presenting a subset of options at a time based on user selection, reducing available choices and improving readability through ordered intent labels.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If a large intent space with hundreds or thousands of candidate intents is provided to labelers, then the system can cover more diverse user intents, but labelers experience choice overload and ambiguity, resulting in inconsistent labeling
Solution Approach 1:
The patent segments the large intent space into a hierarchical structure with parent intents at higher levels and child intents at lower levels. Labelers first select from parent intents, then progressively refine to child intents. This segmentation reduces the number of options presented at each decision point, eliminating choice overload while maintaining comprehensive intent coverage across the entire hierarchy.
Solution Approach 2:
The patent introduces a hierarchical dimension to the intent selection process. Instead of presenting all intents in a single flat list, the system organizes intents across multiple hierarchical levels (parent, child, grandchild intents). This dimensional transformation allows labelers to navigate through layers of abstraction, reducing cognitive load while preserving the ability to select from a comprehensive set of intents.
2Adaptability or versatility
If all available intent labels are presented simultaneously to labelers, then complete intent options are available, but labelers struggle to uniquely identify the correct intent due to choice overload
Solution Approach 1:
The patent segments the complete set of intent labels into hierarchical groups, presenting only the relevant subset at each level. Labelers first see parent intents, then progressively see child intents only after selecting a parent. This segmentation ensures that the complete intent space remains available for coverage while preventing choice overload by limiting simultaneous options to manageable subsets.
Solution Approach 2:
The patent implements preliminary action by requiring labelers to first select parent intents before accessing child intent options. This preliminary selection step filters the available intent space, preparing a narrowed-down set of relevant options for the next labeling decision. This sequential approach improves identification accuracy by eliminating irrelevant options before the labeler makes the final intent selection.
3Productivity
If intent labels are generated following the hierarchy order, then the labeling process is systematic, but the readability and natural ordering of intent labels may be compromised
Solution Approach 1:
The patent implements dynamics by allowing the intent label ordering to adapt based on context. During the labeling process, intents are presented in hierarchical order for systematic selection. However, when generating the final intent label output, the system dynamically reorders the components to reflect natural language conventions and improve readability, rather than strictly maintaining the hierarchical presentation order.
Solution Approach 2:
The patent applies local quality by treating different stages of the labeling process differently. In the selection phase, hierarchical ordering provides systematic structure. In the output phase, natural language ordering provides readability. Each stage has its own optimal ordering strategy applied locally, rather than imposing a single global ordering constraint throughout the entire process.
Data Source
AI summary
In some embodiments, intent labeling may be facilitated for machine learning model training. In some embodiments, a set of natural language inputs may be obtained. Based on an action set, content related to intent labeling may be presented on a user interface. Based on an entity hierarchy and user selection of an agent-side action of the action set, a first entity subset of entities associated with the agent-side action may be presented on the user interface. Based on the entity hierarchy and user selection of a first entity from the first entity subset, a second entity subset of entities associated with the first entity may be presented on the user interface. Based on user selection of a second entity from the second entity subset, the natural language input may be associated with an ordered intent label that includes the agent-side action, the first entity, and the second entity.


