Dialog System Intent Recognition with Human Curation

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Dialog systems often fail to determine the intent of user input natural language queries, leading to poor information retrieval and user accessibility issues, as they struggle to find correspondence between natural language expressions and database queries.

Innovation Solution

A dialog system equipped with a trained intent recognition model, which uses a deep learning model to understand natural language queries and updates the model based on human-cured candidate responses, allowing it to identify intent tags and generate relevant responses, even for queries with initially unidentifiable intent.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If a dialog system uses a trained intent recognition model to identify user intent, then the system can provide relevant information retrieval, but the system fails to determine intent for queries with unidentifiable intent patterns

Engineering Contradiction:

Improveintent identification accuracyVSAvoidcoverage of intent types

Core Design Contradiction:

ReliabilityVSAdaptability or versatility

Solution Approach 1:

The system performs preliminary actions by generating candidate responses and intent tags before human curation is available. The dialog generation engine creates candidate responses and associated intent tags in advance, which are then refined by human curators. This allows the system to proactively expand its intent recognition capabilities rather than passively waiting for failures to occur.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The system implements feedback loops where human curators review and refine candidate responses and intent tags generated by the dialog generation engine. The refined intent tags are then used to retrain the intent recognition model, creating a continuous improvement cycle. This feedback mechanism allows the system to learn from both successful and unsuccessful intent identification cases.

Inventive Principle:

Principle #23Feedback

2Adaptability or versatility

If the system generates candidate responses for unidentifiable queries, then it can expand its intent recognition capabilities, but this requires human curation resources and time

Engineering Contradiction:

Improveintent recognition coverageVSAvoidmodel update time

Core Design Contradiction:

Adaptability or versatilityVSLoss of time

Solution Approach 1:

The system performs preliminary generation of candidate responses and intent tags before human curation is needed. By having the dialog generation engine create these candidates in advance, the system minimizes the actual human curation time required, as curators only need to review and refine pre-generated candidates rather than create everything from scratch.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The system partially serves itself by automatically generating candidate responses and intent tags without human intervention. The dialog generation engine autonomously creates these candidates based on unidentifiable queries, reducing the burden on human curators to manually analyze every failed intent recognition case.

Inventive Principle:

Principle #25Self-service

3Adaptability or versatility

If the intent recognition model is continuously updated with new intent tags, then it can handle diverse queries, but the system complexity increases

Engineering Contradiction:

Improvequery handling diversityVSAvoidmodel training pipeline complexity

Core Design Contradiction:

Adaptability or versatilityVSDevice complexity

Solution Approach 1:

The system introduces an intermediary layer (the dialog generation engine) that sits between the intent recognition model and the human curators. This intermediary automatically generates candidate responses and intent tags, serving as a bridge that simplifies the overall system architecture by automating the content generation process while maintaining human-in-the-loop refinement.

Inventive Principle:

Principle #24Intermediary (Mediator)

Solution Approach 2:

The dialog generation engine serves multiple functions: it generates candidate responses for unidentifiable queries, creates associated intent tags, and provides training data for model updates. This multi-functional component reduces overall system complexity by consolidating several operations into a single versatile module.

Inventive Principle:

Principle #6Universality (Multi-functionality)

Data Source

PatentUS11948566B2Extensible search, content, and dialog management system with human-in-the-loop curation

Publication Date: 2024.04.02 ADOBE INC

AI summary

The present disclosure describes systems and methods for extensible search, content, and dialog management. Embodiments of the present disclosure provide a dialog system with a trained intent recognition model (e.g., a deep learning model) to receive and understand a natural language query from a user. In cases where intent is not identified for a received query, the dialog system generates one or more candidate responses that may be refined (e.g., using human-in-the-loop curation) to generate a response. The intent recognition model may be updated (e.g., retrained) the accordingly. Upon receiving a subsequent query with similar intent, the dialog system may identify the intent using the updated intent recognition model.