Automated Dialog Analysis System for Speech Recognition Error Detection

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing speech recognition systems face challenges in identifying dialog problems from logged data, as manual analysis is tedious and time-consuming, and automated analysis is often application-specific and limited in extensibility, making it difficult to pinpoint the source of user frustration and system mismatches.

Innovation Solution

An automated dialog analysis technique that analyzes log data using dialog move information such as turn types and prompt types to identify likely problems, allowing for rapid identification of underperforming dialog states and sources of user confusion without the need for manual transcription.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If manual analysis of session data is conducted to diagnose dialog problems, then measurement precision of user behavior is improved, but loss of time and productivity deteriorate due to the long and tedious process

Engineering Contradiction:

Improveaccuracy of user behavior analysisVSAvoidtime required for data analysis

Core Design Contradiction:

Measurement precisionVSLoss of time

Solution Approach 1:

The patent replaces manual mechanical analysis of session data with an automated computer-based analysis system. The system automatically processes speech and DTMF session data, transcriptions, and dialog logs to identify dialog problems, eliminating the tedious manual review process while maintaining or improving analysis accuracy through systematic algorithmic evaluation of user interactions.

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

Solution Approach 2:

The patent introduces an automated analysis system as an intermediary between raw session data and diagnostic insights. This intermediary system processes large volumes of dialog data, applies analysis rules, and generates diagnostic reports automatically, serving as a bridge that transforms unprocessed session logs into actionable diagnostic information without requiring direct manual intervention.

Inventive Principle:

Principle #24Intermediary (Mediator)

2Productivity

If automated analysis is implemented to reduce analysis time, then productivity is improved, but adaptability deteriorates because implementations are generally application-specific and limited in extensibility

Engineering Contradiction:

Improvespeed of dialog problem identificationVSAvoidextensibility across different applications

Core Design Contradiction:

ProductivityVSAdaptability or versatility

Solution Approach 1:

The patent creates a universal analysis system that can handle multiple types of session data (speech, DTMF, transcriptions, dialog logs) and apply consistent analysis rules across different applications. The system is designed to be extensible and adaptable to various dialog interaction models through configuration rather than hardcoding, allowing it to serve multiple applications with different interaction patterns while maintaining high productivity.

Inventive Principle:

Principle #6Universality (Multi-functionality)

Solution Approach 2:

The patent implements a dynamic analysis system where the analysis rules and parameters can be adjusted and configured based on specific application requirements. The system adapts to different dialog interaction models by allowing customization of analysis criteria, making it flexible enough to handle diverse applications while maintaining automated high-speed analysis capabilities.

Inventive Principle:

Principle #15Dynamics

3Ease of operation

If speech recognition is used to capture user input, then ease of operation is improved, but measurement precision deteriorates due to the imperfection of speech recognizers requiring manual transcriptions for true analysis

Engineering Contradiction:

Improveconvenience of user inputVSAvoidaccuracy of user input data

Core Design Contradiction:

Ease of operationVSMeasurement precision

Solution Approach 1:

The patent uses transcriptions as an intermediary layer between speech recognition output and analysis. Rather than relying directly on potentially inaccurate speech recognition results, the system processes manual or corrected transcriptions to ensure accurate capture of user intent, while still maintaining the ease of voice-based user input for the actual interaction.

Inventive Principle:

Principle #24Intermediary (Mediator)

Data Source

PatentUS7853453B2Analyzing dialog between a user and an interactive application

Publication Date: 2010.12.14 MICROSOFT TECHNOLOGY LICENSING LLC

AI summary

A computer implemented method of analyzing dialog between a user and an interactive application having dialog turns. The method includes receiving information indicative of dialog turns between the system and at least one user in an application. The turns are related to one or more tasks of the application. A diagnostic module operable on a computer is utilized to obtain an indication of performance of the application relative to said one or more tasks.