Incremental Speech UI Response via Local Remote Recognition Segmentation

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Current speech recognition systems face delays in user interface responses due to reliance on a single recognition result, which can lead to perceived delays and inaccuracies, especially when local recognizers are used in resource-constrained environments and may produce false accepts due to similarity with inputs intended for more powerful recognizers.

Innovation Solution

Implementing a speech recognition response manager that initiates a user interface response based on local recognition results and modifies it incrementally with remote recognition results, allowing for early response initiation and correction, thereby reducing perceived delays and improving accuracy through incremental UI responses and asynchronous evidence processing.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Speed

If a local recognizer is used to provide fast response, then response speed is improved, but recognition accuracy deteriorates due to limited system resources and coverage

Engineering Contradiction:

Improveresponse speedVSAvoidrecognition accuracy

Core Design Contradiction:

SpeedVSMeasurement precision

Solution Approach 1:

The patent segments the recognition task into two parts: local recognition handles the time-critical initial processing to provide fast response, while remote recognition handles the accuracy-critical final verification. This segmentation allows each component to optimize for its specific function without compromising the other.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The local recognizer performs preliminary recognition action immediately upon receiving user input, generating an initial result before the remote recognizer finishes processing. This preliminary action ensures fast response while the more accurate remote recognition result is still being computed.

Inventive Principle:

Principle #10Preliminary action

2Measurement precision

If a remote recognizer is used to provide superior coverage and accuracy, then recognition accuracy is improved, but response time deteriorates due to network delays

Engineering Contradiction:

Improverecognition accuracyVSAvoidresponse time

Core Design Contradiction:

Measurement precisionVSLoss of time

Solution Approach 1:

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The system dynamically adjusts the response based on the availability of recognition results. When local recognition result is available, immediate UI response is triggered. When remote recognition result becomes available, the UI response is modified to incorporate the more accurate remote result, allowing the system to adapt between speed and accuracy requirements.

Inventive Principle:

Principle #15Dynamics

3Measurement precision

If the system waits for remote recognition result before providing UI response, then recognition accuracy is improved, but perceived delay increases

Engineering Contradiction:

Improverecognition accuracyVSAvoidperceived delay

Core Design Contradiction:

Measurement precisionVSLoss of time

Solution Approach 1:

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The system uses feedback from the remote recognition result to modify the initial UI response. The feedback mechanism allows the system to correct inaccuracies while maintaining the speed advantage of immediate response, effectively resolving the contradiction between accuracy and perceived delay.

Inventive Principle:

Principle #23Feedback

4Speed

If local recognition result is accepted with high confidence, then response speed is maintained, but false accepts occur due to similarity with inputs intended for remote recognizers

Engineering Contradiction:

Improveresponse speedVSAvoidrecognition reliability

Core Design Contradiction:

SpeedVSReliability

Solution Approach 1:

The system uses feedback from the remote recognition result to verify and correct the local recognition result. This feedback mechanism allows the system to maintain speed by accepting local results initially, while then verifying accuracy through remote recognition to eliminate false accepts.

Inventive Principle:

Principle #23Feedback

Solution Approach 2:

The remote recognition result acts as an intermediary verification layer between the local recognition result and the final UI response. This intermediary check ensures reliability by catching false accepts before the UI response is finalized, while allowing the system to maintain fast response by using the local result as the initial basis.

Inventive Principle:

Principle #24Intermediary (Mediator)

Data Source

PatentUS9384736B2Method to provide incremental UI response based on multiple asynchronous evidence about user input

Publication Date: 2016.07.05 MICROSOFT TECHNOLOGY LICENSING LLC

AI summary

Techniques disclosed herein include systems and methods for managing user interface responses to user input including spoken queries and commands. This includes providing incremental user interface (UI) response based on multiple recognition results about user input that are received with different delays. Such techniques include providing an initial response to a user at an early time, before remote recognition results are available. Systems herein can respond incrementally by initiating an initial UI response based on first recognition results, and then modify the initial UI response after receiving secondary recognition results. Since an initial response begins immediately, instead of waiting for results from all recognizers, it reduces the perceived delay by the user before complete results get rendered to the user.