Chatbot Full Duplex Speech Latency Reduction

Solution Overview

Problem

Conventional chatbots using message-based conversation modes fail to replicate the natural, real-time interaction of human conversations, leading to latency and a feeling of paused responses, which limits the imitation of human-like dialogue.

Innovation Solution

Implementing a full duplex communication technique that predicts the user's intended expression through speech recognition, allowing for immediate response message generation and output when the user finishes speaking, thereby minimizing latency and enhancing the naturalness of voice conversations.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Loss of time

If message-based conversation mode is used, then conversation structure is clear and manageable, but response latency increases and naturalness of interaction deteriorates

Engineering Contradiction:

Improveresponse latencyVSAvoidconversation naturalness

Core Design Contradiction:

Loss of timeVSEase of operation

Solution Approach 1:

The system performs preliminary speech recognition and intent analysis during the user's speech input phase, preparing potential response messages before the user finishes speaking. This preliminary processing allows the chatbot to generate responses in advance, significantly reducing the latency between user input completion and chatbot response output.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The chatbot maintains continuous speech recognition and response generation processes without interrupting the user's speech. The system continuously processes user input and prepares responses in real-time, ensuring that the useful action of conversation flows continuously without pauses, thereby improving both response speed and naturalness.

Inventive Principle:

Principle #20Continuity of useful action

2Measurement precision

If response messages are generated after complete speech recognition, then accuracy is improved, but response time increases

Engineering Contradiction:

Improvespeech recognition accuracyVSAvoidresponse time

Core Design Contradiction:

Measurement precisionVSLoss of time

Solution Approach 1:

The system performs preliminary speech recognition and intent analysis during the user's speech input phase, preparing potential response messages before the user finishes speaking. This preliminary processing allows the chatbot to generate responses in advance, significantly reducing the latency between user input completion and chatbot response output.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The system generates multiple potential response messages based on partial speech recognition results, even before the user completes their speech. By preparing multiple candidate responses in advance and selecting the most appropriate one later, the system reduces response time while maintaining accuracy through subsequent verification against the complete speech input.

Inventive Principle:

Principle #16Partial or excessive action

3Ease of operation

If voice conversation is implemented, then user interaction convenience is improved, but latency and paused response feeling increase

Engineering Contradiction:

Improveinteraction convenienceVSAvoidconversation latency

Core Design Contradiction:

Ease of operationVSLoss of time

Solution Approach 1:

The system performs preliminary speech recognition and intent analysis during the user's speech input phase, preparing potential response messages before the user finishes speaking. This preliminary processing allows the chatbot to generate responses in advance, significantly reducing the latency between user input completion and chatbot response output.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The chatbot maintains continuous speech recognition and response generation processes without interrupting the user's speech. The system continuously processes user input and prepares responses in real-time, ensuring that the useful action of conversation flows continuously without pauses, thereby improving both response speed and naturalness.

Inventive Principle:

Principle #20Continuity of useful action

Data Source

PatentEP3714453B1Full duplex communication for conversation between chatbot and human

Publication Date: 2022.03.16 MICROSOFT TECHNOLOGY LICENSING LLC

Figure 1
Figure 2
Figure 3

AI summary

The present disclosure provides a technical solution related to full duplex communication for voice conversation between chatbot and human beings. More particularly, by using such technique, the conventional conversation mode with message as center in the art is subverted so as to realize a conversation mode in full duplex mode. The entire expression that a user intents to express may be predicted when obtaining intermediate result of speech recognition, and response messages may be generated in advance based on the predicted whole expression so that the generated response message may be output immediately when a response condition is satisfied, e.g., it is determined that a user has finished a paragraph of talking. With such technical solution, the latency from the end of voice input of a user and the start of speech output of a chatbot may be minimized.