Voice Assistant Handling of Concurrent Multi-User Utterances
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current automated assistants struggle to differentiate between multiple concurrent spoken utterances from different users, leading to incorrect actions, resource wastage, and prolonged interactions due to unprocessed utterances.
Innovation Solution
An automated assistant that can determine whether to combine or selectively respond to multiple requests from multiple users, considering user verification and context, and renders selectable elements for user selection to process intended utterances.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the automated assistant processes all concurrent spoken utterances from multiple users, then the completeness of request processing is improved, but the complexity of differentiating between users and requests deteriorates
Solution Approach 1:
The patent segments the audio data into separate spoken utterances attributed to different users using speaker diarization technology. This segmentation allows the system to process each user's request independently, resolving the contradiction by making the differentiation process systematic rather than complex ad-hoc analysis.
Solution Approach 2:
The patent introduces an intermediary processing layer that includes speaker diarization and request differentiation modules. This intermediary layer sits between audio capture and request processing, automatically attributing utterances to users and filtering out non-request speech, thereby reducing the complexity burden on the main processing system.
2Speed
If the automated assistant uses fixed criteria to differentiate between concurrent utterances, then the processing speed is improved, but the accuracy of identifying intended requests deteriorates
Solution Approach 1:
The patent employs dynamic differentiation criteria that adapt based on contextual analysis. Rather than fixed rules, the system uses machine learning models that evaluate multiple features (speaker identity, contextual relevance, intent detection) to dynamically determine which utterances constitute requests, thereby maintaining high accuracy without sacrificing processing speed through rigid fixed criteria.
Solution Approach 2:
The patent changes multiple parameters simultaneously for request identification: speaker identity attribution, contextual relevance scoring, and intent detection thresholds. By adjusting these parameters dynamically based on the situation, the system achieves both speed and accuracy in identifying intended requests among concurrent utterances.
3Speed
If the automated assistant processes concurrent utterances without user verification, then the responsiveness is improved, but the security and resource efficiency deteriorate
Solution Approach 1:
The patent performs preliminary speaker verification and request validation before initiating full request processing. This preliminary action includes verifying user identity and determining whether the utterance constitutes a valid request, allowing the system to quickly filter out non-requests or unauthenticated users without sacrificing security, while still maintaining responsiveness for authenticated users with valid requests.
Data Source
Figure 1A
Figure 1B
Figure 1C
AI summary
Implementations set forth herein relate to an automated assistant that is responsive to spoken utterances spoken by multiple different users simultaneously, or otherwise within a narrow window of time, in furtherance of initializing one or more actions. For instance, when a primary user is interacting with an automated assistant, a secondary user may also provide a spoken utterance. In response, the automated assistant can selectively determine whether any input from the secondary user should affect a request from the primary user. In some instances, the automated assistant can elect to either disregard the secondary input, separately respond to the secondary input, or use some amount of content of the secondary input to further the request from the primary user. Incorporation of secondary input when fulfilling a primary request can be fluidly performed to resemble human conversation and eliminate any unnecessary engagement between the automated assistant and the users.