Multi-Hotword Assistant Collaboration for Single-Utterance Queries
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing speech-enabled devices require separate queries to multiple assistant-enabled devices or digital assistant services, as they are resource-constrained and cannot continuously run full automated speech recognition, making it challenging to trigger multiple devices or services simultaneously with a single utterance.
Innovation Solution
Implementing a collaboration routine that allows multiple assistant-enabled devices to recognize partial hotwords and collaborate to fulfill a single utterance query, using low-power hotword detection models to wake up and initiate speech recognition, and sharing processing tasks between devices.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Use of energy by moving object
If devices operate in sleep state and wake up only on hotword detection, then energy consumption is reduced, but the ability to respond to multiple devices in a single utterance is limited
Solution Approach 1:
The system performs preliminary hotword detection in a low-power sleep state, allowing devices to wake up selectively when their specific hotword is detected. This preliminary action enables energy-efficient operation while maintaining the capability to respond to multiple devices through the collaboration routine that identifies multiple hotwords in a single utterance.
Solution Approach 2:
The collaboration routine provides multi-functionality by enabling a single utterance containing multiple hotwords to trigger coordinated responses across multiple devices. The system universally handles both single-device and multi-device scenarios through the same hotword detection and collaboration mechanism.
2Measurement precision
If full automated speech recognition is continuously run, then speech processing accuracy is improved, but resource consumption increases
Solution Approach 1:
The speech recognition process is segmented into two stages: low-power hotword detection for wake-up, and full automated speech recognition only when needed. This segmentation allows the system to maintain high speech processing accuracy for the actual query while minimizing resource consumption by running full ASR only after hotword detection succeeds.
Solution Approach 2:
The system uses periodic action by switching between sleep state with hotword detection and active state with full speech recognition. Full ASR is executed periodically only when triggered by successful hotword detection, rather than continuously, thereby reducing resource consumption while maintaining accuracy when needed.
3Loss of energy
If separate queries are required for multiple devices, then device resource constraints are respected, but user interaction complexity increases
Solution Approach 1:
The collaboration routine merges multiple device responses into a single unified interaction. When multiple hotwords are detected in one utterance, the system combines the queries and executes them together, allowing users to interact with multiple devices simultaneously using a single speech input rather than requiring separate queries for each device.
4Adaptability or versatility
If multiple hotwords are recognized in a single utterance, then device collaboration is enabled, but hotword detection complexity increases
Solution Approach 1:
The collaboration routine acts as an intermediary layer between hotword detection and device execution. It mediates the process by receiving the utterance, detecting multiple hotwords, determining which devices should be invoked, and coordinating their collaborative execution. This intermediary approach manages the complexity of multi-hotword detection while enabling versatile device collaboration.
Data Source
Figure 1A
Figure 1B
Figure 1C
AI summary
A method (400) for combining hotwords (24) in a single utterance (20) includes receiving, at a first assistant-enabled device (AED) (110), audio data (14) corresponding to an utterance directed toward the first AED and a second AED among two or more AEDs where the audio data includes a query (22) specifying an operation to perform. The method also detects, using a hotword detector (130), a first hotword assigned to the first AED that is different than a second hotword assigned to the second AED. In response to detecting the first hotword, the method initiates processing on the audio data to determine that the audio data includes a term preceding the query that at least partially matches the second hotword. Based on the at least partial match, the method executes a collaboration routine (200) to cause the first AED and the second AED to collaborate with one another to fulfill the query.