Multi-Assistant Hotword Collaboration in a Single Utterance
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing assistant-enabled devices require separate queries to multiple devices or digital assistant services, necessitating multiple spoken commands for simultaneous operations, which is inefficient and time-consuming.
Innovation Solution
Implementing a method for combining device-specific hotwords in a single utterance, allowing multiple assistant-enabled devices to collaborate and interpret partial hotwords to fulfill a query, thereby enabling simultaneous operation through a single spoken command.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If separate queries are used for multiple devices, then each device can be controlled independently, but the user must speak multiple commands which is inefficient and time-consuming
Solution Approach 1:
The patent combines multiple device-specific hotwords into a single utterance that can trigger multiple devices simultaneously. The system detects partial hotwords for different devices within one continuous speech input, merges the detection results, and coordinates execution across multiple devices, allowing users to control multiple devices with a single spoken command rather than separate queries for each device.
2Adaptability or versatility
If multiple hotwords are detected in a single utterance, then multiple devices can be triggered simultaneously, but the hotword detection system becomes more complex
Solution Approach 1:
The patent segments the hotword detection process into distinct stages: initial hotword detection for the first device, subsequent partial hotword detection for additional devices, and coordinated execution. The system maintains separate hotword models for different devices but processes them in a structured sequence within a single utterance, managing complexity through phased detection rather than simultaneous processing of all hotwords.
Solution Approach 2:
The patent introduces a coordination system that acts as an intermediary between multiple hotword detection processes. When partial hotwords for multiple devices are detected in a single utterance, the coordination system manages the interaction between devices, determines execution priority, and coordinates the triggering process, thereby managing system complexity while enabling multi-device functionality.
3Use of energy by moving object
If devices operate in sleep state with hotword detection, then energy consumption is reduced, but the system can only wake up for exact hotword matches not partial matches
Solution Approach 1:
The patent enables partial hotword matching where the system can wake from sleep state and process utterances that contain partial matches for device hotwords. Instead of requiring complete hotword sequences, the system detects partial hotwords within the utterance, activates the device, and then processes the full query, thereby reducing the strictness of wake-trigger requirements while maintaining energy-efficient sleep operation.
Data Source
AI summary
A method for combining hotwords in a single utterance receives, at a first assistant-enabled device (AED), audio data corresponding to an utterance directed toward the first AED and a second AED among two or more AEDs where the audio data includes a query specifying an operation to perform. The method also detects, using a hotword detector, a first hotword assigned to the first AED that is different than a second hotword assigned to the second AED. In response to detecting the first hotword, the method initiates processing on the audio data to determine that the audio data includes a term preceding the query that at least partially matches the second hotword assigned. Based on the at least partial match, the method executes a collaboration routine to cause the first AED and the second AED to collaborate with one another to fulfill the query.