Automated Assistant Mouth and Gaze Detection for Resource Efficiency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing automated assistant systems inefficiently utilize computing resources and network bandwidth due to indiscriminate data transmission and processing, leading to unnecessary power consumption and resource waste.
Innovation Solution
Adapt automated assistants by detecting user mouth movement and directed gaze to selectively initiate communication and processing, reducing unnecessary data transmission and processing through localized adaptation of user interface and sensor data handling.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If automated assistant systems continuously transmit and process sensor data, then user interaction responsiveness is improved, but computing resource consumption and power usage increase
Solution Approach 1:
The system performs preliminary detection of mouth movement and gaze direction locally at the client device before transmitting data to the remote system. This preliminary action filters out unnecessary data transmission, reducing network bandwidth consumption and remote computing resource usage while maintaining responsive interaction by processing critical data locally in advance
Solution Approach 2:
The patent implements local processing of sensor data at the client device, where mouth movement detection and gaze analysis are performed locally before any data is transmitted remotely. This local quality approach allows the system to make intelligent decisions about data transmission, reducing overall computing resource consumption while preserving interaction responsiveness for actual user commands
2Measurement precision
If automated assistant systems transmit all sensor data to remote systems, then processing accuracy is improved, but network bandwidth consumption increases
Solution Approach 1:
The system extracts and transmits only the most relevant information (mouth movement detection and gaze direction) to the remote system after performing local processing. This extraction approach reduces network bandwidth consumption by filtering out unnecessary sensor data while maintaining processing accuracy for identifying user interaction intent
Solution Approach 2:
The client device performs preliminary detection and filtering of sensor data locally, identifying only the critical features (mouth movement and gaze) that need to be transmitted. This preliminary action ensures that transmitted data maintains high processing accuracy while significantly reducing network bandwidth requirements
3Measurement precision
If automated assistants continuously monitor and process data, then interaction accuracy is improved, but computational load increases
Solution Approach 1:
The patent implements local processing of sensor data at the client device, where mouth movement detection and gaze analysis are performed locally before any data is transmitted remotely. This local quality approach allows the system to make intelligent decisions about data transmission, reducing overall computing resource consumption while preserving interaction responsiveness for actual user commands
Solution Approach 2:
The system monitors sensor data continuously but only performs heavy processing and transmission during periods when mouth movement and gaze are detected. This periodic action approach maintains interaction accuracy by processing data only when needed, while reducing computational load during idle periods
Data Source
AI summary
Adapting an automated assistant based on detecting: movement of a mouth of a user; and/or that a gaze of the user is directed at an assistant device that provides an automated assistant interface (graphical and/or audible) of the automated assistant. The detecting of the mouth movement and/or the directed gaze can be based on processing of vision data from one or more vision components associated with the assistant device, such as a camera incorporated in the assistant device. The mouth movement that is detected can be movement that is indicative of a user (to whom the mouth belongs) speaking.