Custom Deep Learning Circuits for Always-On Command Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional always-on devices face challenges in maintaining low power consumption while providing accurate event detection and command recognition, leading to limited battery life and privacy concerns due to reliance on network-based computing and intermittent network connectivity.
Innovation Solution
A deep learning system integrated into a portable device, utilizing a customized neural network topology with fixed and programmable weights, enables efficient voice and audio command recognition directly on the device, reducing power consumption and eliminating the need for network data transmission.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If network-based computing is used for command recognition, then detection accuracy can be improved, but power consumption increases and battery life decreases
Solution Approach 1:
The system segments the computing tasks by separating always-on lightweight audio processing from periodic deep learning inference. The audio interface and feature extraction run continuously at low power, while the neural network processor remains dormant and only activates when voice activity is detected, thus dividing energy consumption across different operational states.
Solution Approach 2:
The system performs preliminary audio processing and voice activity detection before engaging the power-intensive neural network processor. By pre-processing audio signals and identifying potential voice commands in advance, the system prepares data structures and triggers neural network execution only when necessary, avoiding continuous high-power operation.
2Speed
If the device remains always on to accept commands, then responsiveness is improved, but power consumption increases
Solution Approach 1:
The system implements periodic operation through duty cycling, where the application processor alternates between active and sleep states. The audio interface operates continuously at minimal power to detect wake words, triggering periodic activation of the full processing stack only when commands are detected, thus achieving responsiveness without continuous high-power operation.
Solution Approach 2:
The system dynamically adjusts its operational state based on input detection. The audio processing pipeline remains active in a low-power mode continuously, but the application processor and neural network processor dynamically transition from sleep to active states only when voice activity or specific audio patterns are detected, optimizing the balance between responsiveness and power consumption.
3Power
If network data transmission is used for command processing, then processing capability is improved, but reliability decreases due to network connectivity issues
Solution Approach 1:
The system introduces an on-device neural network processor as an intermediary between the audio interface and the application processor. This local inference engine processes voice commands directly on the device without requiring network connectivity, acting as a mediator that enables reliable local processing while maintaining the option for cloud-based processing when needed.
Solution Approach 2:
The device performs self-service processing by executing neural network inference locally on-device. The system processes voice commands and recognizes patterns without external assistance, enabling autonomous operation independent of network availability. This self-sufficient processing capability ensures reliability during network outages while maintaining full functionality.
Data Source
AI summary
A system can include a circuit holistically customized for the detection of commands in an audio or video input signal to meet certain application-specific requirements. The circuit can have a neural network topology that is hardwired to perform detection based on application-specific detection criteria. The hardwired custom circuit can provide improved energy efficiencies compared to similar functionality carried out using software and generic hardware modules. The system can also include a sound change trigger module and perform non-voiced sound detection.