Dual Neural Network Vocal Command Recognition
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing Voice User Interface (VUI) systems face latency and privacy concerns due to reliance on cloud networks for Automatic Speech Recognition (ASR), which also consume excessive energy and resources, especially when devices lack processing and memory capabilities.
Innovation Solution
Implementing a dual neural network pipeline on edge devices for autonomous vocal command recognition, where a first neural network continuously performs Keyword Spotting (KWS) and activates a second, more complex network to detect command words without cloud involvement, optimizing resource usage and reducing latency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If cloud network is used for ASR processing, then processing capability and accuracy are improved, but latency and energy consumption increase
Solution Approach 1:
The patent segments the speech recognition system into two parts: a lightweight Keyword Spotting (KWS) model running locally on the edge device for real-time keyword detection, and a more complex ASR model running on the cloud for accurate speech recognition. This segmentation allows the system to achieve both low latency (through local KWS) and high accuracy (through cloud ASR) without requiring the entire ASR model to run locally or continuously connect to the cloud.
Solution Approach 2:
The system performs preliminary Keyword Spotting detection locally on the edge device before transmitting audio data to the cloud for ASR processing. This preliminary action filters and prepares the audio data, ensuring that only relevant segments needing ASR processing are transmitted, thereby reducing overall latency and energy consumption while maintaining recognition accuracy.
2Measurement precision
If cloud network is used for ASR processing, then processing capability is improved, but energy consumption increases
Solution Approach 1:
The patent segments the speech recognition system into two parts: a lightweight Keyword Spotting (KWS) model running locally on the edge device for real-time keyword detection, and a more complex ASR model running on the cloud for accurate speech recognition. This segmentation allows the system to achieve both low latency (through local KWS) and high accuracy (through cloud ASR) without requiring the entire ASR model to run locally or continuously connect to the cloud.
Solution Approach 2:
The system performs preliminary Keyword Spotting detection locally on the edge device before transmitting audio data to the cloud for ASR processing. This preliminary action filters and prepares the audio data, ensuring that only relevant segments needing ASR processing are transmitted, thereby reducing overall latency and energy consumption while maintaining recognition accuracy.
3Measurement precision
If complex neural network is used for vocal command detection, then detection accuracy is improved, but processing workload increases
Solution Approach 1:
The patent segments the speech recognition system into two parts: a lightweight Keyword Spotting (KWS) model running locally on the edge device for real-time keyword detection, and a more complex ASR model running on the cloud for accurate speech recognition. This segmentation allows the system to achieve both low latency (through local KWS) and high accuracy (through cloud ASR) without requiring the entire ASR model to run locally or continuously connect to the cloud.
Solution Approach 2:
The system dynamically activates the second (more complex) neural network only after the first neural network detects a keyword. This dynamic activation ensures that the complex ASR model is processed only when necessary, optimizing the balance between processing workload and detection accuracy.
4Speed
If continuous operation of second neural network is implemented, then response time is reduced, but energy consumption increases
Solution Approach 1:
The system dynamically activates the second (more complex) neural network only after the first neural network detects a keyword. This dynamic activation ensures that the complex ASR model is processed only when necessary, optimizing the balance between processing workload and detection accuracy.
Solution Approach 2:
The system uses periodic action by continuously monitoring audio data with the first neural network and only activating the second neural network periodically when keywords are detected, rather than continuously operating both networks. This approach reduces energy consumption while maintaining fast response time for vocal command recognition.
Data Source
AI summary
A method to detect a vocal command, the method including: analyzing audio data received from a transducer configured to convert audio into an electric signal and analyzing the data using a first neural network. The method also includes detecting a keyword from the audio data using the first neural network on the edge device, the first neural network being trained to recognize the keyword. The method further includes activating a second neural network after the keyword is identified by the first neural network and analyzing the audio data using the second neural network, the second neural network being trained to recognize a set of vocal commands. The method to detect a vocal command may also include detecting the vocal command word using the second neural network.