On-Device NLP Procedural Memory for Low-Latency Voice Response
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing natural language processing systems face challenges in efficiently processing user inputs on-device due to the need for multiple components and potential latency issues, especially in privacy mode or when communication with remote systems is interrupted.
Innovation Solution
Implementing a procedural memory component trained to process ASR and NLU data, allowing on-device processing of user inputs by learning from past interactions to determine responsive outputs, reducing the need for cloud-based systems and minimizing latency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If multiple components are used for natural language processing, then processing capability is improved, but system complexity and latency increase
Solution Approach 1:
The patent combines multiple natural language processing components (ASR, NLU, dialogue management, natural language generation) into a single integrated neural network model. This unified model processes user inputs end-to-end, eliminating the need for separate components and their associated interfaces, thereby reducing system complexity while maintaining comprehensive processing capability.
Solution Approach 2:
The unified neural network model performs multiple functions that traditionally required separate components: speech-to-text conversion, intent recognition, dialogue state tracking, and text generation. This multi-functional model reduces the number of components needed while preserving all necessary processing capabilities.
2Productivity
If multiple components are used for natural language processing, then processing capability is improved, but latency increases
Solution Approach 1:
By merging multiple processing stages into a single unified neural network model, the patent eliminates the sequential processing delays between components. The model processes user inputs in a single pass through its architecture, reducing the cumulative latency that would result from multiple separate component executions.
3Measurement precision
If cloud-based systems are used, then processing accuracy is improved, but reliability during communication interruptions deteriorates
Solution Approach 1:
The unified neural network model is designed to operate autonomously on local devices without requiring continuous cloud connectivity. It independently processes user inputs and generates responses using its internal knowledge and processing capabilities, ensuring system reliability even when cloud-based systems are unavailable due to communication interruptions.
Data Source
AI summary
The present disclosure provide techniques for processing natural language inputs on a user device or a system. In some embodiments, a device includes a component configured to predict an output(s) responsive to a user input, where the component learns the output based on historic processing performed by a more robust natural language processing system. The component processes ASR and NLU data to determine: (1) a natural language output that is presented as an audio or a visual output, (2) an action to be performed by a skill, and/or (3) a skill to respond to the user input. The component is automatically updated for new intents, features, business logic, etc., based on learning from the natural language processing system.