Shared Utterance Encoder for Scalable Speech Application Routing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current speech recognition systems face challenges in efficiently determining the appropriate application for user commands, especially as the number of supported applications grows, leading to increased computational load, latency, and memory footprint, due to the need for retraining models with each new application addition and handling vocabulary overlap.
Innovation Solution
The implementation of a multi-task learning system using a shared utterance encoder with BiLSTM models and application-specific feedforward layers, which allows for individual training of new applications without modifying the encoder, reducing the computational burden and enabling more accurate determination of applicable applications.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If the system supports more applications, then the versatility and functionality of the speech recognition system is improved, but the computational load, latency, and memory footprint increase due to the need for retraining models
Solution Approach 1:
The system segments the application selection process into two independent stages: (1) a shared utterance encoder that processes the input utterance and generates a feature representation, and (2) multiple application-specific feedforward layers that independently process the encoded features to determine applicability. This segmentation allows new applications to be added by simply training new feedforward layers without retraining the encoder, thus reducing latency and avoiding the need to reprocess existing applications.
Solution Approach 2:
The shared utterance encoder is designed as a universal component that serves all applications simultaneously. It processes utterances and generates feature representations that are reused by multiple application-specific feedforward layers. This multi-functionality allows the encoder to benefit from data across all applications during training while new applications can leverage the pre-trained encoder without requiring retraining, thereby reducing computational load and latency.
2Adaptability or versatility
If the system supports more applications, then the versatility is improved, but the memory footprint and computational resources increase due to storing and processing vocabulary for each application
Solution Approach 1:
The system merges the vocabulary processing functionality into a shared utterance encoder that is trained on data from multiple applications simultaneously. This consolidation eliminates the need to store and process separate vocabulary representations for each application, as the encoder learns unified feature representations that capture patterns across all applications. The memory footprint is reduced because the encoder stores shared knowledge rather than redundant application-specific vocabulary data.
Solution Approach 2:
The shared utterance encoder serves as a universal resource that all applications can utilize. Instead of each application maintaining its own vocabulary and processing pipeline, the encoder provides a common foundation that reduces overall memory requirements. The encoder's parameters are updated using data from all applications, creating a compact representation that captures cross-application patterns without requiring separate storage for each application's vocabulary.
3Measurement precision
If the system retrains models with each new application addition, then the accuracy in identifying applicable applications is maintained, but the computational burden and time required for updates increase
Solution Approach 1:
The system segments the model architecture into a fixed shared encoder and flexible application-specific feedforward layers. When new applications are added, only the new feedforward layers need to be trained, not the entire model. This segmentation enables rapid system updates because the time-consuming encoder training is performed once and reused, while new applications can be integrated by training only their specific layers, significantly improving update speed without sacrificing accuracy.
Solution Approach 2:
The shared utterance encoder is trained in advance on data from multiple applications before new applications are added. This preliminary training creates a robust feature representation that captures general patterns across applications. When new applications are introduced, the pre-trained encoder provides a strong foundation, allowing the new application-specific layers to be trained quickly and accurately without requiring retraining of the entire system, thus maintaining accuracy while improving update productivity.
Data Source
AI summary
Devices and techniques are generally described for application determination in speech processing. Input data corresponding to a spoken utterance may be received. Speech recognition processing may be performed on the input data to generate text data. A machine learning encoder may generate a vector representation of the input data. A first binary classifier may determine a first probability that the input data corresponds to a first speech-processing application. A second binary classifier may determine a second probability that the input data corresponds to a second speech-processing application. A selection between the first speech-processing application and the second speech-processing application may be made based at least in part on the first probability and the second probability.