Neural Speech-to-Meaning Recognition Without Transcription
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional virtual assistants face inefficiencies and high costs due to complex components, data imperfections, and the need for language and application-specific fine-tuning, leading to inaccuracies in natural language understanding and intent recognition.
Innovation Solution
A neural network-based system for intent and variable recognition that processes digitized speech directly into actionable intent data structures, using intent and variable recognizers trained on diverse voices and phrasings, with optional domain recognition, to reduce complexity and improve accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional ASR and NLU components are used, then speech audio can be processed to produce intent, but the system complexity and data collection costs increase significantly
Solution Approach 1:
The patent combines automatic speech recognition (ASR) and natural language understanding (NLU) into a single integrated neural network model. This unified model processes speech audio directly to produce intent data structures, eliminating the need for separate ASR and NLU components. The merging reduces system complexity while maintaining or improving intent recognition accuracy through end-to-end training on diverse voices and phrasings.
Solution Approach 2:
The neural network model is designed to perform multiple functions: it simultaneously handles speech recognition, language understanding, and intent classification. The model is trained on diverse voices and phrasings across multiple languages, making it universally applicable to various speech inputs without requiring language-specific or domain-specific fine-tuning, thereby reducing overall system complexity.
2Measurement precision
If multiple transcription hypotheses are generated and processed, then intent recognition accuracy may improve, but processing time and computational resources increase
Solution Approach 1:
The patent extracts and eliminates the intermediate transcription hypothesis generation step from the conventional pipeline. Instead of generating multiple transcription hypotheses and processing each through NLU, the unified neural network processes speech audio directly to produce intent data structures. This extraction of the intermediate step reduces processing time while maintaining accuracy through direct end-to-end learning.
3Adaptability or versatility
If conventional ASR and NLU components are used, then speech processing can be performed, but data collection and fine-tuning costs increase for each language and application
Solution Approach 1:
The neural network model is designed with universal applicability across multiple languages and domains. It is trained on diverse voices and phrasings from multiple languages during the initial training phase, enabling it to handle various speech inputs without requiring separate fine-tuning for each language or domain. This universality significantly reduces data collection and fine-tuning costs compared to conventional systems that require language-specific adaptation.
4Device complexity
If intermediate transcription stages are eliminated, then system complexity is reduced, but processing accuracy may be affected
Solution Approach 1:
The patent merges ASR and NLU into a single neural network model that processes speech audio directly to produce intent data structures. This merging eliminates intermediate transcription stages and the interfaces between separate components, reducing system complexity. The end-to-end training approach ensures that the model learns optimal processing pathways, maintaining or improving processing accuracy despite the elimination of intermediate stages.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
The present disclosure relates to a method of training a speech-to-meaning model, the method comprising: determining a multiplicity of words that may be values of a variable in a phrasing of an intent; determining a multiplicity of parameter sets representative of voices of users of a virtual assistant; synthesizing a multiplicity of speech audio segments for the multiplicity of words, the segments being synthesized according to the multiplicity of parameter sets; and training, using the synthesized speech audio segments, a variable recognizer that is able to compute a probability of the presence of any of the multiplicity of words in speech audio.