NLU Training With Phonetic Text Variants for Audio Input
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
NLU engines trained on clean text data struggle with misalignment and performance degradation when deployed in audio-based applications due to differences in phonetic representations and ASR errors, leading to inaccurate understanding of audio input.
Innovation Solution
Generate phonetically-related text portions from text samples and train NLU engines using these portions to enhance accuracy in audio-based applications, accounting for distinctions between text and audio inputs.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If NLU engines are trained on clean text data, then training simplicity and data availability are improved, but performance accuracy in audio-based applications deteriorates
Solution Approach 1:
The patent creates phonetically-related text portions by generating phonetic transcriptions of audio data and converting them back to text form. These phonetic copies simulate ASR errors and audio-specific variations while maintaining the original semantic meaning, allowing the NLU engine to be trained on text-based representations that capture audio characteristics without requiring actual audio processing infrastructure during training.
Solution Approach 2:
The patent transforms the training data parameters by introducing phonetic variations and speech disfluencies into the text training corpus. By modifying the text representations to include phonetically-related alternatives and simulated ASR errors, the training data better reflects the characteristics of audio-based input while maintaining text-based training simplicity.
2Device complexity
If NLU engines are trained on clean text data, then data processing complexity is reduced, but semantic alignment with audio input deteriorates
Solution Approach 1:
The patent generates phonetic transcriptions of audio data and converts them back to text form, creating phonetically-related text portions that serve as intermediate representations. These copies bridge the semantic gap between clean text and audio input by incorporating phonetic variations, speech disfluencies, and ASR-like errors while maintaining text-based processing simplicity.
Solution Approach 2:
The phonetically-related text portions act as an intermediary layer between clean text training data and audio-based input. This intermediate representation captures the phonetic and pragmatic characteristics of spoken language while remaining in text form, allowing the NLU engine to learn semantic alignment with audio input without directly processing audio signals during training.
3Reliability
If NLU engines are trained on phonetically-related text portions, then robustness to ASR errors is improved, but training data generation complexity increases
Solution Approach 1:
The patent performs phonetic transcription and text regeneration in advance during the offline training phase. By pre-generating phonetically-related text portions and incorporating them into the training corpus before deployment, the system prepares the NLU engine to handle ASR errors and audio-specific variations without incurring computational complexity during real-time operation.
Solution Approach 2:
The patent creates phonetic copies of audio data and converts them to text form, generating a augmented training corpus that includes phonetically-related variations. This copying process captures ASR-like errors and speech disfluencies in advance, allowing the NLU engine to learn robust semantic representations without requiring complex real-time audio processing or error correction mechanisms.
Data Source
AI summary
A method, computer program product, and computing system for generating a plurality of potential vocalizations of a plurality of text samples. A plurality of phonemes associated with the plurality of potential vocalizations are identified. A plurality of phonetically-related text portions are generated based upon, at least in part, the plurality of phonemes. A natural language understanding (NLU) engine is trained using the plurality of phonetically-related text portions.