Speech Recognizer Using Shared Exclusive Attributes
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional speech recognition technologies struggle to effectively utilize text data and lack robustness against noise, speaker variations, and channel information, leading to limitations in speech processing performance.
Innovation Solution
A speech recognizer based on shared and exclusive attributes is developed, which includes a method of training using a parallel speech corpus and a non-parallel speech corpus, along with a parallel text corpus and a non-parallel text corpus. This approach allows for the separation of attributes through a graph structure, enabling effective representation learning of speech-text multimodal information.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional speech recognition models are trained only on speech data, then speech feature extraction capability is improved, but text data utilization is insufficient and robustness against noise and speaker variations deteriorates
Solution Approach 1:
The patent combines speech data and text data into a unified training framework, merging previously separate training processes into a single multimodal model. The speech encoder and text encoder are trained together with shared and separate loss functions, enabling the model to leverage both speech and text data simultaneously, thereby improving robustness while maintaining speech feature extraction capability.
Solution Approach 2:
The patent segments the representation learning into shared attributes and separate attributes. The speech encoder outputs both shared representation (useful for both speech and text) and separate representation (specific to speech). This segmentation allows the model to utilize text data for robustness while preserving speech-specific features through the separate representation branch.
2Device complexity
If speech characteristics are not processed based on speaker, noise, and channel information, then model simplicity is maintained, but robustness against noise and speaker variations deteriorates
Solution Approach 1:
The patent segments speech characteristics into speaker-specific features and content features. The model architecture separates these through different processing paths, allowing speaker information to be extracted and utilized without complicating the overall model structure. This enables robustness improvement while maintaining relative simplicity through modular design.
Solution Approach 2:
The patent creates a universal model that handles multiple functions: speech recognition, speaker verification, and noise robustness. By designing the speech encoder to extract both content and speaker information simultaneously, the model achieves multi-functionality without requiring separate specialized models, thus maintaining simplicity while improving robustness.
3Device complexity
If optimization method for interaction between speech data and text data is not implemented, then training process simplicity is maintained, but speech processing performance deteriorates
Solution Approach 1:
The patent segments the training objective into shared loss and separate loss components. The shared loss optimizes the common representation space for both speech and text, while the separate loss optimizes speech-specific features. This segmented approach provides a systematic optimization method that improves speech processing performance while keeping the training process structured and manageable.
Data Source
AI summary
Provided is a method of training a speech recognizer based on shared and exclusive attributes. The method includes: inputting a parallel speech corpus constituting a labeled speech corpus and a non-parallel speech corpus into a speech encoder constituting a speech recognizer; outputting a representation vector representing training speech as an output of the speech encoder; inputting a parallel text corpus constituting the labeled speech corpus and a non-parallel text corpus into a text encoder; outputting a representation vector representing text as an output of the text encoder; and receiving and decoding, by a decoder, each of the representation vectors of the speech encoder and the text encoder.