Prompt-Guided Speech Recognition Using Feature Sequence Conversion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing speech recognition models based on end-to-end neural networks are limited by model capacity and training methods, leading to suboptimal recognition accuracy.
Innovation Solution
A speech recognition scheme that processes target speech content using a speech encoding unit to generate a speech encoding representation, converts it into a speech feature sequence, constructs an input feature sequence with a prompt feature sequence, and processes it with a language model to generate a recognition result.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If end-to-end neural network models are used for speech recognition, then the system can achieve automated processing, but the model capacity and training methods are limited leading to suboptimal recognition accuracy
Solution Approach 1:
The speech recognition system is divided into multiple independent modules: speech encoding unit, conversion unit, and language model. Each module processes specific aspects of speech recognition independently, allowing optimization of each component without increasing overall model complexity. This segmentation enables the system to achieve high accuracy through coordinated module interactions rather than relying on a single large end-to-end model.
Solution Approach 2:
The patent introduces intermediate representations (speech encoding representations and speech feature sequences) that bridge the gap between raw speech input and language model processing. These intermediaries enable efficient information transformation and facilitate the integration of speech processing capabilities with language understanding, thereby improving recognition accuracy without requiring increased model capacity.
2Measurement precision
If traditional speech recognition models are used, then the system structure is simpler, but the recognition accuracy is suboptimal due to model capacity limitations
Solution Approach 1:
By segmenting the recognition system into specialized modules (speech encoding, feature conversion, language modeling), each module can be optimized for its specific function. This allows parallel processing and independent optimization, maintaining high processing efficiency while achieving superior accuracy through the coordinated output of multiple specialized components.
Solution Approach 2:
The speech encoding unit and conversion unit serve multiple functions: they process speech input, extract relevant features, transform representations, and prepare data for language model processing. This multi-functionality reduces the need for separate specialized components, maintaining processing efficiency while enabling accurate recognition through versatile intermediate processing stages.
Data Source
AI summary
Embodiments of the disclosure relates to a method, apparatus, device and storage medium for speech recognition. An example method includes: obtaining target speech content; processing, with a speech encoding unit, the target speech content to generate a speech encoding representation; converting, with a conversion unit, the speech encoding representation into a speech feature sequence; constructing an input feature sequence based on the speech feature sequence and a prompt feature sequence, the prompt feature sequence being constructed based on a predetermined prompt item; and processing the input feature sequence with a language model to generate a speech recognition result of the target speech content. The embodiments of the disclosure can implement, with a language model, speech recognition based on a feature sequence.