Speech Recognition Model Training with Attribute-Guided Expert Assignment
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current speech recognition models based on Mixture-Of-Experts (MoE) suffer from high training costs and lower accuracy due to unsupervised sample assignment to experts, leading to inefficient feature extraction and recognition.
Innovation Solution
A supervised training method for speech recognition models that classifies speech samples under target attributes, using encoding and decoding networks to adjust network parameters based on attribute-specific losses, and incorporates shared expert layers to improve fitting ability.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If unsupervised training with random sample assignment to experts is used, then the training process is simpler to implement, but the training cost becomes very high and recognition accuracy decreases
Solution Approach 1:
The patent applies preliminary action by pre-classifying speech samples into different attribute categories before training. The system pre-divides the training data into distinct groups based on speech attributes (such as speaker identity, speech content, or acoustic characteristics), and then assigns these pre-classified samples to corresponding expert networks. This preliminary organization of data enables supervised training where each expert receives appropriately labeled samples, thereby improving recognition accuracy while maintaining implementation simplicity through automated classification algorithms.
2Ease of manufacture
If unsupervised training with random sample assignment is used, then the training process requires less human annotation, but the number of samples and experts required becomes very large
Solution Approach 1:
The patent applies segmentation by dividing the training data into distinct attribute-based segments and assigning each segment to a dedicated expert network. Instead of randomly distributing all samples across a large number of experts, the system segments the data by speech attributes (e.g., speaker categories, content types, acoustic features) and creates corresponding expert networks for each segment. This segmentation approach reduces the total number of experts needed while improving the quality of training for each expert, as each expert receives only the samples relevant to its designated attribute category.
Solution Approach 2:
The patent applies local quality by ensuring that each expert network receives training samples with specific local characteristics matching its designated attribute category. Rather than treating all samples uniformly, the system assigns samples with particular local qualities (such as specific speaker identities, speech content types, or acoustic characteristics) to corresponding experts. This localized training approach improves the efficiency of each expert and reduces the overall quantity of samples and experts needed, as each expert only processes the subset of samples relevant to its specialized function.
3Extent of automation
If random sample assignment to experts is used, then the training process is more automated, but the feature accuracy rate becomes lower
Solution Approach 1:
The patent applies feedback by implementing a supervised training framework where the system uses ground truth labels from pre-classified speech samples to guide the training process. The feedback mechanism operates through automated comparison of predicted attribute categories with actual labels, allowing the system to adjust and optimize expert network parameters. This feedback loop enables automated training while maintaining high feature accuracy, as the system continuously refines expert performances based on measured deviations from correct classifications, thereby resolving the contradiction between automation and precision.
Data Source
AI summary
A speech recognition method is provided. The method includes: obtaining a to-be-recognized speech and a speech recognition model, including an encoding network and a decoding network, after training; during each stage of encoding the to-be-recognized speech using the encoding network, classifying the to-be-recognized speech under a target speech attribute to obtain a predicted attribute category, and performing encoding to obtain a first encoding feature according to the predicted attribute category under the target speech attribute; decoding the first encoding feature according to the decoding network to obtain a recognition text of the to-be-recognized speech, the speech recognition model being adjusted according to at least a first loss, which represents a difference between a preset attribute category annotated in a speech sample and a sample attribute category recognized and obtained by the speech recognition model under the target speech attribute.