Multimodal Speech LLM Training With Text and Voice Reward Scoring
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing artificial intelligence-based data generation models for human-machine speech interaction fail to accurately generate response speech data with diverse speech features such as speech rate, timbre, and emotion, leading to insufficient accuracy and quality in speech question-answering interactions.
Innovation Solution
A training method for a multimodal speech language large model that sets reward scores from both text content and speech feature perspectives, optimizing response speech data quality by adjusting model parameters based on these scores, using techniques like reinforcement learning and semantic alignment to enhance speech clarity, rate, and emotional expression.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If existing AI-based data generation models are used for speech interaction, then the basic speech question-answering function can be achieved, but the accuracy and quality of response speech data with diverse speech features (speech rate, timbre, emotion) is insufficient
Solution Approach 1:
The patent applies parameter changes by introducing a reward model that evaluates and optimizes multiple speech parameters including speech rate, timbre, emotion, and clarity. The training process adjusts model parameters based on reward scores derived from comparing generated speech features against reference speech data, enabling precise control over the diversity and quality of speech output characteristics
Solution Approach 2:
The patent implements feedback mechanisms through the reward model that continuously evaluates generated response speech data and provides reward scores based on speech feature comparison. This feedback loop allows the model to iteratively improve its speech generation capabilities by adjusting parameters based on the evaluated performance against reference data
2Manufacturing precision
If reinforcement learning and semantic alignment techniques are applied to optimize speech features, then the quality of response speech data improves, but the training process complexity increases
Solution Approach 1:
The patent applies segmentation by dividing the training process into distinct stages: pre-training the multimodal speech language model, training the reward model separately, and then conducting fine-tuning with the reward model. This modular approach manages complexity by handling different aspects of speech quality optimization in separate, manageable phases rather than simultaneously
Data Source
AI summary
A training method for a multimodal speech language large model is provided. The implementation is: obtaining first response speech data generated by the multimodal speech language large model by inputting first inquiry speech data into the multimodal speech language large model; determining an inquiry text corresponding to the first inquiry speech data and a response text corresponding to the first response speech data; determining, based on the inquiry text and the response text, a first score; determining, based on speech features of the first inquiry speech data and speech features of the first response speech data, a second score, where the speech features include at least one of speech clarity, speech rate feature, timbre feature, intonation feature, and emotion feature; and adjusting, based on the first score and the second score, parameters of the multimodal speech language large model.