A speech large language model translation reinforcement method based on emotion enhancement

Abstract

The application discloses a speech large language model translation reinforcement method based on emotion enhancement and relates to the field of artificial intelligence. The method solves the problem that how to efficiently integrate speech modal information into an existing large language model to make the large language model complete an end-to-end speech automatic translation task. The method comprises the following steps: collecting a multilingual data set containing speech and text pairs of various emotional expressions, and performing preprocessing to form a multilingual corpus pair of original speech and text and translated speech and text; inputting the speech part of the multilingual corpus pair into a speech large language model to make the speech large language model translate; using an emotion matching degree scoring model to evaluate the translation result in step 2, and judging whether the speech emotion in the translation result is consistent with the emotion in the multilingual corpus pair of the original speech and text; using the scoring model to score different results of the same question, performing DPO algorithm according to the scores from high to low, and updating the parameters of the speech large language model.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, specifically to a translation enhancement method for large speech language models based on emotion enhancement. Background Technology

[0002] With the rapid development of deep learning and large language models (LLMs), machine translation technology has made significant progress. Although traditional neural machine translation (NMT) systems perform well in handling plain text translation tasks, the widespread use of multimodal data (such as images, speech, and text) in practical applications has made the effective processing and fusion of multimodal information an important research topic.

[0003] In automatic speech translation tasks, the fusion and processing of speech information is crucial. Current research primarily focuses on enhancing the understanding and generation capabilities of models by extending their architecture and incorporating speech data. For example, researchers have significantly improved the performance of large language models in multimodal tasks by designing specialized pre-training tasks and employing speech expert learning methods. However, recent research has focused on how to efficiently integrate speech modal information into existing decoder-only large language models and activate their potential capabilities in automatic speech translation tasks to enable end-to-end automatic speech translation. Summary of the Invention

[0004] This invention addresses the lack of efficient integration of speech modal information into existing decoder-only large language models, and aims to activate their potential in automatic speech translation tasks, enabling them to complete end-to-end automatic speech translation tasks. This has been a focus of recent research.

[0005] To solve the above-mentioned technical problems, the present invention is achieved through the following technical solution:

[0006] Option 1: This invention proposes a translation enhancement method for large speech language models based on emotion enhancement. The method includes the following steps:

[0007] Step 1: Collect a multilingual dataset containing at least one emotional expression in speech and text pairs; preprocess the multilingual dataset to form multilingual corpus pairs of original speech and text and translated speech and text.

[0008] Step 2: Input the speech portion of the multilingual corpus pairs in the multilingual dataset from Step 1 into the speech big language model, and let the speech big language model translate them.

[0009] Step 3: Use the sentiment matching scoring model to evaluate the translation results in Step 2. If the sentiment of the speech in the translation result in Step 2 is consistent with the sentiment of the original speech and text in the multilingual corpus in Step 1, a positive reward is given; otherwise, a negative reward is given.

[0010] Step 4: Use the scoring model described in Step 3 to score the different results of the same question in Step 2, and sort them from high to low according to the scores to perform the DPO algorithm and update the parameters of the speech large language model.

[0011] Furthermore, a preferred embodiment is provided, wherein the method for preprocessing the multilingual dataset in step 1 includes noise reduction, feature extraction, and language filtering.

[0012] Furthermore, in a preferred embodiment, step 2 further includes the step of designing an emotion-based speech translation template.

[0013] Furthermore, a preferred embodiment is provided in which the emotion-based speech translation template is implemented using an instruction template.

[0014] Furthermore, a preferred embodiment is provided in which the large speech language model described in step 2 is implemented using the Qwen2-audio model.

[0015] Furthermore, a preferred implementation is provided in which the emotional fit scoring model for speech in step 3 is implemented using a Hubert-based emotional classification model.

[0016] Furthermore, a preferred embodiment is provided in which the Hubert emotion classification model includes scores for three dimensions of speech: arousal, dominance, and alertness.

[0017] Furthermore, a preferred embodiment is provided, wherein the DPO algorithm in step 4 is:

[0018]

[0019] Where x is the input speech, y w The option that best conveys emotion in the translation results is y. l The option indicating emotional inconsistency in the translation results is π. θ (y|x) represents the current model's predicted probability for the translation y, π ref σ represents the probability distribution of the reference model, β is a hyperparameter that controls the degree to which the model deviates from the reference model, and σ is the logistic regression function used for binary classification.

[0020] Option 2: A computer storage medium for storing a computer program, wherein when the computer program is read by the computer, the computer executes the method described in any one of Options 1.

[0021] Option 3: A computer device, including a processor and a storage medium, wherein the memory stores a computer program, and when the processor runs the computer program stored in the memory, the processor executes the method described in any one of Options 1.

[0022] The advantages of this invention are:

[0023] The emotion-enhanced large language model translation reinforcement method described in this invention employs an emotion-feedback-based reinforcement learning strategy that includes emotion information quantification technology and emotion-based reward function construction technology, which can improve the emotion transmission ability of large speech models in speech-to-text automatic translation tasks at low cost.

[0024] The method described in this invention can deeply mine emotional information in speech and accurately map it to the translation result, effectively enhancing the performance of large speech models in emotional processing during speech translation, especially when processing speech content rich in emotion. Through emotional information quantification technology, the emotions in speech can be accurately expressed in a computable manner, providing a foundation for subsequent processing. Furthermore, the emotion-based reward function construction technique can reasonably guide the model to optimize in the correct direction of emotion transmission during training, thereby ensuring that the translation result maintains a high degree of emotional consistency with the original speech, greatly improving the naturalness and emotional coherence of the translation.

[0025] This invention is also applicable to the field of automatic speech translation tasks. Attached Figure Description

[0026] Figure 1 This is a block diagram illustrating the overall structure of a speech-based large language model translation enhancement method based on emotion enhancement, as described in Implementation Method 1.

[0027] Figure 2 This is a flowchart of the reinforcement learning process based on emotion feedback in the speech large language model translation enhancement method described in Implementation Method 1.

[0028] Figure 3 This is a schematic diagram of an emotion-based speech translation template in the emotion-enhanced speech large language model translation enhancement method described in Implementation Method 1.

[0029] Figure 4 This is a schematic diagram illustrating the emotion-enhanced translation effect in the emotion-enhanced large language model translation enhancement method described in Implementation Method 1. Detailed Implementation

[0030] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them.

[0031] Implementation Method 1: This implementation method provides a translation enhancement method based on a large speech language model with emotion enhancement. The method includes the following steps:

[0032] Step 1: Collect a multilingual dataset containing at least one emotional expression in speech and text pairs; preprocess the multilingual dataset to form multilingual corpus pairs of original speech and text and translated speech and text.

[0033] Step 2: Input the speech portion of the multilingual corpus pairs in the multilingual dataset from Step 1 into the speech big language model, and let the speech big language model translate them.

[0034] Step 3: Use the model with the highest emotional fit as the scoring model. If the emotional content of the speech in the translation result of Step 2 is consistent with the emotional content of the original speech and text multilingual corpus in Step 1, a positive reward is given; otherwise, a negative reward is given.

[0035] Step 4: Use the scoring model described in Step 3 to score the different results of the same question in Step 2, and sort them from high to low according to the scores to perform the DPO algorithm and update the parameters of the speech large language model.

[0036] Implementation Method 2: This implementation method further defines the speech-based large language model translation enhancement method based on emotion enhancement described in Implementation Method 1. The method for preprocessing the multilingual dataset in step 1 includes noise reduction, feature extraction, and language selection.

[0037] Implementation Method 3: This implementation method further defines the emotion-enhanced speech large language model translation enhancement method described in Implementation Method 1. Step 2 also includes the step of designing an emotion-based speech translation template.

[0038] Implementation Method Four: This implementation method further defines the emotion-enhanced speech large language model translation enhancement method described in Implementation Method Three. The emotion-enhanced speech translation template is implemented using an instruction template.

[0039] Implementation Method 5: This implementation method further defines the translation enhancement method based on the emotion enhancement speech large language model described in Implementation Method 1. The speech large language model mentioned in step 2 is implemented using the Qwen2-audio model.

[0040] The speech language model used in step 2 of this embodiment is not limited to the Qwen2-audio model; existing speech language models can also be used to translate it.

[0041] Implementation Method Six: This implementation method further defines the speech-based large language model translation enhancement method described in Implementation Method One. In step 3, the speech emotion fit scoring model is implemented using a Hubert-based emotion classification model.

[0042] Implementation Method Seven: This implementation method further defines the speech-based large language model translation enhancement method based on emotion enhancement described in Implementation Method Six. The Hubert emotion classification model includes scores for three dimensions: arousal, dominance, and alertness.

[0043] Implementation Method Eight: This implementation method further defines the emotion-enhanced speech-based large language model translation enhancement method described in Implementation Method One. The DPO algorithm in step 4 is as follows:

[0044]

[0045] Where x is the input speech, y w The option that best reflects the emotional consistency in the translation results is y. l The option indicating emotional inconsistency in the translation results is π. θ (y|x) represents the current model's predicted probability for the translation y, π ref σ represents the probability distribution of the reference model, β is a hyperparameter that controls the degree to which the model deviates from the reference model, and σ is the logistic regression function used for binary classification.

[0046] Implementation Method Nine: This implementation method proposes a computer storage medium for storing a computer program. When the computer program is read by a computer, the computer executes the method described in any one of Implementation Methods One to Eight.

[0047] Implementation Method 10: This implementation method provides a computer device, including a processor and a storage medium, wherein a computer program is stored in the memory, and when the processor runs the computer program stored in the memory, the processor executes the method described in any one of Implementation Methods 1 to 8.

[0048] Implementation Method Eleven: This implementation method provides an example, which is used to explain the above-described implementation methods one through ten. The specific example is as follows:

[0049] See Figures 1 to 4This embodiment aims to improve translation quality by incorporating emotional information into speech and text translation using existing emotion classification models. While fine-tuning can be achieved by combining emotional information with instructions, combining the advanced and popular reinforcement learning DPO algorithm with a large speech language model will yield even better results.

[0050] in, Figure 1 This is a block diagram of the speech large language model structure on which this implementation is based. In the reinforcement learning phase, we will mainly train the parameters of the improved large model and the audio projection network.

[0051] In large-scale speech language models, automatic speech translation tasks not only need to focus on semantic accuracy but also require the effective processing and transmission of emotional information within the speech. Emotional information plays a crucial role in speech, directly impacting the naturalness and emotional consistency of the translation. Therefore, reinforcement learning strategies based on emotional feedback will be introduced to further enhance the emotional transmission capabilities of large-scale speech models when handling speech translation tasks.

[0052] Direct Policy Optimization (DPO) is an algorithm used in reinforcement learning to optimize policies. DPO aims to directly optimize policy performance by searching the policy space to find the optimal policy parameters that maximize policy performance in a given environment. The core idea of ​​the DPO algorithm is gradient-based optimization, which adjusts policy parameters by calculating the expected reward (policy gradient) for performing an action in a given state. The sentiment-based DPO optimization process is as follows: Figure 2 As shown.

[0053] We collect multilingual data of speech-text pairs containing various emotional expressions. We preprocess the speech data, including noise reduction, feature extraction, and language filtering, ultimately constructing multilingual corpora of source and translated speech-text pairs.

[0054] Input the speech portion of the corpus from the dataset in step 1 into the speech model and have the speech model translate it. For the same translation question, different results should be generated multiple times so that responses that match the sentiment and responses that do not match the sentiment can be selected later.

[0055] Determine a suitable sentiment relevance scoring model. If the sentiment in the translation result from step 3 matches the sentiment of the original speech, a positive reward is given; otherwise, a negative reward is given. Considering the difficulty in recognizing the sentiment in the output text, the BLEU score should also be taken into account; the higher the BLEU score, the more positive the reward should be. The sentiment relevance scoring model can use GPT-4 for scoring.

[0056] The emotional compatibility scoring model in step 3 is used to score the different results of the same question in step 2, and the DPO algorithm is performed to sort the scores from high to low, thereby updating the model parameters.

[0057] Currently, many large-scale speech and language models rely primarily on Transformer-based encoder models for their audio encoders. However, these models perform poorly in recognizing human emotions within speech. Therefore, this implementation will employ a different approach. Figure 3 The provided speech translation instruction template aims to improve emotion recognition performance. By introducing more flexible instruction templates and enhancing the model's ability to process speech with different emotions, the accuracy and naturalness of the translation are improved.

[0058] This implementation requires a pre-trained speech-language model with excellent multilingual speech understanding capabilities; therefore, we choose Qwen2-audio as the base model. Qwen2-audio possesses powerful multilingual processing capabilities, effectively capturing speech features and semantic relationships in different languages. By adopting this model, we aim to improve the performance of the speech translation system in multilingual scenarios, thereby ensuring the accuracy and fluency of the translation. Furthermore, the Qwen2-audio audio encoder is also a pre-trained encoder based on Whistper. Although the Whistper model performs well in understanding speech content and semantic relationships, it has certain limitations in preserving and understanding emotion. This limitation may affect the accuracy and naturalness of the translation system when processing speech that needs to convey emotional nuances. Therefore, this implementation incorporates the emotional information of the speech into the model to achieve better translation results.

[0059] This implementation will employ the current best-performing Hubert-based sentiment classification model, which scores speech across three dimensions: Arousal, Dominance, and Valence. The sentiment with the highest score will be used as the emotional information for that speech and embedded into the translation instruction template to guide the model in generating different responses. For each translation generated based on this sentiment, a BLEU score will be used to evaluate it against a standard translation. A higher BLEU score indicates better alignment with the original meaning, while a lower BLEU score reflects lower emotional alignment. The responses ranked according to BLEU scores will serve as a reference for the DPO algorithm to optimize model parameters.

[0060] In this embodiment, the DPO algorithm based on sentiment affinity is a simplified reinforcement learning algorithm that avoids the complexity of traditional reinforcement learning by directly optimizing preferences. This method does not require explicit modeling of rewards; instead, it optimizes the policy by maximizing the classification loss function. The core of this algorithm is to increase the probability of the model generating correct outputs and reduce the probability of generating incorrect outputs.

[0061] In this context, the DPO algorithm can be optimized by increasing the probability of translations that maintain sentiment consistency. DPO is optimized using binary cross-entropy loss, resulting in a higher probability of sentiment-consistent translations and a lower probability of sentiment-inconsistent translations. The sentiment-consistency-based DPO loss function formula in dataset D in this implementation is as follows:

[0062]

[0063] Where x is the input speech, y w The option that best reflects the emotional consistency in the translation results is y. l The option indicating emotional inconsistency in the translation results is π. θ (y|x) represents the current model's predicted probability for the translation y, π ref σ represents the probability distribution of the reference model, β is a hyperparameter that controls the degree to which the model deviates from the reference model, and σ is the logistic regression function used for binary classification.

[0064] The goal of this formula is to increase emotional consistency in translation. w The probability of and reduction of emotional inconsistency in translation. l The model generates multiple translations for each speech input, and the DPO (Demographer on the Probability of Emotional Consistency) sorts these translations based on sentiment consistency and adjusts the model parameters to maximize the score of the sentiment-consistent translations.

[0065] This implementation employs an emotion-based reinforcement learning strategy, incorporating emotion information quantification and emotion-based reward function construction techniques. This strategy effectively enhances the emotion transmission capabilities of large speech models in speech-to-text automatic translation tasks at low cost. It deeply mines emotional information in speech and accurately maps it to the translation results, efficiently improving the emotion processing performance of large speech models during speech translation. This is particularly evident when handling speech content rich in emotion. Emotion information quantification accurately expresses the emotions in speech in a computable manner, providing a foundation for subsequent processing. The emotion-based reward function construction technique guides the model to optimize in the correct direction of emotion transmission during training, ensuring that the translation results maintain a high degree of emotional consistency with the original speech, significantly improving the naturalness and emotional coherence of the translation. The final effect is illustrated below. Figure 4 As shown.

[0066] Those skilled in the art will understand that the above description is merely a preferred embodiment of the present invention, and the features described in the various embodiments and / or claims of this disclosure can be combined or combined in various ways, even if such combinations or combinations are not explicitly described in this disclosure. This is not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

[0067] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention. Clearly, those skilled in the art can make various alterations and modifications to the invention without departing from its spirit and scope. Thus, if these modifications and modifications of the invention fall within the scope of the claims and their equivalents, the invention is also intended to include these modifications and modifications.

Claims

1. A speech large language model translation reinforcement method based on emotion enhancement, characterized in that, The method comprises the following steps: Step 1, collecting a multilingual data set containing at least one emotional expression of voice and text pairs, preprocessing the multilingual data set to form a multilingual corpus pair of original voice and text and translated voice and text; Step 2, inputting the voice part of the multilingual corpus pair in the multilingual data set in step 1 into a voice large language model, and allowing the voice large language model to translate it; Step 3, using an emotional fit scoring model to evaluate the translation result in step 2, if the voice emotion in the translation result in step 2 is consistent with the emotion in the multilingual corpus pair of original voice and text in step 1, give a positive reward; If they are not consistent, give a negative reward; Step 4, using the scoring model in step 3 to score different results of the same question in step 2, and sorting according to the score from high to low to perform DPO algorithm, and updating the parameters of the voice large language model.

2. The emotion-enhanced speech large language model translation reinforcement method according to claim 1, wherein, The method for preprocessing the multilingual data set in step 1 includes noise reduction, feature extraction, and language selection. 3.The emotion-enhanced speech large language model translation reinforcement method of claim 1, wherein, Step 2 also includes the step of designing a voice translation template based on emotion. 4.The emotion-enhanced speech large language model translation reinforcement method of claim 1, wherein, The voice translation template based on emotion is implemented by using an instruction template. 5.The emotion-enhanced speech large language model translation reinforcement method of claim 1, wherein, The voice large language model in step 2 is implemented by using a Qwen2-audio model. 6.The emotion-enhanced speech large language model translation reinforcement method of claim 1, wherein, The emotional fit scoring model for voice in step 3 is implemented by using a HuBert-based emotion classification model.

7. The emotion-enhanced speech big language model translation reinforcement method according to claim 6, characterized in that, The HuBert emotion classification model includes scoring of three dimensions of voice Arousal, Dominance and Valence. 8.The emotion-enhanced speech big language model translation reinforcement method of claim 1, wherein, The DPO algorithm in step 4 is: where x is the input speech, is the option of consistent sentiment in the translation result, is the option of inconsistent sentiment in the translation result, represents the prediction probability of the current model for the translation y, represents the probability distribution of the reference model, is a hyperparameter that controls the degree of deviation of the model from the reference model, is a logistic regression function for binary classification.

9. A computer storage medium storing a computer program, characterized in that When the computer program is read by the computer, the computer executes the method of any one of claims 1 to 8. 10.A computer device, comprising a processor and a memory, and characterized in that, The memory stores a computer program, and when the processor runs the computer program stored in the memory, the processor executes the method of any one of claims 1 to 8.

Images