Method, system and computer program product for generating training data for speech emotion recognition models

Abstract

A method for generating training data for speech emotional recognition models is provided. The method includes: receiving prompt data, where the prompt data includes dialogue scenario description information, dialogue participant information, and script content or instructions for automatically generating a script; generating textual dialogue data based on the prompt data by utilizing a large language model, where the textual dialogue data includes at least one utterance and an emotion label corresponding to each utterance; and converting the textual dialogue data into emotional speech dialogue data by utilizing at least one speech model. In addition, a system and a computer program product using the method is also provided.

Claims

1. A method for generating training data for a speech emotion recognition (SER) model, comprising: Receive prompt data, which includes dialogue scene description information and dialogue participant information, wherein the prompt data also includes at least one of script content and instructions for automatically generating script; Based on the prompt data as input to a large language model, use the large language model to generate text dialogue data, which includes at least one statement and an emotion tag corresponding to each statement. And using at least one speech model to convert the text dialogue data into speech dialogue data with emotion, including: using a first speech model to convert the text dialogue data into speech dialogue data without emotion. And using a second speech model, based on the emotion marker in the text dialogue data, the speech dialogue data without emotion is converted into speech dialogue data with emotion.

2. The method as described in request item 1, wherein the prompt data further includes dialog content distribution characteristics.

3. The method as described in claim 1, wherein the emotion tag includes: One of the following: happiness, neutrality, sadness, frustration, anger, surprise, fear, disgust, and contempt.

4. A system for generating training data for a speech emotion recognition (SER) model, the system comprising: Memory stores at least one instruction; A processor, coupled to the memory, wherein when the processor executes the at least one instruction, it is configured to: receive cue data, the cue data including dialogue scenario description information and dialogue participant information, wherein the cue data also includes at least one of script content and instructions for automatically generating a script; and, based on the cue data as input to a large language model, generate text dialogue data using the large language model, the text dialogue data including at least one statement and an emotion tag corresponding to each statement. And using at least one speech model to convert the text dialogue data into speech dialogue data with emotion, including: using a first speech model to convert the text dialogue data into speech dialogue data without emotion. And by using a second speech model, based on the emotion marker in the text dialogue data, the speech dialogue data without emotion is converted into speech dialogue data with emotion.

5. The system as described in claim 4, wherein the prompt information further includes dialog content distribution characteristics.

6. The system as described in claim 4, wherein the emotion tag includes: One of the following: happiness, neutrality, sadness, frustration, anger, surprise, fear, disgust, and contempt.

7. A computer program product comprising at least one instruction, which, when executed by a processor of an electronic device, enables the electronic device to perform the method as described in any one of claims 1 to 3.

Images