Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Method and device for automatic generation of emotional speech based on generative confrontation network

An automatic generation and generative technology, applied in the field of emotion recognition, which can solve the problems of poor voice expressiveness, unnaturalness, and inability to synthesize voice according to the specified identity.

Active Publication Date: 2022-03-08
ZHEJIANG UNIV OF TECH
View PDF6 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0005] Aiming at the defect that the speech synthesized by the current speech generation technology is poor in expressiveness, unnatural, without emotional color, and cannot be synthesized according to the specified identity, the present invention provides a method and device for automatically generating emotional speech based on generative confrontation network , this method can make the generated speech more natural, make the generated speech with emotional color identity information, etc., and expand the application scenarios of speech generation technology

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Method and device for automatic generation of emotional speech based on generative confrontation network
  • Method and device for automatic generation of emotional speech based on generative confrontation network
  • Method and device for automatic generation of emotional speech based on generative confrontation network

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0043] In order to make the object, technical solution and advantages of the present invention clearer, the present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described here are only used to explain the present invention, and do not limit the protection scope of the present invention.

[0044] A kind of embodiment of the method for automatically generating emotional speech based on generative confrontation network provided by the present invention is introduced below, see figure 1 ,2, specifically including the following steps:

[0045] 1) Data set preparation: Prepare the Librispeech voice data set for training the voiceprint recognition model, take the train-clean-100 data set in the Librispeech voice data set to train the voiceprint recognition model; prepare the EMO for training the voice emotion recognition model -DB Speech Emotion Dataset, North ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a method for automatically generating emotional speech by a generative confrontation network, comprising: (1) preparing a speech data set, a speech emotion data set and a language data set; (2) using the speech data set to train a voiceprint based on ResCNN Recognition model, using speech emotion datasets to train CNN-based speech emotion recognition models, using language datasets to train speech generation models; (3) Using multiple speech generation models as generators, using voiceprint recognition models and speech emotion recognition models As a discriminator, a generative adversarial network is formed, and the generative adversarial network is retrained using speech data sets, speech emotion data sets, and language data sets to obtain a speech generation model that can generate specific identity-specific emotional speech. (4) Using the speech generation model to automatically generate emotional speech. This method can make the generated speech more natural and with emotional identity information.

Description

technical field [0001] The invention belongs to the field of emotion recognition, and in particular relates to a method and device for automatically generating emotional speech by a generative confrontation network. Background technique [0002] With the increasing update and development of human-computer interaction methods, human-computer interaction methods have entered the era of multimedia user interface from the mechanical age. In recent years, due to the development of speech recognition technology and speech generation technology, people have gradually abandoned traditional ways of interacting with machines such as keyboards, mice, and touch screens. And computers also have the ability to "listen" and "speak" like people. "Listening" is actually speech recognition technology, which has developed rapidly and greatly improved the ability of computers to "listen". "Say" refers to speech generation technology. Speech generation technology has been greatly developed un...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Patents(China)
IPC IPC(8): G10L13/02G10L13/033G10L17/00G10L17/04G10L17/06G10L17/18G10L19/02G10L25/30G10L25/63
CPCG10L13/02G10L13/033G10L17/04G10L17/06G10L17/18G10L19/0212G10L25/30G10L25/63
Inventor 陈晋音叶林辉郑海斌
Owner ZHEJIANG UNIV OF TECH
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products