Optimal human-machine conversations using emotion-enhanced natural speech using hierarchical neural networks and reinforcement learning

Abstract

A system and method for emotion-enhanced natural speech using dilated convolutional neural networks, wherein an audio processing server receives a raw audio waveform from a dilated convolutional artificial neural network, associates text-based emotion content markers with portions of the raw audio waveform to produce an emotion-enhanced audio waveform, and provides the emotion-enhanced audio waveform to the dilated convolutional artificial neural network for use as a new input data set.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS[0001]This application claims priority to U.S. provisional patent application Ser. No. 62 / 516,672, titled “OPTIMAL HUMAN-MACHINE CONVERSATIONS USING EMOTION-ENHANCED NATURAL SPEECH USING HIERARCHICAL NEURAL NETWORKS AND REINFORCEMENT LEARNING”, which was filed on Jun. 8, 2016, and is also a continuation-in-part of U.S. patent application Ser. No. 15 / 442,667 titled “SYSTEM AND METHOD FOR OPTIMIZING COMMUNICATION OPERATIONS USING REINFORCEMENT LEARNING”, filed on Feb. 25, 2017, which claims priority to U.S. provisional patent application 62 / 441,538, titled “SYSTEM AND METHOD FOR OPTIMIZING COMMUNICATION OPERATIONS USING REINFORCEMENT LEARNING”, filed on Jan. 2, 2017, and is also a continuation-in-part of U.S. patent application Ser. No. 15 / 268,611, titled “SYSTEM AND METHOD FOR OPTIMIZING COMMUNICATIONS USING REINFORCEMENT LEARNING”, filed on Sep. 18, 2016, the entire specifications of each of which are incorporated herein by reference.BACKGROUND...

AI Technical Summary

Benefits of technology

The patent describes a system and method for generating emotion-enhanced natural speech using hierarchical neural networks and an optimization component for choosing content and emotion level in conversations. The system includes an audio processing server that receives a raw audio waveform from a neural network and produces an emotion-enhanced audio waveform by associating text-based emotion content markers with portions of the audio waveform. The server then provides the emotion-enhanced audio waveform as an input data set to a dilated convolutional artificial neural network for use in natural speech audio generation. The technical effect of this invention is a more advanced and efficient way to generate emotion-enhanced natural speech audio.

Problems solved by technology

Recognizing human emotion in transcribed or recorded speech is a difficult task for computer programs, and producing convincing emotion in text-to-speech is often a labor-intensive process involving manual configuration and emotion-tagging.

Images