A natural interaction method of virtual learning environment based on speech emotion recognition

A speech emotion recognition and learning environment technology, applied in the field of deep learning, can solve problems such as the inability to find emotion description features, and achieve the effects of strengthening descriptiveness, promoting improvement, and improving representational performance

Inactive Publication Date: 2019-01-04
CHONGQING UNIV OF POSTS & TELECOMM
View PDF5 Cites 20 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

Therefore, it is not possible to find a complete emotional description feature by considering only the short-term energy features in the acoustic features.

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • A natural interaction method of virtual learning environment based on speech emotion recognition
  • A natural interaction method of virtual learning environment based on speech emotion recognition
  • A natural interaction method of virtual learning environment based on speech emotion recognition

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0043] The preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0044] like figure 1 As shown, a natural interaction method for a virtual learning environment based on speech emotion recognition includes the following steps:

[0045] Step 101: Carry out resampling to the voice signal of the student user collected by kinect in real time, divide into frames and add windows, mute processing, obtain a short-term single-frame signal, and the n-th frame signal is x(n);

[0046] Step 102: Perform fast Fourier transform on x(n) to obtain frequency domain data, find its power spectrum, and use Mel filter bank to obtain the Mel spectrogram of the frame;

[0047] like image 3 As shown, step 103: input the obtained mel spectrogram features into the built convolutional neural network, and perform convolution operation, each filter of the convolution layer acts on a mel spectrogram, using convolution The characte...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention relates to a natural interactive method of a virtual learning environment based on speech emotion recognition, belonging to the field of depth learning. The method comprises the following steps: 1, collecting speech signals of students and users through kinect, resampling, adding windows by frames, and mute processing to obtain short-time single frame signals; 2, carrying out fast Fourier transform on that signal to obtain the frequency domain data, obtaining the pow spectrum thereof, and adopting a Mel filter bank to obtain a Mel spectrum diagram; 3, inputting the features of the Mel spectrum map into a convolution neural network, performing convolution operation and pooling operation, and inputting the matrix vectors of the last desample layer to the whole connecting layerto form a vector output feature; 4, compressing and inputting the output characteristic into a bi-directional long-short time memory neural network; 5, inputting the output features into a support vector machine to classify and output a classification result; 6, feeding back the classification result to the virtual learning system for virtual learning environment interaction. The invention driveslearners to adjust the learning state and enhances the practicability of the virtual learning environment.

Description

technical field [0001] The invention belongs to the field of deep learning and relates to a natural interaction method for a virtual learning environment based on speech emotion recognition. Background technique [0002] Speech signal is the most convenient, fastest and most natural way of communication for human beings, and it carries a lot of emotional information. Therefore, the analysis and research of speech emotion recognition is of great significance in the field of human-computer interaction. For example, in the teacher's distance education class for students, this technology can be used to help teachers analyze the emotional state of students answering questions, correlate and predict the learning state of students at this time, and then adjust their classroom teaching so that classroom teaching can truly achieve People-oriented; In terms of medical treatment, doctors can monitor the emotional state of patients in the ward remotely and in real time. When they are i...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06N3/04G06N3/08G10L25/63G10L25/30
CPCG06N3/08G10L25/30G10L25/63G06N3/048G06N3/044G06N3/045
Inventor 蔡林沁陈富丽陆相羽胡雅心
Owner CHONGQING UNIV OF POSTS & TELECOMM
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap