Emotion prediction method and system based on face and sound

A forecasting method and forecasting system technology, applied in the computer field, can solve problems such as low precision and inability to meet the needs of practical applications

Active Publication Date: 2021-05-28
成都视海芯图微电子有限公司
View PDF9 Cites 2 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0003] The technical problem to be solved by the present invention is that the emotion prediction in the prior art usually uses a simple classification algorithm to identify and classify a single emotion, the accuracy is not high, and it cannot meet the needs of practical applications. The purpose is to provide an emotion prediction method based on human face and voice And the system can efficiently and accurately realize the emotion prediction task

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Emotion prediction method and system based on face and sound
  • Emotion prediction method and system based on face and sound
  • Emotion prediction method and system based on face and sound

Examples

Experimental program
Comparison scheme
Effect test

Embodiment

[0042] Such as figure 1 As shown, a kind of emotion prediction method and system based on human face and voice of the present invention specifically comprises the following steps:

[0043] Step S1, collecting face images and voice recording samples;

[0044] In step S2, the face image is input into the convolutional neural network to extract the spatial feature information in the image in a specific form; at the same time, the face image is input into the local binary convolution network to extract the texture feature information of the image, and the face image Spatial feature information and texture feature information are fused to output enhanced face features;

[0045] Step S3, input the speech recording into the preprocessing model to calculate the mel spectrogram of each window; input the mel spectrogram into the voice coding model to model the correlation of short-term mel spectrograms in various ranges, and then pass the speech The segment embedding fuser model outpu...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses an emotion prediction method and system based on faces and sound, and the method comprises the steps: firstly collecting a face image and a voice recording sample, inputting the face image into a convolutional neural network, and extracting the spatial feature information in the image in a specific form; inputting the face image into a local binary convolutional network to extract texture feature information of the image, and fusing the spatial feature information and the texture feature information of the face image to output enhanced face features; then inputting voice recording into a preprocessing model to calculate the Mel spectrogram of each window, inputting the Mel spectrograms into a voice coding model to model the correlation of the short-term Mel spectrograms of each range, and outputting voice features through a voice segment embedding fusion device model; and finally, fusing the face features and the voice features, and inputting the fused features into an output model to predict emotion.

Description

technical field [0001] The invention relates to the field of computers, in particular to an emotion prediction method and system based on human face and voice. Background technique [0002] Emotion prediction is an important research area, which is widely used in various fields, including medical care, security and human-computer interaction, etc. Predicting emotions accurately and quickly is a daunting task due to the complexity of emotions expressed and manifested in different degrees or intensities. At present, emotion prediction mostly uses simple classification algorithms to identify and classify individual emotions, but the accuracy is not high and cannot meet the needs of practical applications. Contents of the invention [0003] The technical problem to be solved by the present invention is that the emotion prediction in the prior art usually uses a simple classification algorithm to identify and classify a single emotion, the accuracy is not high, and it cannot m...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06K9/62G06K9/00G10L25/63G10L25/18G10L25/30
CPCG10L25/63G10L25/18G10L25/30G06V40/168G06V40/174G06F18/24G06F18/253Y02D10/00
Inventor 张旻晋许达文
Owner 成都视海芯图微电子有限公司
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap