Unlock instant, AI-driven research and patent intelligence for your innovation.

Covariance measure driven small sample GIF short video emotion recognition method and system

A technology of emotion recognition and short video, which is applied in character and pattern recognition, instruments, biological neural network models, etc. It can solve problems such as limited data sets, limited samples, and difficulties in learning emotional information, and achieve the effect of improving accuracy

Active Publication Date: 2019-12-03
NANJING UNIV OF POSTS & TELECOMM
View PDF8 Cites 9 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

In the era of social media, images and video information have become the mainstream way of communication for people. However, the emotion recognition of short videos is difficult due to the difficulty of learning emotional information from video sequences and the existing data sets for short video emotion analysis are limited and the number of samples is limited. also limited

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Covariance measure driven small sample GIF short video emotion recognition method and system
  • Covariance measure driven small sample GIF short video emotion recognition method and system
  • Covariance measure driven small sample GIF short video emotion recognition method and system

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0048] The technical scheme of the present invention is described in detail below in conjunction with accompanying drawing:

[0049] Such as figure 1 As shown, the embodiment of the present invention discloses a covariance metric-driven small-sample GIF short video emotion recognition method. This method first randomly extracts category C from the N categories of data to be trained as a support set (S for short), and takes K (K ≥ 1) short video samples of GIF from category C for each training, and uses the support set In the remaining data sets, M (M≥1) GIF short video samples are randomly selected from each category as the query set (Q for short) to be queried by the model. Then a 3D convolutional neural network is used to extract sequence features in the short videos of category C of the supporting dataset. The short video of the sample to be queried is simultaneously input into the 3D convolutional neural network for feature extraction, and then a covariance measurement m...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a covariance measurement driven small sample GIF short video emotion recognition method and system. The method comprises the following steps: firstly, extracting samples from adata set to form a support set and a query set, and segmenting the extracted samples into a plurality of candidate frame images according to a set frame rate; extracting sequence features in the sample by using C3D, extracting visual features of frame images by using CNN, performing Hadamard inner product operation frame by frame to obtain final image visual features, and extracting high-level semantic features after fusion of the sequence features and the visual features by using a ConvLSTM technology; splicing the features extracted from the support set sample and the features extracted from the query set sample, measuring the similarity between the query set sample and each type of emotion of the support set sample through a covariance measurement module, and finally predicting a finalresult through a softmax classifier. The emotion information of the image in the GIF can be effectively concerned, the time sequence characteristics of the video stream are considered, the method issuitable for multi-classification video emotion analysis under the condition of small samples, and the classification accuracy is improved.

Description

technical field [0001] The invention relates to a covariance measurement-driven small-sample GIF short video emotion recognition method and system, belonging to the technical field of computer vision video emotion recognition. Background technique [0002] In recent years, with the enrichment of social multimedia content, visual sentiment analysis has gradually become a research hotspot. According to the different content forms of visual research, visual sentiment analysis can be divided into two categories: image sentiment analysis and video sentiment analysis. [0003] In the sentiment analysis of images, literature [S.Siersdorfer, E.Minack, F.Deng, and J.S.Hare, "Analyzing and predicting sentiment of images on the social web." InACM MM, pages 715–718, 2010. ] proposed to analyze the emotion of images from the level of pixel-level features, however, due to the complexity of emotions, the underlying features cannot express the emotional meaning of images very well. With th...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Applications(China)
IPC IPC(8): G06K9/00G06K9/62G06N3/04
CPCG06V20/40G06V20/46G06N3/045G06F18/214G06F18/2415
Inventor 刘天亮陈敏戴修斌周亮
Owner NANJING UNIV OF POSTS & TELECOMM