Virtual reality video emotion recognition method and system based on time sequence characteristics
What is Al technical title?
Al technical title is built by PatSnap Al team. It summarizes the technical point description of the patent document.
A technology of virtual reality and emotion recognition, applied in neural learning methods, character and pattern recognition, instruments, etc., to achieve the effects of avoiding noise interference, reducing data subjectivity, and reducing individual differences
Pending Publication Date: 2022-06-03
SOUTH CHINA UNIV OF TECH
View PDF0 Cites 0 Cited by
Summary
Abstract
Description
Claims
Application Information
AI Technical Summary
This helps you quickly interpret patents by identifying the three key elements:
Problems solved by technology
Method used
Benefits of technology
Problems solved by technology
[0004] In order to solve the problem of lack of cross-paradigm continuous emotion labeling regression model in the time dimension in the virtual reality scene video emotion recognition level, starting from the establishment of a virtual reality scene audio and video continuous emotion data set, the present invention proposes a virtual reality video emotion recognition based on temporal features method and system
Method used
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more
Image
Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
Click on the blue label to locate the original text in one second.
Reading with bidirectional positioning of images and text.
Smart Image
Examples
Experimental program
Comparison scheme
Effect test
Embodiment 1
[0027] like figure 1 As shown, this embodiment provides a virtual reality video emotion recognition method based on time series features, and the method mainly includes the following steps:
[0028] S1. Establish a virtual reality scene audio and video data set with continuous emotional labels. The content of the data set includes manually extracted continuous emotional labels, audio features, visual features, physiological signal features, etc. (EEG, BVP, GSR, ECG).
[0029] This step builds a virtual reality scene audio and video dataset with continuous emotional labels, such as figure 2 As shown, the specific process includes:
[0030] S11. Collect virtual reality scene videos containing different emotional contents, conduct SAM self-evaluation on the collected N virtual reality scene videos by M healthy subjects, and screen out F virtual reality scenes in each emotional quadrant according to the evaluation scores. scene video.
[0031] S12. Set up a continuous SAM self...
Embodiment 2
[0053] Based on the same inventive concept as Embodiment 1, this embodiment provides a virtual reality video emotion recognition system based on time series features, including:
[0054] The data set establishment module is used to establish a virtual reality scene audio and video data set with continuous emotional labels. The content of the data set includes manually extracted continuous emotional labels, audio features, visual features and physiological signal features;
[0055] The preprocessing module is used for cross-paradigm data preprocessing of the virtual reality scene video to be recognized;
[0056] The feature extraction module performs feature extraction on the preprocessed data, and uses a deep learning network to extract deep features from audio, visual, time series and physiological signals;
[0057] The multi-modal regression model generation and training module trains a single-modal virtual reality scene video emotion regression model, and integrates to gene...
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more
PUM
Login to view more
Abstract
The invention belongs to the field of cross fusion of cognitive psychology, virtual reality technology and emotion continuous recognition, and relates to a virtual reality video emotion recognition method and system based on time sequence characteristics, and the method comprises the steps: building a virtual reality scene audio and video data set with a continuous emotion label; performing cross-normal-form data preprocessing on the virtual reality scene video to be identified; performing feature extraction on the preprocessed data, and extracting depth features from audio, visual, time sequence and physiological signals by using a deep learning network; training a single-mode virtual reality scene video emotion regression model, and fusing, generating and training a multi-mode emotion regression neural network model; and inputting a to-be-recognized virtual reality scene video into the multi-modal emotion regression neural network model, and outputting a continuous emotion regression result. According to the method, a new way can be provided for emotion evaluation of a virtual reality scene video on the basis of multi-modal features of time sequence, vision, audio and physiological signals, and continuous emotion recognition is efficiently and accurately performed.
Description
technical field [0001] The invention belongs to the cross-integration field of cognitive psychology, virtual reality technology and continuous emotion recognition, in particular to a method and system for virtual reality video emotion recognition based on time sequence features. Background technique [0002] Emotion induction and emotion recognition are one of the hotspots in the field of emotion research. They have important applications and research values in the fields of recommendation systems, game design, psychological research, human-computer interaction, and human-computer emotion perception. Virtual reality scenes have been widely used in education, medical care, entertainment, brain-computer interface, etc. due to their high immersion and high sense of substitution. At the same time, they have received extensive attention and research in the field of emotion induction. Continuous emotional evaluation is particularly important. [0003] In the current research on...
Claims
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more
Application Information
Patent Timeline
Application Date:The date an application was filed.
Publication Date:The date a patent or application was officially published.
First Publication Date:The earliest publication date of a patent with the same application number.
Issue Date:Publication date of the patent grant document.
PCT Entry Date:The Entry date of PCT National Phase.
Estimated Expiry Date:The statutory expiry date of a patent right according to the Patent Law, and it is the longest term of protection that the patent right can achieve without the termination of the patent right due to other reasons(Term extension factor has been taken into account ).
Invalid Date:Actual expiry date is based on effective date or publication date of legal transaction data of invalid patent.
Login to view more 
Login to view more