Deepfake detection method based on emotion recognition and pupil size calculation
A pupil size and emotion recognition technology, applied in computing, speech recognition, computer components, etc., can solve the problems of insufficient application scenarios and lack of generalization ability, and achieve the effect of strong generalization ability and wide application range
Active Publication Date: 2021-04-23
ZHEJIANG UNIV OF TECH
View PDF6 Cites 0 Cited by
- Summary
- Abstract
- Description
- Claims
- Application Information
AI Technical Summary
Problems solved by technology
[0007] The present invention provides a deepfake detection method based on emotion recognition and pupil size calculation, which can overcome the lack of comprehensive application scenarios of existing deepfake detection technology, and often cause overfitting of certain deepfake methods and lack of generalization ability question
Method used
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View moreImage
Smart Image Click on the blue labels to locate them in the text.
Smart ImageViewing Examples
Examples
Experimental program
Comparison scheme
Effect test
Embodiment Construction
[0042] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be noted that the following embodiments are intended to facilitate the understanding of the present invention, but do not limit it in any way.
[0043] Such as figure 1 As shown, a Deepfake detection method based on emotion recognition and pupil size calculation, including:
[0044] Step 1, Data Processing
[0045] (1-1) Data set
[0046] The CASIA Chinese emotion corpus is used as the training data set of the speech recognition model Y. The CASIA Chinese emotion corpus is recorded by the Institute of Automation, Chinese Academy of Sciences (Institute of Automation, Chinese Academy of Sciences). It includes four professional speakers, six emotions angry (angry ), happy (happy), fear (fear), sad (sad), surprise (surprise) and neutral (neutral), a total of 9600 sentences with different pronunciations. Among them, 300 sentences are in th...
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More PUM
Login to View More Abstract
The invention discloses a Deepfake detection method based on emotion recognition and pupil size calculation, and the method comprises the steps: (1) dividing voice data into a training set X and a test set Q, carrying out data processing, and carrying out the training and testing of a training voice recognition model Y; (2) dividing the text data into a training set N and a test set P, performing data processing, and training and testing a training text sentiment classification model M; (3) for a Deepfake video to be detected, extracting an audio, inputting the audio into the speech recognition model Y, and inputting an output text into the text emotion classification model M to obtain emotion corresponding to the text; (4) converting the Deepfake video to be detected into picture frames, and detecting the sizes of pupils of human eyes; and (5) matching the detected pupil size of the human eye with the emotion obtained by the text emotion classification model M, and if the detected pupil size of the human eye is not matched with the emotion obtained by the text emotion classification model M, determining that the video is false. False videos generated by different Deepfake methods can be well detected, and the generalization ability is high.
Description
technical field [0001] The invention belongs to the technical field of machine learning, and in particular relates to a deepfake detection method based on emotion recognition and pupil size calculation. Background technique [0002] Speech recognition technology is to allow computers to understand what people are saying, to realize voice communication between humans and machines, and to output human words in the form of text. In recent years, speech recognition technology has made remarkable progress, and it has begun to enter everyone's life from the laboratory, such as voice assistants and voice translation in smartphones. Commonly used methods in speech recognition technology include stochastic model method, probabilistic syntax analysis, methods based on linguistics and acoustics, and methods using artificial neural networks, among which the most common method is stochastic model method. [0003] For example, the Chinese patent document whose publication number is CN106...
Claims
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More Application Information
Patent Timeline
Login to View More Patent Type & Authority Applications(China)
IPC IPC(8): G06F16/35G10L15/06G10L15/16G10L25/63G10L25/24G06T7/00G06K9/00G06K9/62
CPCG06F16/353G10L15/063G10L15/16G10L25/63G10L25/24G06T7/0002G10L2015/0631G06T2207/30201G06T2207/10016G06V40/161G06V40/18G06V20/40G06F18/22
Inventor 刘毅王鹏程陈晋音
Owner ZHEJIANG UNIV OF TECH