Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Face smiling scoring method and face emotion classification method

A smile and face technology, applied in the field of face smile processing, can solve problems such as inaccurate scoring, differences in eye curvature, and unsatisfactory scoring results, and achieve universal applicability, high accuracy, and improved precision.

Pending Publication Date: 2020-06-09
星宏集群有限公司
View PDF8 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

Generally, there are two ways to calculate: (1) Only use the key points of the mouth to calculate: use the ratio of lip height to mouth corner width as the smile score, or directly use the mouth corner width as the smile score. Due to individual differences, the score is not objective enough; (2 ) Calculation based on the key points of the eyes and mouth: Calculate the smile score based on the height-to-width ratio of the lips and the curvature of the eyes. Due to age, genetic and other factors, the curvature of the eyes will vary greatly, and the score is not accurate enough
The unreasonable calculation method of the above-mentioned existing smile scoring scheme leads to unsatisfactory scoring results

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Face smiling scoring method and face emotion classification method
  • Face smiling scoring method and face emotion classification method
  • Face smiling scoring method and face emotion classification method

Examples

Experimental program
Comparison scheme
Effect test

Embodiment approach 1

[0051] The existing technology only uses the width of the mouth corners or the ratio of the height of the lips to the width of the mouth corners as the smile score. Due to individual differences, there will be a large gap, and the calculation method is unreasonable. The calculation formula of the smile score, the method includes the following steps:

[0052] Extract a rectangular frame of the face from the face image, extract the key points of the face including the key points of the mouth from the rectangular frame of the face, and calculate the smile score according to the key points of the mouth;

[0053] The smile score is smileScore,

[0054] In the formula, Dis1=(disL1+...+disLi+...+disLn) / n,

[0055] Dis2=(D1+...+Di+...+Dn) / n;

[0056] Li is the line between the marked point on the upper edge of the upper lip and the corresponding marked point on the lower edge of the lower lip; disLi is the distance from Li, and Di is the distance from the marked point on the lower...

Embodiment approach 2

[0072] In the prior art, the accuracy of emotion label classification needs to be improved. This embodiment provides a face emotion classification method, which redefines the emotion label based on the smile score in Embodiment 1 to improve the accuracy.

[0073] The method includes the following steps:

[0074] Perform emotion recognition and classification on face images, and define emotional labels, including 'happy', 'neutral', 'angry', 'disgust', 'fear', 'sad', 'surprise';

[0075] Redefine the emotion label according to the smile score smileScore, change the smileScore>TH2, and not 'happy' and 'neutral' to 'happy'; TH2 is the default smile score value.

[0076] It should be noted that emotion recognition and classification can be performed based on the hdf5 model trained by Keras. In addition, when performing emotion recognition and classification on face images, the emotion recognition and classification is performed on corrected face images, and the corrected face ima...

Embodiment approach 3

[0079] On the basis of the above redefinition of the emotion label, in order to make the design of the smile score more suitable for practical application, this embodiment provides a face smile scoring method, which integrates the emotion label and whether the mouth is open or not to update the smile score in segments.

[0080] The method specifically includes the following steps:

[0081] Judging whether to open the mouth according to the key points of the mouth;

[0082] If Dis3 / Dis1>TH1, it is judged to open the mouth, otherwise it is closed; TH1 is the default judgment value;

[0083] According to whether the mouth is open or not, the smile score is re-determined in combination with the emotional label;

[0084] Wherein Dis3=(d1+...+dj+...+dm) / m;

[0085] In the formula, dj is the distance between the marked point on the lower edge of the upper lip and the corresponding marked point on the upper edge of the lower lip, and m is an integer greater than or equal to 1.

[0...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a face smiling scoring method and a face emotion classification method, and the method comprises the steps: extracting a face rectangular frame from a face image, extracting face key points comprising mouth key points from the face rectangular frame, and calculating a smiling score according to the mouth key points, wherein the smile score is smileScore, emotion recognitionclassification is carried out on the face image, emotion labels are defined, and the emotion labels comprise 'happy', 'new', 'angry', 'disgust', 'fear', 'sad' and 'surprise'; redefining an emotion label according to the smile score smile Score, and modifying the smile Score which is greater than TH2 and is not 'happy' and 'new' into 'happy', wherein TH2 is a preset smile score value. The formulafor calculating the smile score provided by the invention has universal applicability, conforms to the facial expression change rule of large-sample crowds, and does not have individual difference; the design of redefining the wrong emotion label with the high smiling score, fusing the emotion label and updating the smiling score by opening the mouth in a segmented mode better conforms to subjective judgment of people.

Description

technical field [0001] The invention relates to the field of processing human face smiles, in particular to a method for scoring human face smiles and a method for classifying human face emotions. Background technique [0002] The current smile scoring scheme is mainly based on information mining based on the key points of the face to obtain a smile score. Generally, there are two ways to calculate: (1) Only use the key points of the mouth to calculate: use the ratio of lip height to mouth corner width as the smile score, or directly use the mouth corner width as the smile score. Due to individual differences, the score is not objective enough; (2 ) Calculation based on the key points of the eyes and mouth: Calculate the smile score based on the height-to-width ratio of the lips and the curvature of the eyes. Due to factors such as age and genetics, the curvature of the eyes will vary greatly, and the score is not accurate enough. The unreasonable calculation method of the ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
IPC IPC(8): G06K9/00
CPCG06V40/174G06V40/172G06V40/168
Inventor 冯希宁
Owner 星宏集群有限公司
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com