Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Facial paralysis detection method based on visual perception and audio information

A technology of audio information and visual perception, applied in speech analysis, character and pattern recognition, instruments, etc., can solve the problem that the detection and diagnosis of facial nerve palsy disease cannot be quickly and easily simplified.

Inactive Publication Date: 2021-02-02
郑州苏一电子科技有限公司
View PDF0 Cites 2 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0004] The present invention overcomes the problem that the detection and diagnosis of facial nerve paralysis in the prior art cannot be quickly simplified, and provides a facial paralysis detection method based on visual perception and audio information

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Facial paralysis detection method based on visual perception and audio information
  • Facial paralysis detection method based on visual perception and audio information
  • Facial paralysis detection method based on visual perception and audio information

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0038] The facial paralysis detection method based on visual perception and audio information of the present invention will be further described below in conjunction with the accompanying drawings and specific embodiments: As shown in Figure (1), the present embodiment contains the following steps: Step 1: deploy RGBD above a display screen The camera collects the face RGB image and the depth image (Depth image) of the person to be tested. The camera angle is facing straight, and the field of view covers the entire area of ​​the face. At the same time, the camera can collect the audio information of the person to be tested.

[0039] Step 2: Use the RGB image to obtain the RGB image I of the face area 1 and the depth map I 2 ;

[0040] The purpose of this step is to eliminate the influence of the background on the face area detection and isolate irrelevant working conditions. The specific operation method is: send the collected RGB image into the semantic segmentation networ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a facial paralysis detection method based on visual perception and audio information. The problem that detection and diagnosis of facial nerve paralysis diseases cannot be rapidly simplified is solved. The method comprises the following steps: collecting RGB and Depth images; obtaining an RGB image I1 and a depth image I2 of the face area; obtaining 2D key points and 3D keypoints of the face; guiding a to-be-tested person to read characters; specifying a characteristic value reflecting mouth movement to obtain a characteristic sequence of the mouth movement; collectingaudio information, performing framing and windowing operation, extracting a Mel-frequency cepstrum coefficient of each frame, and correcting a mouth motion sequence; performing feature extraction onthe corrected mouth motion sequence Cnew to obtain a motion similarity index S; analyzing the frequency distribution of the audio to obtain a speaking clearness degree D; and integrating S and D to obtain a facial paralysis detection result. According to the technology, the characteristic sequence of mouth movement is corrected by using the audio information, so that the accuracy is higher.

Description

technical field [0001] The invention relates to the technical field of machine vision image processing, in particular to a facial paralysis detection method based on visual perception and audio information. Background technique [0002] Facial paralysis is a disease characterized by facial expression muscle motor dysfunction, with general symptoms of crooked mouth and eyes and uncoordinated facial expressions. Because of the inability to control facial muscles, people with facial paralysis often have limited facial movement and other symptoms, including drooling, speech difficulties, and nasal congestion, among others. [0003] In the prior art, the detection of facial paralysis usually requires the person to be tested to perform specified actions, and the detection of facial paralysis is realized by observing the situation of the personnel completing the specified actions. However, when people deliberately complete the specified movements, they have a strong subjective con...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Applications(China)
IPC IPC(8): G06K9/00G06K9/34G10L25/24G10L25/66
CPCG10L25/24G10L25/66G06V40/165G06V40/171G06V10/267
Inventor 陈永宁袁梦刘亚刚
Owner 郑州苏一电子科技有限公司
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com