Automatic lip language identification system suitable for Chinese language

Abstract

The invention relates to an automatic lip language identification system suitable for Chinese language, comprising a wear-type camera, a man-machine interaction module, a lip contour positioning module, a geometric vector acquisition module, a motion vector acquisition module, a characteristic matrix building module, a transformation matrix T acquisition module, a conversion characteristic matrix acquisition module, a memory A, a memory B and a canonical correlation discriminatory analysis module. The wear-type camera is used for recording Chinese character sound image sequences, transmitting the Chinese character sound image sequences to the lip contour positioning module through the man-machine interaction module, and detecting and tracking lip contours by utilizing a convolution virtual electrostatic field Snake module; the geometric vector acquisition module and the motion vector acquisition module respectively extract geometric and motion characteristics from the lip contours and join up the geometric and motion characteristics as an input characteristic matrix of the canonical correlation discriminatory analysis module; and the canonical correlation discriminatory analysis module calculates the similarity among the characteristic matrixes and acquires identification results after processing. Compared with the traditional lip language identification systems, the system has higher identification accuracy.

Description

technical field [0001] The invention relates to an automatic lip recognition system, in particular to an automatic lip recognition system suitable for Chinese, and belongs to the technical field of automatic lip recognition. Background technique [0002] Lip recognition or lip reading is an attractive field in Human-Computer Interaction (HCI), which plays an important role in Automatic Speech Recognition (ASR) systems. Human language perception is a naturally multimodal process. Hearing-impaired people take full advantage of lip cues, and even normal people can use visual information to enhance language understanding, especially in noisy environments. Utilizing information from the visual channel will effectively improve the performance and robustness of modern automatic language recognition systems. [0003] The lip language recognition task generally includes three main steps: ① detect the face and lip area in the pronunciation image sequence; ② extract features suitable...

AI Technical Summary

Problems solved by technology

However, so far, no relevant literature and practical application of the canonical correlation discriminant analysis method for automatic lip recognition have been found.

Images