Combined lip reading and voice recognition multimodal interface system

Abstract

The present invention provides a combined lip reading and voice recognition multimodal interface system, which can issue a navigation operation instruction only by voice and lip movements, thus allowing a driver to look ahead during a navigation operation and reducing vehicle accidents related to navigation operations during driving. The combined lip reading and voice recognition multimodal interface system in accordance with the present invention includes: an audio voice input unit; a voice recognition unit; a voice recognition instruction and estimated probability output unit; a lip video image input unit; a lip reading unit; a lip reading recognition instruction output unit; and a voice recognition and lip reading recognition result combining unit that outputs the voice recognition instruction

Description

CROSS-REFERENCE TO RELATED APPLICATION[0001]This application claims under 35 U.S.C. §119(a) the benefit of Korean Patent Application No. 10-2009-0089637 filed on Sep. 22, 2009, the entire contents of which are incorporated herein by reference.BACKGROUND OF THE INVENTION[0002]The present invention relates, in general, to a combined lip reading and voice recognition multimodal interface system. More particularly, in preferred embodiments, the present invention relates to a combined lip reading and voice recognition multimodal interface system, which can suitably issue a navigation operation instruction primarily, preferably only, by voice and lip movements, thus preferably allowing a driver to look ahead during a navigation operation and suitably reducing vehicle accidents related to navigation operations during driving.[0003]Presently, with the development of automobile technology and the increasing use of vehicles in daily life, there has been increasing interest and demand for safe...

AI Technical Summary

Benefits of technology

[0009]The present provides, in preferred aspects, a combined lip reading and voice recognition multimodal interface system, which implements a lip reading system that effectively detects lips from a face image through a camera, suitably tracks lip movements, and suitably recognizes a voic...

Problems solved by technology

Although the use of the touch screen can minimize input errors, a user has to use his or her hands and eyes at the same time, which makes it difficult to operate the navigation system during driving, and also distracts the user's attention, thus increasing the risk of an accident.

However, this method is susceptible to audio noise, and therefore a malfunction in recognition may occur in a noisy environment.

However, at present, there has not been any consistent research on all the processes.

Its performance is susceptible to an initial position, and quick movements of lips in speech cannot be robustly tracked, thereby making it difficult to obtain stable feature values when tracking on a video.

Further, while research has been conducted on recognizer algori...

Images