System and method of interaction-type foreign-language mouth shape and tongue shape simultaneous-correction

Abstract

The invention relates to a system and a method of interaction-type foreign-language mouth shape and tongue shape simultaneous-correction. Structured light is utilized for facial mouth shape recognition of users, mouth shape, speech and text information of a teacher and students is compared and analyzed on a server, and thus mouth shape problems in a foreign-language reading process are found. According to the method, a reading mouth shape and tongue shape correction process is enabled to more have pertinence in foreign-language learning through multiple modes of ''counselling'', ''following reading'', ''repeating'', ''tongue shape correction'' and the like, interaction among the multiple users is better, and mouth shape and tongue shape correction can be more accurately and effectively carried out.

Description

technical field [0001] The invention belongs to the field of intelligent recognition, in particular to the field of simultaneous recognition of mouth shape and tongue shape by using structured light. Background technique [0002] Mouth-shape problems lead to incorrect pronunciation has always been a problem that plagues foreign language speakers. Many methods have been explored in the prior art to solve this problem. For example: watch a standard pronunciation video and improve it against the mouth movements demonstrated by the characters in the video. But this method is not interactive, and users cannot measure whether their mouth is correct. In order to solve this problem, in the prior art, there is also a method of collecting user's facial features, and comparing the user's facial features with standard facial features in video or images, so as to find out the gap. In particular, some technologies also segment the pronunciation, extract the pronunciation of a certain u...

AI Technical Summary

Problems solved by technology

Although users can use this system / method to understand their own mouth shape and pronunciation, there are still the following problems: ①Users can only compare with standard images, lack of interactivity, and cannot put forward additional demonstration requirements for their own special problems

There is also no way to compare with other learners, which is not conducive to stimulating learning motivation

② Facial recognition is not accurate

First of all, the existing identification requirements do not require high accuracy, so there is no incentive in this field to excessively increase the cost and improve the accuracy

Secondly, the video or image captured by the camera is two-dimensional information, and the human face, especially near the mouth, has rich three-dimensional information. Losing this information will lead to inaccurate mouth shape recognition

③Existing systems are all viewed and used on computers, TVs or other relatively fixed, large-sized, and powerful devices with strong processing capabilities. The use occasions are fixed and inconvenient to use. Unable to achieve high-precision facial information collection

⑤The method of segmenting the voice signal and then finding the corresponding image signal for comparison is complex and depends on the accuracy of voice recognition, which will lead to inaccurate comparison results

⑥Mouth shape is not the only problem of inaccurate pronunciation. The reason for inaccurate pronunciation lies in the double influence of mouth shape and tongue shape. The existing technology does not realize the distinction between the two, so the training efficiency is low

Images