A Lip Recognition Method and Device Based on Dual Discriminator Generative Adversarial Network

Abstract

The present application relates to a lip language recognition method and device based on a double discriminator generation confrontation network, the method includes: deriving face pictures from different angles from the video, and obtaining lip recognition according to different head deflection angles in the face pictures part multi-angle data set; and according to the generator data set, identity discriminator data set, and angle discriminator data set obtained from the lip multi-angle data set, regenerate the confrontation network data model; use the confrontation network data model to be identified Lip recognition is performed on the video, and a 0° lip image is converted; a lip feature vector is extracted from the 0° lip image, and the lip feature vector is modeled and classified to obtain a lip classification result. Partial classification results are exported to recognize the language. The embodiment of the present invention has a visual effect similar to the real environment, can well guide the model to adapt to the actual application environment, and further improve the accuracy of the lip language recognition model.

Description

technical field [0001] The present application relates to the field of artificial intelligence, in particular to a lip recognition method and device based on a dual-discriminator generative adversarial network. Background technique [0002] Lip recognition is a complex task combining computer vision and natural language processing. It can be used to automatically infer the text content contained in visual and auditory information. It has a wide range of applications, such as recovering speech from silent surveillance videos or movies. In recent years, the development of lip language recognition is mainly driven by the following two aspects: First, the rapid development of deep learning technology, which is a technology derived from neuroscience, has made great achievements in image processing, language models and other fields. success. The second is the proposal of large data sets, which provide a large amount of training data and complex environmental changes for lip recog...

AI Technical Summary

Problems solved by technology

However, in the feature extraction stage, most methods only use simple data preprocessing, such as random cropping, horizontal flipping, increasing contrast, etc. These preprocessing methods can only alleviate the overfitting problem to a certain extent, and cannot solve it well. Effect of speaker state such as face deflection on feature extraction

Images