Lip language recognition method and device, computer equipment and storage medium

Abstract

The invention discloses a lip language recognition method and device, computer equipment and a storage medium. The lip language recognition method comprises the following steps: carrying out standardization processing on a frame rate of an obtained original video, and separating the obtained standard video to obtain an effective audio stream and an effective video stream; tracking a human face in the effective video stream by using a human face recognition algorithm, extracting a mouth area in the human face, and obtaining a frame lip action video so as to obtain a lip image sequence; segmenting the lip image sequence by adopting a sequence segmentation rule to obtain a segmented image sequence; sequentially inputting the segmented image sequences corresponding to the lip image sequences into a lip image recognition model for recognition, and obtaining a lip image recognition result; inputting the effective audio stream into a speech recognition model to obtain a speech recognition result; and when the similarity between the lip image recognition result and the original video reaches a preset value, taking the lip image recognition result as a lip language recognition result of the original video so as to ensure the accuracy of the lip image recognition result.

Description

technical field [0001] The invention relates to the field of biological identification, in particular to a lip recognition method, device, computer equipment and storage medium. Background technique [0002] In recent years, lip language recognition has been widely used in public security fields such as intelligent human-computer interaction, audio damage, video surveillance, military and criminal investigation security, and has become a research hotspot in the industry. At the same time, it has very important practical significance in the field of hearing-impaired aphasia. Traditional lip language recognition technologies mostly use traditional algorithms such as HMM (Hidden Markov Model), LBP (Local Binary Pattern) of texture features, or deep learning algorithms such as convolutional neural networks. The input is a single frame The picture does not take into account the semantic information before and after the frame of the picture, only the spatial channel features are o...

AI Technical Summary

Problems solved by technology

[0003] The embodiment of the present invention provides a lip language recognition method, device, computer equipment and storage medium to solve the problem in the prior art that the lip language corresponding to a video stream cannot be accurately recognized

Images