Speaking recognition method based on video analysis, system and equipment and medium

Abstract

The invention provides a speech recognition method based on video analysis, and the method comprises the steps: carrying out the cutting and graying of each video frame in target video data, and obtaining an input image corresponding to each video frame; performing face detection processing on all the input images to obtain a face detection frame; screening the face detection frames corresponding to all the input images, and taking the face detection frame corresponding to each video frame conforming to a preset face screening rule as a final face detection frame of the frame; and calculating a feature result corresponding to each final face detection frame according to the lip contour and the face key points in the final face detection frames, inputting a plurality of feature results into a preset speaking recognition model for recognition, and obtaining a speaking recognition result corresponding to the to-be-recognized person. According to the speech recognition method based on video analysis, the obtained speech recognition result is more accurate, and the speech recognition method can adapt to different forms when the person to be recognized speaks.

Description

technical field [0001] The invention relates to the field of intelligent interaction, in particular to a speech recognition method, system, device and medium based on video analysis. Background technique [0002] In the field of intelligent interaction, when starting the intelligent interaction system, it is necessary to first determine whether the person to be recognized is speaking. When the person to be recognized is speaking, the intelligent interaction system starts the sound pickup function and executes the subsequent voice interaction function. At present, in the field of intelligent interaction, the judgment of whether the person to be recognized is speaking is based on lip feature points combined with simple threshold analysis to judge whether to speak or to judge whether to speak through audio analysis combined with lip feature analysis. The threshold analysis of the above speech recognition process cannot achieve the robustness of the model and is not suitable for...

AI Technical Summary

Problems solved by technology

[0003] In order to overcome the deficiencies of the prior art, one of the purposes of the present invention is to provide a speech recognition method based on video analysis, which can solve the problem that the speech recognition of the person to be recognized in the traditional intelligent interactive system has certain limitations and is not intelligent enough. The problem

[0004] The second object of the present invention is to provide a speech recognition system based on video analysis, which can solve the problem that the speech recognition of the person to be recognized has certain limitations and is not intelligent enough in the traditional intelligent interactive system

[0005] The third object of the present invention is to provide an electronic device, which can solve the problem that the speech recognition of the person to be recognized has certain limitations and is not intelligent enough in the traditional intelligent interactive system

Images