Video data processing method and device, equipment and storage medium

Abstract

The invention relates to the field of artificial intelligence, and discloses a video data processing method and device, equipment and a storage medium, which are used for improving the accuracy of video data processing. The video data processing method comprises the steps of inputting a plurality of video frames into a preset face recognition model for face information extraction to obtain a plurality of candidate face information, and performing context interaction processing on the plurality of candidate face information to obtain interaction information corresponding to each candidate face information; inputting the video stream into a preset lip language recognition model for lip language recognition to obtain a lip language text; according to the interaction information corresponding to each candidate face information, inputting the video stream into a preset natural language processing model for audio recognition to obtain an audio text, and calculating the similarity between the lip language text and the audio text to obtain a target similarity; and determining target face information from the plurality of candidate face information according to the target similarity. The invention also relates to a block chain technology, and the target face information can be stored in a block chain node.

Description

technical field [0001] The present invention relates to the field of artificial intelligence, in particular to a video data processing method, device, equipment and storage medium. Background technique [0002] With the development of artificial intelligence technology, multi-modal voice interaction methods are becoming more and more important. Due to the current multi-modal interaction technology, it is impossible to associate sound with images, thereby locking the interactive object. As a result, most voice interaction devices today, In complex multi-person interaction scenarios, it is impossible to accurately judge the input object, resulting in the inability to determine whether to continue the current business process or exit the current business process based on the current interaction logic. [0003] Most of the smart voice devices now, in complex public places, if multiple users make inquiries at the same time, because the interactive device cannot know which user th...

AI Technical Summary

Problems solved by technology

[0002] With the development of artificial intelligence technology, multi-modal voice interaction methods are becoming more and more important. Due to the current multi-modal interaction technology, it is impossible to associate sound with images, thereby locking the interactive object. As a result, most voice interaction devices today, In complex multi-person interaction scenarios, it is impossible to accurately judge the input object, resulting in the inability to determine whether to continue the current business process or exit the current business process based on the current interaction logic

[0003] Most of the smart voice devices now, in complex public places, if multiple users make inquiries at the same time, because the interactive device cannot know which user the input currently received comes from, the current user interaction logic is interrupted, resulting in the current The business process of serving customers is interrupted, seriously affecting the overall interactive experience of current service users

Images