Video character recognition method based on multi-modal feature fusion deep network

Abstract

The invention discloses a video character recognition method based on a multi-modal feature fusion deep network. The video character recognition method is a deep learning target recognition multi-modal fusion algorithm provided specially for the target recognition problem of multi-modal character video feature data. According to the video character recognition method, a network structure of the algorithm is composed of a plurality of single-modal multi-layer sensor identification modules and a multi-modal feature fusion module. The video character recognition method comprises the steps of: preprocessing multi-modal data generated by a video, training a plurality of deep networks by using the preprocessed different modal data, on the basis, subjecting features generated by a plurality of sub-networks to weighted fusion, and combining a feature weighted fusion module with models of different modals to achieve a better identification result. By adopting the video character recognition method, a multi-modal feature set weight fusion strategy is used for constructing a video figure target recognizer on a public video figure data set (iQIYI-VID-2019) for multi-modal features generated bythe preprocessed video, multi-model integration is not needed, and the average precision mean value of a single model reaches 89.52%.

Description

technical field [0001] The invention relates to a video character recognition method based on a multimodal feature fusion deep network, which belongs to the field of computer applications. Background technique [0002] With the rapid development of the Internet today, major video websites have massive video data and hundreds of millions of online video users. Compared with traditional images, video content information is richer. Video person recognition technology has a wide range of application scenarios, such as intelligent recommendation scenarios for advertising and user customization; intelligent creation scenarios for background music, emoticon package generation, and short video synthesis; violence, blood, pornography and other violations Smart moderation scenarios for videos. [0003] Person recognition has become a popular computer application direction. With the development of deep learning technology, face recognition, speech recognition, gesture recognition, ga...

AI Technical Summary

Problems solved by technology

However, the correlation and transferability between the various modal information in the video data make the traditional method of pattern recognition relying on a single information low in the accuracy of video person recognition, because the characteristics of each individual modal are not enough. Covers all elements of person identification

Images