All-media news intelligent cataloguing method based on multi-modal information fusion understanding

Abstract

The invention discloses an all-media news intelligent cataloguing method based on multi-modal information fusion. The method comprises the steps: carrying out the unified representation and fusion understanding of video information, voice information, subtitle bar information and character information in all-media news, so as to obtain multi-modal fusion features; and realizing automatic slicing, automatic cataloguing description and automatic scene classification of news by using multi-modal fusion features. The full-media news-oriented automatic comprehensive cataloguing method has the beneficial effects that a complete process of full-media news-oriented automatic comprehensive cataloguing is realized, and the accuracy and generalization of the cataloguing method are improved by generating the splitting mark, the news cataloguing description, the news classification label, the news keyword and the news character on the basis of fusion of multiple modalities of the video, the audio and the text; the manual cataloguing time is greatly reduced.

Description

technical field [0001] The invention relates to the field of intelligent media, in particular to an all-media news intelligent cataloging method based on multimodal information fusion and understanding. Background technique [0002] With the popularization of smart mobile devices and the development of digital media content, all-media news content including images, videos, voices, texts, etc. are produced at high frequency and spread rapidly. The problems of massive resources and high labor costs make the media industry more and more urgent. The pursuit of intelligent methods in resource management processes such as classification, cataloging, and retrieval. [0003] Omni-media news is a typical multi-modal content carrier, which contains multi-modal information such as news scene or studio images, news subtitle text, news broadcast voice, etc., which makes the intelligent technology for single-modal processing unable to effectively replace cataloging tasks manual process i...

AI Technical Summary

Problems solved by technology

[0010] Aiming at the problem of effect bottleneck and manual dependence caused by the single-modal processing mode of news content in the existing cataloging process, and the problem of compatibility of cataloging algorithms for news report content, the present invention proposes an all-media based on multi-modal information fusion understanding A news smart cataloging approach that increases the completeness, validity and compatibility of the knowledge needed for cataloging from news stories, enabling a more accurate and automated smart cataloging process

Images