A CMR model for uniformly retrieving cross-media information

Abstract

The invention discloses a CMR model for uniformly retrieving cross-media information, and aims to provide a cross-media retrieval model with accurate and rapid information. The method is realized through the following technical scheme: a multi-modal media information semantic feature unified expression and association module queries heterogeneous information input by the input and cross-media datamodule; the multi-modal semantic features are mapped into the same feature space and a multi-modal semantic association rule is constructed based on the multi-modal semantic feature extraction resultand the mapping from the bottom-layer features to the high-layer semantic features, thereby realizing the association between the bottom-layer features and the high-layer semantic features of the cross-media information and the association of the high-layer semantics of different modal information; the cross-media data index construction module is used for establishing a multi-dimensional retrieval index for the multi-modal data characteristics; the cross-media retrieval model construction module realizes unified retrieval of multi-modal information based on ontology, semantic network and knowledge graph technologies; and conflict detection and self-organization of retrieval results are realized through a multi-modal retrieval result association verification and organization module.

Description

technical field [0001] The invention relates to the field of cross-media information retrieval, in particular to computer science information retrieval cross-media retrieval (cross-media retrieval, CMR) technology. Background technique [0002] Since mankind entered the information age, human society has developed rapidly, and media information resources have expanded from the original text to rich and diverse images, audio, video, multi-dimensional models and other types of information. In the process of identifying information, people often need perceptual knowledge of different types of information, including comprehensive analysis of visual, auditory and other information, to form an overall cognition. To achieve this, it is necessary to use cross-media information retrieval methods for comparison during multimedia information retrieval and management. The development of cross-media retrieval technology is inseparable from the traditional content-based multimedia retrie...

AI Technical Summary

Problems solved by technology

However, this method has certain defects: when establishing these three independent spaces, the semantic association between different types of multimedia files has not been considered; in addition, the two-layer manifold learning method needs to adjust a large number of parameters, and its complexity not suitable for practical use

However, the research object of the existing cross-media information retrieval technology is mainly various media data that has been collected, and less consideration is given to new problems and challenges faced in the mobile Internet environment such as data redundancy in multi-user sharing and network bandwidth constraints.

Images