Cross-media information analysis and retrieval method

Abstract

The invention provides a cross-media information analysis and retrieval method, which comprises the following steps of: performing semantic integration processing on multimode information; performing expansion according to a probability latent semantic analysis model to obtain a multilayer continuous probability latent semantic analysis model for processing a continuous feature vector; learning the multilayer-continuous probability latent semantic analysis model by adopting an asymmetric learning method, and calculating the visual feature vector distribution of an image, the visual feature vector distribution of an audio and topic probability distribution; submitting a training set and a tested media object which serves as a retrieval example by a user, and calculating intra-mode and inter-mode initial similarity values of the image and the audio in the retrieval sample; constructing a propagation model, and updating the intra-mode and inter-mode similarity values according to the propagation model; and performing secondary retrieval according to the updated similarity values.

Description

technical field [0001] The invention relates to the field of multimedia information data retrieval, in particular to a cross-media information analysis and retrieval method. Background technique [0002] Multimedia is a complex of text, images, audio and video, etc. These different types of multimedia data express rich and colorful semantics. There are several major problems in the research of multimedia documents: (1) Since the multimedia data between different modalities is usually unstructured or semi-structured, the underlying characteristics of multimedia data between different modalities are different due to different dimensions and attributes. Due to the structure and incomparability, there is a semantic gap between the underlying features and the high-level semantics, which greatly intensifies the difficulty of cross-modal retrieval between different modalities. (2) The correlation between modes is weak. Different types of multimedia data jointly express a variety ...

AI Technical Summary

Problems solved by technology

There are several major problems in the research of multimedia documents: (1) Since the multimedia data between different modalities is usually unstructured or semi-structured, the underlying characteristics of multimedia data between different modalities are different due to different dimensions and attributes. structure and incomparability, there is a semantic gap between low-level features and high-level semantics, which greatly intensifies the difficulty of cross-modal retrieval between different modalities

(2) Weak correlation between modes

(3) Low retrieval efficiency

(4) In the process of user annotation, the text annotation presents a certain personality with different users, which is greatly affected by subjective factors, so there are inconsistencies and inaccuracies, which affect the retrieval efficiency

For example: a 128-dimensional visual feature vector and a 21-dimensional auditory feature vector, both may express similar semantic concepts, such as explosions and pictures and the sound of explosions, but it is difficult for a computer to measure two degree of relatedness at the semantic level

Images