Cross-modal retrieval method and system based on semantic constraint matrix decomposition hash

Abstract

The invention discloses a cross-modal retrieval method based on semantic constraint matrix decomposition hash, which comprises the following steps of: inputting original characteristic matrixes of different modals, decomposing each modal according to an established individual matrix model to obtain a potential semantic matrix, and constraining two modal potential semantic matrixes by utilizing a semantic similarity matrix to obtain a cross-modal retrieval matrix; generating a hash code according to the potential semantic matrix and the orthogonal rotation matrix, and finally obtaining a classlabel from the hash code by using the potential speech matrix. According to the method, the semantic similarity matrix is used for constraining the modal specific representation of each modal, the retrieval effect is superior to that of an existing matrix decomposition method, and a large number of experiments on three data sets show that the method has better retrieval performance.

Description

technical field [0001] The invention belongs to the field of hash code retrieval, and in particular relates to a cross-modal retrieval method and system based on semantic constraint matrix decomposition hash. Background technique [0002] With the massive growth of multimedia data such as text, image, audio, video, etc., cross-modal retrieval has attracted extensive attention. Taking text and image modalities as examples, the task of cross-modal retrieval is given a queried modality, then query and return its similar results in other different modalities. It has been widely studied and applied in computer vision, text mining and information retrieval, how to effectively conduct cross-modal retrieval has become a research hotspot. [0003] In recent years, hash-based cross-modal retrieval methods have been extensively studied due to their advantages of low storage cost and fast query speed. Existing cross-modal hashing methods mainly project multimodal data into a common se...

AI Technical Summary

Problems solved by technology

However, common latent semantic representation methods can lose individual and useful modality-specific features, cannot fully utilize the intrinsic information of each modality, and correlation matrices or orthogonal rotation transformations cannot well constrain latent semantic representations.

Images