A cross-modal data retrieval method and system based on graph regularization and modality independence

Abstract

The invention discloses a cross-modal data retrieval method and system based on graph regularization and modality independence. The method includes: receiving raw data of different modalities for feature extraction, and establishing a one-to-one corresponding image-text pair A multi-modal data set, the multi-modal data set includes a training set and a test set; by optimizing a predefined objective function, the feature matrix of different modal data in the training set is projected to a common subspace to obtain an image projection matrix and Text projection matrix; according to the image projection matrix and the text projection matrix, the feature matrix of different modal data in the test set is projected to the common subspace; the similarity between the projected matrix and other projection matrices in the common subspace is calculated; according to The similarity is sorted in descending order, and the data corresponding to the first few feature projection matrices are obtained for cross-modal retrieval.

Description

technical field [0001] The disclosure belongs to the technical field of data retrieval, and relates to a cross-modal data retrieval method and system based on graph regularization and modality independence. Background technique [0002] The statements in this section merely provide background technical information related to the present disclosure and do not necessarily constitute online technology. [0003] In the era of big data where multimedia information is growing rapidly, the type of media information has evolved from the original text to text, images, and videos; the way to obtain information has also evolved from broadcasting and newspapers to websites, Weibo, WeChat, and QQ. In this context, the traditional text search methods can no longer meet people's special needs, so cross-media retrieval is proposed, and the correlation between different modalities in cross-media retrieval has attracted more and more attention. [0004] However, the inventors discovered duri...

AI Technical Summary

Problems solved by technology

In this context, the traditional text search methods can no longer meet people's special needs, so cross-media retrieval is proposed, and the correlation between different modalities in cross-media retrieval has also attracted more and more attention.

[0004] However, the inventors discovered during the research and development process that most retrieval methods, such as text retrieval, image retrieval, and video retrieval, etc., mostly focus on single-modal retrieval, and can only provide queries and retrievals of the same modality.

Images