Cross-modal retrieval method and system based on semantic condition association learning

Abstract

According to the cross-modal retrieval method and system based on semantic condition association learning, the multi-label information serves as a new observation mode, and the multi-label semantic relation is effectively integrated into a cross-modal implicit representation learning framework based on a deep neural network. On one hand, the feature learning process of each mode is guided throughlabel semantic information, depth feature representation which keeps a semantic relation and has discrimination ability is obtained, and the cross-modal retrieval performance is improved; on the otherhand, the high-level semantics in the multi-label data are mined by using the deep network, and the typical correlation of different modal features with respect to the high-level semantics is maximized by using a conditional association learning method, so the shared semantic information can be eliminated from each modal data, and a direct association relationship between different modals is established. The method is advantaged in that the influence of the noise label on the cross-modal implicit representation is effectively reduced.

Description

technical field [0001] The invention relates to a cross-modal retrieval technology in the multimedia field, in particular to a high-level semantic conditional association learning technology for cross-modal data. Background technique [0002] Cross-modal retrieval technology is one of the important research topics in the field of multimedia, in order to facilitate users to obtain the required multi-modal information. Cross-modal retrieval can match a given modality data to semantically related data of another modality from massive multimedia information. Therefore, cross-modal retrieval technology needs to solve the problem of how to establish the relationship between heterogeneous modal content. [0003] At present, most cross-modal retrieval algorithms realize the relationship measurement among heterogeneous modalities by learning the common latent space of samples from different modalities. In order to maintain the semantic consistency of cross-modal latent spaces, exis...

AI Technical Summary

Problems solved by technology

[0004] The purpose of the present invention is to propose a cross-modal retrieval method based on semantic conditional association learning for the problems of noise in user label tags and lack of discriminative power in cross-modal implicit space representation

Images