A Cross-media Ranking Method Based on Multimodal Implicit Coupling Expression

Abstract

The invention discloses a cross-media ordering method based on multi-modal implicit coupling expression, which comprises the following steps of 1) constructing an ordering sample of a text retrieval image or an ordering sample of an image retrieval text as a training sample, 2) performing cross-media ordering learning based on the implicit coupling expression on the constructed training sample to obtain an implicit coupling expression excavation model of cross-media data and a cross-media ordering model, 3) constructing the implicit coupling expression between a query file and a candidate file, and 4) performing cross-media retrieval on the learnt cross-media ordering model based on the implicit coupling expression. According to the method, the implicit coupling expression of multi-modal data is introduced into the ordering model, so that the method has higher distinguishing performance compared with the general implicit expression of the multi-modal data. The implicit expression excavation model and the ordering model are simultaneously trained, so that the performance obtained by the method in the image retrieval text or the text retrieval image is better compared with the traditional cross-media ordering model method.

Description

technical field [0001] The invention relates to cross-media retrieval, in particular to a cross-media sorting method based on implicit coupling expression. Background technique [0002] Cross-media data retrieval is an important technical field with practical significance, and sorting cross-media data according to its relevance is an important technology in this field. During the retrieval process, this technology sorts the candidate cross-media data according to the degree of relevance to the user query, and presents the sorting results to the user, which has great value in the search application of cross-media data. [0003] Traditional cross-media ranking methods generally learn a shared space for multimodal data first, then map query documents and candidate documents to feature vectors in the shared space, and finally use a manually specified similarity metric function to calculate query and candidate documents. The relevance between documents, and finally sort the mult...

AI Technical Summary

Problems solved by technology

This method uses the vector in the shared space as the representation of multimodal data, and it is difficult to mine the complex dependencies that should exist within this common representation; at the same time, due to the artificially specified similarity measurement function of the ranking model, it cannot pass the learning A method for discovering the magnitude of the influence of different parts of a multimodal common representation on data associativity

Images