A cross-modal retrieval method based on cyclic generation antagonistic network

Abstract

The invention discloses a cross-modal retrieval method based on a cyclic generation antagonistic network. The method designs a novel dual-channel cyclic generation antagonistic neural network and establishes semantic correlation of cross-modal data by training the neural network. Given different modal data can flow bi-directionally in the network, each modal data can generate another modal data through a group of generative antagonistic networks, and the generated data can be used as the input of the next group of generative antagonistic networks, so that the bi-directional loop generation ofdata can be realized, and the network continuously learns the semantic relationship among the cross-modal data. In order to improve the efficiency of retrieval, The method also uses threshold functionand approximation function to approximate the result of the middle layer of the generator to the corresponding binary hash code, and designs a variety of constraints to ensure the same mode, the similarity and cross-mode of the same kind of data, and the difference of data between classes, thus further improving the accuracy and stability of retrieval.

Description

technical field [0001] The invention belongs to the technical field of multimedia information retrieval, and in particular relates to a cross-modal retrieval method based on a cyclic generative confrontation network. technical background [0002] With the advent of the Internet era, people can access massive amounts of information in various modes including pictures, videos, texts, and audios anytime and anywhere. How to obtain the content they need from these massive amounts of information has become the focus of Internet users. Often rely on the precise search services provided by search engines such as Google, Baidu, and Bing. However, most of the traditional Internet retrieval services are still at the level of single-modal retrieval, and there are few retrieval applications for cross-modal data. The efficiency, accuracy, and stability of retrieval need to be improved, and most of them rely on existing The data label of the data cannot achieve cross-modal retrieval of u...

AI Technical Summary

Problems solved by technology

However, most of the traditional Internet retrieval services are still at the level of single-modal retrieval, and there are few retrieval applications for cross-modal data. The efficiency, accuracy, and stability of retrieval need to be improved, and most of them rely on existing data labels, it is impossible to achieve cross-modal retrieval of unlabeled data

Images