Universal cross-modal retrieval model based on deep hash

Abstract

The invention discloses a universal cross-modal retrieval model based on deep hash. The universal cross-modal retrieval model comprises an image model, a text model, a binary code conversion model and a Hamming space. The image model is used for the feature and semantic extraction of the image data; the text model is used for the feature and semantic extraction of the text data; the binary code conversion model is used for converting the original features into the binary codes; the Hamming space is a common subspace of images and the text data, and the similarity of the cross-modal data can be directly calculated in the Hamming space. According to the universal model for solving cross-modal retrieval by combining deep learning and Hash learning, the data points in an original feature space are mapped into the binary codes in the public Hamming space, similarity ranking is carried out by calculating the Hamming distance between the codes of the data to be queried and the codes of the original data, and therefore a retrieval result is obtained, and the retrieval efficiency is greatly improved. The binary codes are used for replacing the original data storage, so that the requirement of the retrieval tasks for the storage capacity is greatly reduced.

Description

technical field [0001] The invention relates to the field of cross-modal retrieval, especially the cross-modal retrieval of images and texts. Background technique [0002] In recent years, with the vigorous development of the Internet and the popularity of smart devices and social networks, multimedia data has exploded on the Internet. These massive data include various modal forms such as text, image, video, and audio, and the same thing will be described by many different modal data. These data are "heterogeneous and multi-source" in form, but interrelated semantically. People's demand for information acquisition is no longer satisfied with single-modal data retrieval, and the realization of cross-modal retrieval through knowledge collaboration of different modes has become a research hotspot in recent years. [0003] Deep learning has made breakthroughs in single-modal fields, such as natural language processing, imagery, and speech recognition. The powerful abstraction...

AI Technical Summary

Problems solved by technology

[0004] Most existing techniques directly model based on the extracted feature values ​​to achieve cross-modal retrieval, which is very time-consuming for large-scale datasets and requires a lot of storage space

And it only pursues the retrieval accuracy, but ignores the retrieval efficiency, which leads to the huge retrieval delay and low efficiency of the trained model, making it impossible to apply in reality

Images