Image-text cross-modal retrieval based on multi-layer semantic deep Hash algorithm

Abstract

The invention relates to an image combined with deep learning and hashing methods. And a text cross-modal retrieval model. The invention provides a deep cross-modal Hash algorithm based on multilayersemantics in order to solve the limitation that a traditional cross-modal Hash method based on deep learning directly converts multi-label data into a single label problem when processing the multi-label data problem. And the similarity between the data is defined through a co-occurrence relation between the multi-label data, and the similarity is used as supervision information of network training. And designing a loss function which comprehensively considers multi-layer semantic similarity and binary similarity, and training the network, so that the feature extraction and Hash code learningprocesses are unified in one framework, and end-to-end learning is realized. According to the algorithm, semantic correlation information between the data is fully utilized, and the retrieval accuracyis improved.

Description

technical field [0001] The invention relates to the field of cross-modal retrieval, in particular to an image-text cross-modal retrieval algorithm based on multi-layer semantics combined with deep learning and a hash method. Background technique [0002] With the development of mobile Internet and the popularization of smart phones, digital cameras and other devices, the multimedia data on the Internet is growing explosively. In the field of information retrieval, the continuous growth of multimedia big data has brought about the demand for cross-modal retrieval applications. However, the current mainstream search engines, such as Baidu, Google, Bing, etc., only provide one mode of retrieval results. In addition, as deep learning has made a series of breakthroughs in the fields of computer vision and natural language processing, the combination of multimedia big data and artificial intelligence is a common development trend in the future of the two fields. Therefore, combi...

AI Technical Summary

Problems solved by technology

However, with the continuous growth of multimedia data, the feature representation using deep learning faces the challenges of storage space and retrieval efficiency due to the large dimensionality, which makes it unable to adapt to large-scale multimedia data retrieval tasks.

At the same time, the problem of cross-modal retrieval also faces the problem of multiple labels in real data

Most of the existing solutions use the problem of converting the problem into a single-label learning problem of binary correlation, resulting in the learned model not being able to fully preserve the relationship between the data in the original semantic space and affecting the final retrieval results.

Images