A method and device for generating hash codes using multi-layer feature fusion

Abstract

The invention discloses a method and device for producing hash codes by fusion of multi-layer features, including: establishing a similarity matrix of image-text pairs; obtaining the features of different layers through the output of different residual blocks, and combining the The features of the image are converted into feature maps with the same number of channels and sizes, and then fused, and finally the hash code corresponding to the image is obtained through global pooling and full connection and discretization; the multi-scale fusion module is used to generate corresponding multi-scale for each text BOW model, then obtain features of different scales through the convolutional layer and fuse them, and finally obtain the hash code corresponding to the text through the fully connected layer; design a loss function; train the model; use the trained model to input samples into it to obtain the corresponding hashcode. Through the present invention, the generated hash code is more discriminative, and can effectively improve the average retrieval accuracy rate when used for cross-modal retrieval.

Description

technical field [0001] The invention relates to the technical field of image retrieval, in particular to a method and device for generating a hash code by fusion of multi-layer features. Background technique [0002] With the rapid development of the network, more and more data of different modalities appear on the Internet, such as images, texts, etc. People hope to find the information they need in various forms of data. Traditional single-modal retrieval can no longer meet people's needs, so cross-modal retrieval is proposed. The goal of cross-modal retrieval is to use a query from one modality (such as image) to find semantically similar instances in another modality (such as text). However, the similarity measurement between different modality data is very challenging due to the heterogeneity difference between different modality data, and the semantic gap between low-level features and high-level semantics. A common approach to make up this difference is to map diffe...

AI Technical Summary

Problems solved by technology

The above methods based on deep convolutional deep network learning only use the features extracted from a certain layer of deep convolutional neural network to represent data of different modalities. The extracted features are usually high-level features in the network, such as Simonyan, Karen, and Andrew Zisserman. "Very Deep Convolution Networks for Large-Scale Image Recognition." International Conference on Learning Representations. 2015. The output of the fc8 layer of the proposed VGG network, however, most of the high-level features encode semantic features, and a lot of spatial information is lost

Images