Hierarchical tag-based cross-modal hash model construction method, search method and device

Abstract

The invention discloses a hierarchical tag-based cross-modal Hash model construction method, a search method and a device. The method comprises the following steps of receiving a multi-modal data set,and preprocessing; inputting the relative data of the preprocessed data of different modalities into a pre-trained multi-path neural network; according to the pre-trained neural network and the multi-layer perceptron, extracting the characteristic data of different modes respectively, and obtaining the hierarchical Hash representations of different modes; constructing the similarity matrixes of the preprocessed samples on different levels according to the level labels, and evaluating the semantic similarity among the samples according to the inner product of the Hash representation trained bythe median of each level of similarity matrix; adopting the hierarchical labels with different granularities, analyzing the influence of the hierarchical comparison on the neural network performance,and determining the optimal hierarchical ratio; obtaining a Hash code according to each layer of Hash representation; and training the dual-path neural network, optimizing and training the dual-pathneural network by using an SGD gradient descent method, and establishing a deep cross-modal hash model based on hierarchical tags for cross-modal search.

Description

technical field [0001] The disclosure belongs to the technical field of multimedia data search, and relates to a method for constructing a cross-modal hash model based on hierarchical tags, a search method and a device. Background technique [0002] The statements in this section merely provide background information related to the present disclosure and do not necessarily constitute prior art. [0003] With the vigorous development of multimedia devices, the amount of multimedia data on the Internet has increased unprecedentedly. Data often exists in multiple modalities, such as pictures, texts, and videos. There are often semantic correlations between these modalities of data, expressing the same object from different angles, so that people have a clearer and more complete understanding of the data. In real-life applications, such as major e-commerce platforms, people are more and more inclined to search for each other between different modal data. similar data. Theref...

AI Technical Summary

Problems solved by technology

[0005] However, the inventors found in the research and development process that the existing technology still has some deficiencies in the use of tag information. Som

Images