Cross-modal hash retrieval method based on controlled semantic embedding
A cross-modal and modal technology, applied in the field of cross-modal information retrieval, can solve the problems of low retrieval accuracy and the inability of cross-modal hash retrieval methods to achieve semantic decoupling of common subspaces, so as to alleviate the loss of retrieval accuracy, Improve interpretability and quantification, the effect of accurate semantic association
Active Publication Date: 2021-06-11
GUANGDONG UNIV OF TECH
View PDF2 Cites 6 Cited by
- Summary
- Abstract
- Description
- Claims
- Application Information
AI Technical Summary
Problems solved by technology
[0005]
In order to solve the problem that the existing cross-modal hash retrieval method cannot realize the semantic decoupling of the common subspace, resulting in low retrieval accuracy, the present invention proposes a cross-modal hash retrieval method based on controlled semantic embedding, which can Learning highly disjoint representations with controlled semantic structure in an interpretable manner, improving cross-modal retrieval accuracy
Method used
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View moreImage
Smart Image Click on the blue labels to locate them in the text.
Smart ImageViewing Examples
Examples
Experimental program
Comparison scheme
Effect test
Embodiment 1
[0071] The positional relationship described in the drawings is only for illustrative purposes and cannot be construed as a limitation to this patent;
[0072] Such as figure 1 The flow diagram of the cross-modal hash retrieval method based on controlled semantic embedding is shown, see figure 1 , the method includes:
[0073] S1. Determine the tagged multimodal database to be retrieved by cross-modal hash. The multimodal database includes K modalities, expressed as 1,...,k,...,K, where k represents the kth modal In this embodiment, the multimodal database used is MS-COCO 2014; the given MS-COCO 2014 multimedia database contains 2 modes: 85000 picture samples, 85000 text samples, all samples Each has its corresponding label; the label contains a total of 80 categories; according to the deep learning training method, the training set (80,000 image samples and 80,000 text samples) and the test set (5,000 image samples, 5,000 text samples) are divided. The training set is rega...
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More PUM
Login to View More Abstract
The invention provides a cross-modal Hash retrieval method based on controlled semantic embedding, relates to the technical field of cross-modal information retrieval, and solves the problem that the existing cross-modal Hash retrieval method cannot realize semantic decoupling of a public subspace. The method comprises the following steps: firstly, determining a multi-modal database with a label; training a label network; training a controlled semantic embedding network for each mode in the database; according to labels corresponding to all samples in the database, mapping semantic vectors by a label network, and forming binary codes by a quantization method; in the query stage, mapping a controlled semantic embedding network of a corresponding mode of a query sample with a label into a semantic vector, calculating the semantic vector of the query sample and the asymmetric quantization distance of binary codes of all samples in a database and retunring retrieval results according to the sequence from large to small; highly separated public semantic vectors with controlled semantic structures can be learned in an interpretable manner, and the cross-modal retrieval precision is improved.
Description
technical field [0001] The present invention relates to the technical field of cross-modal information retrieval, and more specifically, to a cross-modal hash retrieval method based on controlled semantic embedding. Background technique [0002] With the advent of the Internet era, more and more people upload their multimedia data (such as pictures, text, video, audio, etc.) to the database on the network for storage. Simply storing multimedia data cannot generate economic Benefits have created a strong demand for efficient indexing and retrieval of data across different modalities (such as listening to songs, searching for goods by taking pictures, and searching for movies with screenshots, etc.). The definition of cross-modal retrieval is the way of retrieval based on semantic similarity between different modalities, which can solve the above problems. However, cross-modal retrieval is essentially a sorting problem: according to a given query sample, all samples in the da...
Claims
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More Application Information
Patent Timeline
Login to View More Patent Type & Authority Applications(China)
IPC IPC(8): G06F16/41G06F16/43G06F16/48G06K9/62
CPCG06F16/41G06F16/48G06F16/43G06F18/214Y02D10/00
Inventor 孟敏杨榕武继刚
Owner GUANGDONG UNIV OF TECH