Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

A cross-modal hash retrieval method and system that fuses supervisory information

A cross-modal and hashing technology, which is applied in the field of cross-modal hash retrieval methods and systems that integrate supervision information, can solve the problem of not being able to fully mine complex relationships between multi-modal data, and maintain similarity and semantics. Consistency, the effect of reducing quantization errors

Inactive Publication Date: 2019-07-23
SHANDONG NORMAL UNIV
View PDF8 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

This has the following disadvantages: simply by imposing constraints on the last layer of the neural network of different modalities, the complex relationship between multimodal data cannot be fully exploited

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • A cross-modal hash retrieval method and system that fuses supervisory information
  • A cross-modal hash retrieval method and system that fuses supervisory information
  • A cross-modal hash retrieval method and system that fuses supervisory information

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0049] This embodiment discloses a cross-modal hash retrieval method that fuses supervision information, such as Figure 1-2 shown, including the following steps:

[0050] Phase 1: Unified Hash Code Learning

[0051] Step 1: Build three networks: image network, text network and fusion network. (1) The CNN-F network used by the image network. The original CNN-F model has a total of 8 layers, including 5 convolutional layers and 3 fully connected layers. (2) For the text modality, first represent each text sample as a bag-of-word (BOW) vector, and then input the BOW vector to a text network with two fully connected layers. In particular, the number of hidden units in the last layer of image and text networks is equal, and different values ​​are set according to different encoding lengths and data sets. (3) The fusion network consists of two fully connected layers, which combine the outputs of the image and text networks pairwise. In order to obtain a unified hash code, the ...

Embodiment 2

[0127] The purpose of this embodiment is to provide a computing device.

[0128] A computer system, comprising a memory, a processor, and a computer program stored in the memory and operable on the processor, when the processor executes the program, it realizes:

[0129] Construct image network, text network and fusion network;

[0130] Obtain image and text feature training sample pairs, input image network and text network respectively;

[0131] Using the output features of the image network and the text network as the input of the fusion network, and defining the output of the fusion network;

[0132] Constructing an objective function for learning a unified hash code according to the output of the fusion network and the similarity between pairs;

[0133] Solving the objective function to obtain a unified hash code;

[0134] The unified hash code is used as supervisory information, combined with semantic information, to train a modality-specific hash network.

Embodiment 3

[0136] The purpose of this embodiment is to provide a computer-readable storage medium.

[0137] A computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, the following steps are implemented:

[0138] Construct image network, text network and fusion network;

[0139] Obtain image and text feature training sample pairs, input image network and text network respectively;

[0140] Using the output features of the image network and the text network as the input of the fusion network, and defining the output of the fusion network;

[0141] Constructing an objective function for learning a unified hash code according to the output of the fusion network and the similarity between pairs;

[0142] Solving the objective function to obtain a unified hash code;

[0143] The unified hash code is used as supervisory information, combined with semantic information, to train a modality-specific hash network.

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a cross-modal hash retrieval method and system for fusing supervisory information. The method includes: constructing an image network, a text network and a fusion network; acquiring image and text feature training sample pairs, and inputting the image network and text respectively network; using the output features of the image network and the text network as the input of the fusion network, and defining the output of the fusion network; constructing an objective function of learning a unified hash code according to the output of the fusion network and the similarity between pairs; Solving the objective function to obtain a unified hash code; using the unified hash code as supervisory information, combined with semantic information, to train a hash network of a specific modality. The present invention simultaneously learns feature representation and hash coding based on an end-to-end deep learning framework, can more effectively capture the correlation between different modal data, and contribute to the improvement of cross-modal retrieval accuracy.

Description

technical field [0001] The present disclosure relates to a cross-modal retrieval method, and more specifically, to a cross-modal hash retrieval method and system that fuses supervisory information. Background technique [0002] In recent years, with the dramatic growth of different types of data on the web, Approximate Nearest Neighbor (ANN) search plays an increasingly important role in related applications. For example, information retrieval, data mining, computer vision, etc. Hashing technology has become one of the most popular techniques in ANN search due to its low computational cost and high storage efficiency. The basic idea of ​​hashing is to map high-dimensional data into a compact binary coded Hamming space by learning a hash function, while preserving the similarity structure of the original space as much as possible. At present, many hashing methods applied to unimodal scenarios have been proposed. However, in the real world, data with the same semantics often...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Patents(China)
IPC IPC(8): G06F16/31G06F16/33G06K9/62G06N3/04G06N3/08
CPCG06N3/08G06N3/045G06F18/25
Inventor 张化祥王粒冯珊珊任玉伟刘丽张庆科朱磊
Owner SHANDONG NORMAL UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com