Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

A method and system for cross-modal hash retrieval fusing supervisory information

A cross-modal, hashing technology, applied in the field of cross-modal hash retrieval method and system that fuses supervisory information, can solve the problems of not being able to fully mine the complex relationship between multi-modal data, and achieve similarity and semantics Consistency, the effect of reducing quantization error

Inactive Publication Date: 2019-02-01
SHANDONG NORMAL UNIV
View PDF8 Cites 55 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

This has the following disadvantages: simply by imposing constraints on the last layer of the neural network of different modalities, the complex relationship between multimodal data cannot be fully exploited

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • A method and system for cross-modal hash retrieval fusing supervisory information
  • A method and system for cross-modal hash retrieval fusing supervisory information
  • A method and system for cross-modal hash retrieval fusing supervisory information

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0049] This embodiment discloses a cross-modal hash retrieval method that fuses supervision information, such as Figure 1-2 shown, including the following steps:

[0050] Phase 1: Unified Hash Code Learning

[0051] Step 1: Build three networks: image network, text network and fusion network. (1) The CNN-F network used by the image network. The original CNN-F model has a total of 8 layers, including 5 convolutional layers and 3 fully connected layers. (2) For the text modality, first represent each text sample as a bag-of-word (BOW) vector, and then input the BOW vector to a text network with two fully connected layers. In particular, the number of hidden units in the last layer of image and text networks is equal, and different values ​​are set according to different encoding lengths and data sets. (3) The fusion network consists of two fully connected layers, which combine the outputs of the image and text networks pairwise. In order to obtain a unified hash code, the ...

Embodiment 2

[0127] The purpose of this embodiment is to provide a computing device.

[0128] A computer system, comprising a memory, a processor, and a computer program stored in the memory and operable on the processor, when the processor executes the program, it realizes:

[0129] Construct image network, text network and fusion network;

[0130] Obtain image and text feature training sample pairs, input image network and text network respectively;

[0131] Using the output features of the image network and the text network as the input of the fusion network, and defining the output of the fusion network;

[0132] Constructing an objective function for learning a unified hash code according to the output of the fusion network and the similarity between pairs;

[0133] Solving the objective function to obtain a unified hash code;

[0134] The unified hash code is used as supervisory information, combined with semantic information, to train a modality-specific hash network.

Embodiment 3

[0136] The purpose of this embodiment is to provide a computer-readable storage medium.

[0137] A computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, the following steps are implemented:

[0138] Construct image network, text network and fusion network;

[0139] Obtain image and text feature training sample pairs, input image network and text network respectively;

[0140] Using the output features of the image network and the text network as the input of the fusion network, and defining the output of the fusion network;

[0141] Constructing an objective function for learning a unified hash code according to the output of the fusion network and the similarity between pairs;

[0142] Solving the objective function to obtain a unified hash code;

[0143] The unified hash code is used as supervisory information, combined with semantic information, to train a modality-specific hash network.

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a cross-modal hash retrieval method and system for fusing supervision information. The method comprises the following steps: constructing an image network, a text network and afusion network; The training sample pairs of image and text features are obtained and input into image network and text network respectively. Taking the output characteristics of the image network and the text network as the inputs of the fusion network, and defining the output of the fusion network; Constructing an objective function for learning a unified hash code according to the output of the converged network and the similarity between pairs; Solving the objective function to obtain a unified hash code; The unified hash code is used as the supervisory information and the semantic information is combined to train the hash network of a specific mode. The invention simultaneously learns the feature representation and the hash coding based on the end-to-end depth learning framework, canmore effectively capture the correlation between different modal data, and is conducive to the improvement of the cross-modal retrieval accuracy.

Description

technical field [0001] The present disclosure relates to a cross-modal retrieval method, and more specifically, to a cross-modal hash retrieval method and system that fuses supervisory information. Background technique [0002] In recent years, with the dramatic growth of different types of data on the web, Approximate Nearest Neighbor (ANN) search plays an increasingly important role in related applications. For example, information retrieval, data mining, computer vision, etc. Hashing technology has become one of the most popular techniques in ANN search due to its low computational cost and high storage efficiency. The basic idea of ​​hashing is to map high-dimensional data into a compact binary coded Hamming space by learning a hash function, while preserving the similarity structure of the original space as much as possible. At present, many hashing methods applied to unimodal scenarios have been proposed. However, in the real world, data with the same semantics often...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
IPC IPC(8): G06F16/31G06F16/33G06K9/62G06N3/04G06N3/08
CPCG06N3/08G06N3/045G06F18/25
Inventor 张化祥王粒冯珊珊任玉伟刘丽张庆科朱磊
Owner SHANDONG NORMAL UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com