Cross-modal hash retrieval algorithm based on fine-grained similarity matrix

A similarity matrix and hash algorithm technology, applied in the field of cross-modal retrieval, can solve the problem that the cross-modal hash algorithm cannot mine the similarity information of data items

Active Publication Date: 2021-01-08
FUDAN UNIV
View PDF5 Cites 11 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0006] In order to solve the problem that the existing cross-modal hash algorithm cannot mine the rich similarity information of data items in the origi

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Cross-modal hash retrieval algorithm based on fine-grained similarity matrix
  • Cross-modal hash retrieval algorithm based on fine-grained similarity matrix
  • Cross-modal hash retrieval algorithm based on fine-grained similarity matrix

Examples

Experimental program
Comparison scheme
Effect test

Example Embodiment

[0060]It can be known from the background technology that the existing two-stage cross-modal hash algorithm has two main defects. First, the current two-stage hashing algorithms use coarse-grained similarity matrices, which cannot mine the rich similarity information of data items in the original space. Second, most two-stage hash algorithms use multi-classification methods to train hash codes, which may not get the best hash function. Therefore, in this embodiment, in view of the above two problems, the similarity matrix in a fine-grained definition method is used respectively, and the training method of the hash function is redesigned to solve the above two problems.

[0061]In this embodiment, in the second stage of hash function learning, for the image modal, a CNN-F network pre-trained on ImageNet is used. Keep the first five convolutional layers convl~conv5 and the next two fully connected layers fc6~fc7 unchanged, replace the eighth fully connected layer with a new fully connect...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention belongs to the technical field of cross-modal data retrieval, and particularly relates to a cross-modal hash retrieval algorithm based on a fine-grained similarity matrix. The algorithmprovided by the invention mainly aims at two tasks of image retrieval texts and text retrieval images, and comprises the following steps: hash code reasoning: constructing a fine-grained similarity matrix by utilizing label information of image text pairs, so that hash codes reserve fine-grained similarity information between image text data items; constructing an auto-encoder to enable the hash code to reserve semantic information in the label as much as possible; hash function learning: training two Hash functions, mapping images and texts to Hash codes respectively, wherein target functionsused by Hash code learning include Hash code mapping loss, similarity retention loss with weight and classification loss. The invention has relatively high retrieval precision in two tasks of image search texts and text search images.

Description

technical field [0001] The invention belongs to the technical field of cross-modal retrieval, and in particular relates to a cross-modal hash retrieval algorithm based on a fine-grained similarity matrix. Background technique [0002] With the rapid development of social media, a large amount of multimedia data is generated every day, including text, images, videos, etc. Limited by the high computational complexity and storage complexity, it becomes very difficult to perform accurate nearest neighbor retrieval on these large-scale multimedia data. To solve this problem, many alternative methods have been proposed, among which approximate nearest neighbor retrieval has received more and more attention due to its high retrieval accuracy and low computational overhead. Among various approximate nearest neighbor retrieval methods, hash algorithm is currently the most promising method. The goal of the hash algorithm is to map high-dimensional data to a low-dimensional Hamming s...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06F16/41G06F16/45G06F16/483G06F16/901
CPCG06F16/41G06F16/45G06F16/483G06F16/9014
Inventor 张玥杰全家琦
Owner FUDAN UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap