Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Cross-modal hash retrieval algorithm based on fine-grained similarity matrix

A similarity matrix and hash algorithm technology, applied in the field of cross-modal retrieval, can solve the problem that the cross-modal hash algorithm cannot mine the similarity information of data items

Active Publication Date: 2021-01-08
FUDAN UNIV
View PDF5 Cites 11 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0006] In order to solve the problem that the existing cross-modal hash algorithm cannot mine the rich similarity information of data items in the original space, the present invention proposes a cross-modal hash retrieval algorithm based on a fine-grained similarity matrix. Chi function learning method

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Cross-modal hash retrieval algorithm based on fine-grained similarity matrix
  • Cross-modal hash retrieval algorithm based on fine-grained similarity matrix
  • Cross-modal hash retrieval algorithm based on fine-grained similarity matrix

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0060] It can be known from the background technology that there are two main defects in the existing cross-modal hash algorithm based on two stages. First, the current two-stage hashing algorithm uses a coarse-grained similarity matrix, which cannot mine the rich similarity information of data items in the original space. Second, most two-stage hash algorithms use multi-classification methods to train hash codes, and may not get the best hash function. Therefore, for the above two problems, this embodiment uses a similarity matrix defined in a fine-grained manner, and redesigns a hash function training method to solve the above two problems.

[0061] In this embodiment, in the second stage of hash function learning, for the image modality, the CNN-F network pre-trained on ImageNet is used. Keep the first five convolutional layers convl~conv5 and the next two fully connected layers fc6~fc7 unchanged, replace the eighth fully connected layer with a new fully connected layer co...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention belongs to the technical field of cross-modal data retrieval, and particularly relates to a cross-modal hash retrieval algorithm based on a fine-grained similarity matrix. The algorithmprovided by the invention mainly aims at two tasks of image retrieval texts and text retrieval images, and comprises the following steps: hash code reasoning: constructing a fine-grained similarity matrix by utilizing label information of image text pairs, so that hash codes reserve fine-grained similarity information between image text data items; constructing an auto-encoder to enable the hash code to reserve semantic information in the label as much as possible; hash function learning: training two Hash functions, mapping images and texts to Hash codes respectively, wherein target functionsused by Hash code learning include Hash code mapping loss, similarity retention loss with weight and classification loss. The invention has relatively high retrieval precision in two tasks of image search texts and text search images.

Description

technical field [0001] The invention belongs to the technical field of cross-modal retrieval, and in particular relates to a cross-modal hash retrieval algorithm based on a fine-grained similarity matrix. Background technique [0002] With the rapid development of social media, a large amount of multimedia data is generated every day, including text, images, videos, etc. Limited by the high computational complexity and storage complexity, it becomes very difficult to perform accurate nearest neighbor retrieval on these large-scale multimedia data. To solve this problem, many alternative methods have been proposed, among which approximate nearest neighbor retrieval has received more and more attention due to its high retrieval accuracy and low computational overhead. Among various approximate nearest neighbor retrieval methods, hash algorithm is currently the most promising method. The goal of the hash algorithm is to map high-dimensional data to a low-dimensional Hamming s...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
IPC IPC(8): G06F16/41G06F16/45G06F16/483G06F16/901
CPCG06F16/41G06F16/45G06F16/483G06F16/9014
Inventor 张玥杰全家琦
Owner FUDAN UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com