Cross-modal hash retrieval algorithm based on fine-grained similarity matrix

A similarity matrix and hash algorithm technology, applied in the field of cross-modal retrieval, can solve the problem that the cross-modal hash algorithm cannot mine the similarity information of data items, and achieve the effect of avoiding quantization errors.

Active Publication Date: 2022-07-22
FUDAN UNIV
View PDF5 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0006] In order to solve the problem that the existing cross-modal hash algorithm cannot mine the rich similarity information of data items in the original space, the present invention proposes a cross-modal hash retrieval algorithm based on a fine-grained similarity matrix. Chi function learning method

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Cross-modal hash retrieval algorithm based on fine-grained similarity matrix
  • Cross-modal hash retrieval algorithm based on fine-grained similarity matrix
  • Cross-modal hash retrieval algorithm based on fine-grained similarity matrix

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0060] It can be known from the background art that the existing two-stage cross-modal hashing algorithm has two main defects. First, the current two-stage hashing algorithms all use coarse-grained similarity matrices, which cannot mine the rich similarity information of data items in the original space. Second, most two-stage hash algorithms use multi-classification to train hash codes, which may not get the best hash function. Therefore, in this embodiment, for the above two problems, the similarity matrix in the fine-grained definition mode is respectively used, and the training mode of the hash function is redesigned to solve the above two problems.

[0061] In this embodiment, in the hash function learning in the second stage, for the image mode, the CNN-F network pre-trained on ImageNet is used. Keep the first five convolutional layers convl~conv5 and the next two fully connected layers fc6~fc7 unchanged, and replace the eighth fully connected layer with a new fully con...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention belongs to the technical field of cross-modal data retrieval, in particular to a cross-modal hash retrieval algorithm based on a fine-grained similarity matrix. The algorithm of the invention is mainly aimed at two tasks of image retrieval text and text retrieval image, including: hash code reasoning: using the label information of the image-text pair to construct a fine-grained similarity matrix, so that the hash code preserves the difference between the image-text data items fine-grained similarity information; construct an autoencoder so that the hash code retains the semantic information in the label as much as possible; hash function learning: train two hash functions to map images and text to hash codes respectively, ha The objective functions used in hash code learning include hash code mapping loss, weighted similarity retention loss, and classification loss. The present invention has relatively high retrieval accuracy in both tasks of image search text and text search image.

Description

technical field [0001] The invention belongs to the technical field of cross-modal retrieval, and in particular relates to a cross-modal hash retrieval algorithm based on a fine-grained similarity matrix. Background technique [0002] With the rapid development of social media, a large amount of multimedia data is generated every day, including text, images, videos, etc. Due to the high computational complexity and storage complexity, it is very difficult to perform accurate nearest neighbor retrieval on these large-scale multimedia data. To solve this problem, many alternative methods have been proposed, among which approximate nearest neighbor retrieval has received more and more attention due to its higher retrieval accuracy and lower computational cost. Among various approximate nearest neighbor retrieval methods, hash algorithm is the most potential method at present. The goal of the hash algorithm is to map high-dimensional data into a low-dimensional Hamming space. ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Patents(China)
IPC IPC(8): G06F16/41G06F16/45G06F16/483G06F16/901
CPCG06F16/41G06F16/45G06F16/483G06F16/9014
Inventor 张玥杰全家琦
Owner FUDAN UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap