Large-scale data retrieval method based on improved hash learning algorithm

A large-scale data and learning algorithm technology, applied in the field of deep learning, can solve the problem of low search accuracy

Pending Publication Date: 2021-02-02
OCEAN UNIV OF CHINA
View PDF9 Cites 2 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0007] However, all the above quantization methods allocate the same number of...

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Large-scale data retrieval method based on improved hash learning algorithm
  • Large-scale data retrieval method based on improved hash learning algorithm
  • Large-scale data retrieval method based on improved hash learning algorithm

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0071] A large-scale data retrieval method based on an improved hash learning algorithm, comprising the following steps:

[0072] Step 1: where the existing projection matrix is ​​learned in spectral hashing, principal component analysis hashing, and iterative quantization, such as figure 1 and figure 2 Projection is performed as shown in the projection process in the left part of the upper half area;

[0073] Step 2: Analyze the importance of each projection dimension, and define importance as discriminative power, select a subset of projection dimensions with higher resolution, and use the minimum variance (MV) algorithm (such as figure 1 and figure 2 grouping process in the middle part of the upper half) to group them;

[0074] Step 3: For projection dimensions in the same group, which have similar discriminative power, we adaptively learn a threshold with a two-step iterative algorithm to divide them into the same number of regions;

[0075] Step 4: In the quantizati...

Embodiment 2

[0079] The specific algorithm and steps of embodiment 1 retrieval method are as follows:

[0080] Step 1: For database point x i ∈ R d , we first map it to the projected point u i ∈ R k (Such as figure 1 and figure 2 shown in the projection process on the left part of the upper half). make is an n-dimensional data point. μ represents the average value of the data, P∈R d×k Represents projection matrices learned in spectral hashing, principal component analysis hashing, and iterative quantization. for any x i ∈X, calculate the jth projection dimension:

[0081] u ij =p' j (xi -μ) (1)

[0082] where p j Representing the jth column of P, the purpose of centralizing X is to ensure that the bias in each projected dimension is based on zero.

[0083] Step 2: Analyze the importance of each projection dimension with an analytical model similar to Principal Component Analysis (PCA), and define importance as discriminative power. Let U={{u ij} n i＝1} k j＝1 ∈R n×k is...

Embodiment 3

[0117] Embodiment 3: (verification instance)

[0118] We conduct experiments on five common public datasets SIFT-10K, SIFT-1M, CIFAR10, MNIST and NUS-WIDE-SCENE. The SIFT dataset is an evaluation set, especially for nearest neighbor search applications. There are 10,000 descriptors in the SIFT database, and 35,000 training points.

[0119] The query set is a subset of the training set containing 100 points. SIFT-1M also consists of 128-D SIFT descriptors, but larger than SIFT-10K, which contains 100,000 training points, 1,000,000 database points, and 10,000 query points.

[0120] For the SIFT-10K and SIFT-1M datasets, 100 exact nearest neighbors for each query point are provided in 100×100 and 10000×100 ground-truth neighbor matrices, respectively.

[0121] CIFAR10 is a subset of the tiny image dataset. It consists of 60,000 images of 32×32 pixels divided into 10 categories: airplane, car, bird, cat, deer, dog, frog, horse, boat, and truck. The dataset is divided into a d...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention provides a large-scale data retrieval method based on an improved hash learning algorithm. The method is based on a hash learning algorithm and comprises the following steps: firstly, mapping data to a projection point to obtain a projection dimension; analyzing the importance of each projection dimension, defining the importance as discrimination, selecting a projection dimension subset with high resolution, and grouping the projection dimension subsets by using a minimum variance algorithm; for the projection dimensions in the same group, adaptively learning thresholds by usinga two-step iterative algorithm, and dividing the thresholds into regions with the same number; quantizing the regions obtained in the step S4, and replacing each region with a representative point ofthe region; and calculating a Manhattan distance between the two quantized hash codes, sorting the two quantized hash codes from small to large, finishing search, and outputting a search result. According to the invention, good performance can be kept in large-scale retrieval, and compared with an existing algorithm, the search precision can be remarkably improved.

Description

technical field [0001] The invention belongs to the technical field of deep learning, and in particular relates to a large-scale data retrieval method based on an improved hash learning algorithm. Background technique [0002] Nearest neighbor search is a fundamental problem in many applications such as machine learning, information retrieval, pattern recognition, and computer vision. In recent years we have witnessed the rise of Big Data, characterized by the prevalence of high-dimensional, ever-growing datasets. With the advent of the information age, the amount of multimedia data has increased dramatically, and efficient retrieval of multimodal data has become an urgent need. Traditional single-mode data retrieval, such as image retrieval and text retrieval, can no longer adapt to the reality of the gradual diversification of multimedia data. Given the time involved, precise nearest neighbor searches are difficult in such large datasets. As a result, a large number of ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06F16/432G06F16/22
CPCG06F16/432G06F16/2255G06F16/2219
Inventor 曹媛刘峻玮桂杰
Owner OCEAN UNIV OF CHINA
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap