A Fast and Accurate Neighborhood Retrieval Method for High-Dimensional Data Based on Euclidean Distance

Abstract

Provided is a high-dimensional data accurate neighbor quick searching method based on euclidean distance. The method includes expressing high-dimensional data into a vector form, embedding the high-dimensional data into a two-dimensional space formed by mean value and variance and meanwhile building a sampling index of the original high-dimensional data. When neighbor searching is conducted, the sampling index is first utilized to obtain a filtering threshold when a searching point is input, then the filtering threshold is utilized to filter non-neighbor data in the two-dimensional space to obtain a candidate data set, finally the distances between all candidate data points and the searching point are calculated in a linear traversal mode, and the nearest neighbor point of the searching point is calculated. The method has the advantage of being capable of quickly processing the high-dimensional data and capable of searching for the accurate neighbor point.

Description

technical field [0001] The invention relates to the fields of data processing such as information retrieval, data mining and cluster analysis, and specifically relates to indexing high-dimensional data and performing accurate neighbor query by using the upper and lower bounds of Euclidean distance and a certain data structure. Background technique [0002] With the vigorous development of information technology and the Internet, and the widespread use of multimedia digital devices, we have a massive amount of network information beyond any previous era, which contains a large amount of high-dimensional data, such as pictures, audio, video, etc. How to analyze this Fast and accurate indexing and retrieval of massive high-dimensional data is an urgent problem to be solved. [0003] An important function of indexing and retrieval is nearest neighbor query, which is to query the data most similar to the input data in the database. This is a very basic but important operation. In...

AI Technical Summary

Problems solved by technology

[0004] The traditional neighbor query algorithm has many deficiencies, such as k-dimensional tree, ball tree and other tree structures that adopt the space division strategy. They have better effect on low-dimensional data, but when the data dimension is high, the performance will decline sharply; and Algorithms for processing high-dimensional data, such as local sensitive hashing, adopt the strategy of querying approximate neighbors, which is more efficient but cannot query accurate neighbors

Images