A near similarity fast searching method based on local sensitive hash

Abstract

The invention discloses a near similarity fast searching method based on local sensitive hash. After an index structure is established, a hash value to be searched is divided into several segments andthen spliced to form several spliced segments, and the spliced segments are used as indexes to obtain the result set by nearly similar searching in the index structure. The method of the invention can greatly increase the intra-segment space of the index value, can obtain enough doubling rate on the basis of satisfying the lower original segment number and the minimum similarity of the service requirements, and greatly reduces the time cost in the acceptable space cost.

Description

technical field [0001] The invention relates to a computer algorithm, in particular to a local-sensitive hash-based near-similar fast search method. Background technique [0002] "Locality-sensitive hashing" is a fast approximate similarity search algorithm for massive high-dimensional data. In applications such as information retrieval, data mining, and recommendation systems, there is a need to find similarities in massive high-dimensional data. If you use linear search, it will become very time-consuming for high-dimensional and massive data. In order to solve such a problem, people have designed a special hash function, so that two data with high similarity can be mapped to the same or similar hash value with a high probability, while two data with low similarity can be mapped to the same or similar hash value. Mapped to the same or similar hash value with a very low probability. Such a function is called Locality Sensitive Hashing (LSH). The most fundamental role of...

AI Technical Summary

Problems solved by technology

[0005] Combined with the application scenario, it can be seen that the above segmentation algorithm has the following disadvantages: the common segmentation algorithm cannot take into account the minimum similarity and time overhead at the same time

In the case of a small number of segments, the maximum Hamming distance is small, which increases the minimum similarity, eventually resulting in a smaller range of the result set or even an empty result set; but if the number of segments is increased, although the maximum Hamming distance and the minimum The similarity can meet the business requirements, but the corresponding time overhead will also increase

Images