Sample set processing method and device and sample query method and device

Abstract

Embodiments of the invention provide a method and device for carrying out classified processing and index processing on sample sets and a method and device for querying similar samples. In the classified processing, two stages of clustering are carried out on samples in a sample set, and clustering results are recorded in a first vector table and a second vector table. In the index processing, twostages of indexes are established for each sample in the sample set, the first stage of index points to a coarse clustering center to which the sample belongs, and the second stage of index points toa segmented clustering center corresponding to a segmented vector of the sample. In a process of querying similar samples, two stages of retrieval are carried out on queried samples. In the first stage of retrieval, coarse clustering centers to the queried samples are determined from the first vector table, and comparison samples belonging to the coarse clustering centers are obtained. And in thesecond stage of retrieval, the comparison samples, distances of which satisfy a predetermined condition, are taken as similar samples. In such a way, rapid retrieval and query of samples are realized.

Description

technical field [0001] One or more embodiments of this specification relate to the field of computer technology, and in particular to preprocessing of sample sets, and sample query methods and devices. Background technique [0002] With the upgrading of the Internet, people increasingly use the Internet to carry out more searches and inquiries. For example, people are very accustomed to using various search engines to search for interesting content. At the same time, people's search and query objects are becoming more and more complex. For example, searching for text keywords gradually develops into searching for pictures, searching for music, and so on. The difficulty of searching increases exponentially as the objects to search and query become more complex. First, complex objects usually need to be represented by high-dimensional vectors. Therefore, in the process of searching, it is usually necessary to compare the distance or similarity between many high-dimensional ...

AI Technical Summary

Problems solved by technology

When there is a large amount of data in the sample database to be retrieved, if a brute force search is used to perform high-dimensional vector calculations for each sample, the amount of calculation is very large, resulting in too long query time, making it difficult to meet user needs

Images