Vector database storage and retrieval method

Abstract

The invention relates to the field of computers and artificial intelligence, in particular to a vector database storage and retrieval method and device, electronic equipment and a storage medium. The vector database storage and retrieval method comprises the steps that a data table is obtained, the data table is horizontally segmented to form a plurality of fragments, and each fragment comprises a plurality of copies; the method comprises the following steps: acquiring data, carrying out first mapping according to key data contained in the data, carrying out Hash on the key data to obtain an integer, carrying out modulo calculation and remainder calculation on a plurality of fragments according to the integer to obtain fragment numbers, then carrying out second mapping, and acquiring a plurality of copies corresponding to the fragment numbers according to the fragment numbers; wherein the fragment numbers are used for representing fragments, and each fragment corresponds to one fragment number; according to the first mapping and the second mapping, a vector storage engine on the metadata server is established for obtaining the data, and the vector storage engine is used for representing a data searching mode. The method has the advantages that the retrieval accuracy is improved, the retrieval delay is reduced, and the reliability and usability are improved.

Description

technical field [0001] The invention relates to the field of computer and artificial intelligence, in particular to the field of distributed storage technology, and in particular to a vector database storage and retrieval method. Background technique [0002] With the popularity of artificial intelligence applications, "vector similarity retrieval" as the most basic service is applied to more and more artificial intelligence businesses. Such as face recognition, voice recognition, news recommendation, etc. Most of the usage methods are to first use the neural network model to convert pictures, text, etc. into feature vectors, and then choose a distance calculation method (such as inner product distance, cosine distance, etc.). In this way, given any vector, K vectors with similar distances can be found. We call this retrieval KNN retrieval (K Nearest Neighbors). However, although there are various open source algorithm libraries for KNN retrieval, most of them only stay a...

AI Technical Summary

Problems solved by technology

[0003] Aiming at the above-mentioned defects of the prior art, the present invention provides a vector database storage and retrieval method, which solves the above-mentioned technical problems, and provides a method that can solve the problem of index sharing of the same data by multiple people and the large amount of data that leads to system performance. A Vector Database Storage and Retrieval Method for Poor Problems

Images