Vector query method and device, electronic equipment and storage medium

Abstract

The invention provides a vector query method and device, electronic equipment and a storage medium, and the method comprises the steps: carrying out the clustering processing of a plurality of data sets corresponding to a query vector, so as to obtain a plurality of vector clusters of each data set, and each vector cluster is provided with a cluster center; for each data set in the multiple data sets, p neighbor cluster centers and the similarity between the neighbor cluster centers and the query vectors are determined in cluster centers of multiple vector clusters of the data sets according to the query vectors, and p is an integer larger than 0; according to the weight of each data set in the multiple data sets and the similarity between p neighbor cluster centers of each data set and the query vector, q target cluster centers are determined in all neighbor cluster centers of the multiple data sets, and q is an integer larger than 0 and smaller than the number of all neighbor cluster centers of the multiple data sets; k neighbor vectors are determined in the vector clusters to which the q target cluster centers belong according to the query vector, and k is an integer larger than 0.

Description

Technical Field

[0001] One or more embodiments of the present specification relate to the field of database technology, and in particular, to a vector query method and device, an electronic device, and a storage medium. Background Art

[0002] With the rise of large language models, the demand for DB4AI (Database for AI) continues to grow. This is largely due to the need to convert sample data into vector form during AI (Artificial Intelligence) training and reasoning, which creates a need to manage these vectors. Currently, there is some research and development focused on creating database systems that specifically handle vector data types, such as Milvus; there are also some attempts to extend the functionality of traditional relational databases by adding plug-ins that support vector data types.

[0003] In the related art, whether it is a database system that specializes in processing vector data types or a traditional relational database with added plug-ins, the efficiency of performing vector queries still needs to be improved. Summary of the invention

[0004] In view of this, one or more embodiments of the present specification provide a vector query method and device, an electronic device, and a storage medium.

[0005] To achieve the above objectives, one or more embodiments of this specification provide the following technical solutions: According to a first aspect of one or more embodiments of this specification, a vector query method is proposed, the method comprising: Clustering the multiple data sets targeted by the query vector respectively to obtain multiple vector clusters of each data set, wherein the vector cluster has a cluster center; For each of the multiple data sets, determining p neighboring cluster centers among cluster centers of multiple vector clusters of the data set according to the query vector, and similarities between the neighboring cluster centers and the query vector, where p is an integer greater than 0; According to the weight of each data set in the multiple data sets and the similarity between the p nearest neighbor cluster centers of each data set and the query vector, q target cluster centers are determined from all the nearest neighbor cluster centers of the multiple data sets, where q is an integer greater than 0 and less than the number of all the nearest neighbor cluster centers of the multiple data sets; According to the query vector, k nearest neighbor vectors are determined in the vector clusters to which the q target cluster centers belong, where k is an integer greater than 0.

[0006] In one or more embodiments of the present specification, determining p neighboring cluster centers among cluster centers of multiple vector clusters in the data set according to the query vector, and the similarity between the neighboring cluster centers and the query vector, includes: respectively determining similarities between the query vector and cluster centers of a plurality of vector clusters of the data set; Among the cluster centers of the multiple vector clusters in the data set, p cluster centers having the highest similarity to the query vector are determined as neighbor cluster centers.

[0007] In one or more embodiments of the present specification, determining q target cluster centers from all neighbor cluster centers of the multiple data sets according to the weight of each data set in the multiple data sets and the similarity between the p neighbor cluster centers of each data set and the query vector includes: For each of the multiple data sets, respectively modifying the similarity between the p nearest neighbor cluster centers of the data set and the query vector according to the weight of the data set; Among all the nearest cluster centers of the multiple data sets, q nearest cluster centers having the highest similarity to the query vector are determined as target cluster centers.

[0008] In one or more embodiments of the present specification, q is equal to p.

[0009] In one or more embodiments of this specification, the method further includes: For each neighbor cluster center among all neighbor cluster centers of the multiple data sets, determining a primary key cluster corresponding to the vector cluster to which the neighbor cluster center belongs, and determining a cluster center of the primary key cluster; Among all the neighbor cluster centers of the multiple data sets, if the similarity between the cluster centers of the primary key clusters corresponding to the vector clusters to which any two neighbor cluster centers belong is greater than a preset threshold, the vector clusters to which the two neighbor cluster centers belong are merged, and the two neighbor cluster centers are merged into multiple cluster centers.

[0010] In one or more embodiments of the present specification, determining q target cluster centers from all neighbor cluster centers of the multiple data sets according to the weight of each data set in the multiple data sets and the similarity between the p neighbor cluster centers of each data set and the query vector includes: For each neighbor cluster center that has not been merged among all the neighbor cluster centers of the multiple data sets, modifying the similarity between the neighbor cluster center and the query vector according to the weight of the data set to which the neighbor cluster center belongs; For each multi-cluster center obtained by merging all the nearest cluster centers of the multiple data sets, determining the similarity of the multi-cluster centers according to the weights of the data sets to which the two nearest cluster centers corresponding to the multi-cluster center belong and the similarity between the two nearest cluster centers corresponding to the multi-cluster center and the query vector; Among all the unmerged neighbor cluster centers and all the merged multi-cluster centers of the multiple data sets, q neighbor cluster centers and / or multi-cluster centers having the highest similarity to the query vector are determined as target cluster centers.

[0011] In one or more embodiments of the present specification, determining k nearest neighbor vectors in the vector cluster to which the centers of the q target clusters belong according to the query vector, where k is an integer greater than 0, includes: Determine the similarity between the query vector and each vector in the vector cluster to which the q target cluster centers belong; Among the vector clusters to which the centers of the q target clusters belong, k vectors having the highest similarity to the query vector are determined as neighbor vectors.

[0012] In one or more embodiments of this specification, the method further includes: According to the primary keys corresponding to the k nearest neighbor vectors, n nearest neighbor primary keys are determined.

[0013] According to a second aspect of one or more embodiments of this specification, a vector query device is provided, the device comprising: A clustering module, used for clustering the multiple data sets targeted by the query vector respectively to obtain multiple vector clusters of each data set, wherein the vector cluster has a cluster center; A first screening module is used to determine, for each of the multiple data sets, p nearest neighbor cluster centers among cluster centers of multiple vector clusters of the data set according to the query vector, and similarities between the nearest neighbor cluster centers and the query vector, wherein p is an integer greater than 0; A second screening module is used to determine q target cluster centers from all the neighbor cluster centers of the multiple data sets according to the weight of each data set in the multiple data sets and the similarity between the p neighbor cluster centers of each data set and the query vector, wherein q is an integer greater than 0 and less than the number of all the neighbor cluster centers of the multiple data sets; A query module is used to determine k nearest neighbor vectors in the vector clusters to which the centers of the q target clusters belong according to the query vector, wherein k is an integer greater than 0.

[0014] According to a third aspect of one or more embodiments of this specification, a computer program product is provided, comprising a computer program / instruction, which implements the steps of the method described in the first aspect when executed by a processor.

[0015] According to a fourth aspect of one or more embodiments of this specification, an electronic device is provided, including: processor; a memory for storing processor-executable instructions; The processor implements the method as described in the first aspect by running the executable instructions.

[0016] According to a fifth aspect of one or more embodiments of the present specification, a computer-readable storage medium is provided, on which computer instructions are stored, and when the instructions are executed by a processor, the steps of the method described in the first aspect are implemented.

[0017] The technical solutions provided by the embodiments of this specification may have the following beneficial effects: The vector query method provided in the embodiment of the specification first performs clustering processing on multiple data sets targeted by the query vector to obtain multiple vector clusters of each data set, then for each data set in the multiple data sets, according to the query vector, p nearest cluster centers are determined in the cluster centers of the multiple vector clusters of the data set, and the similarity between the nearest cluster centers and the query vector, then according to the weight of each data set in the multiple data sets and the similarity between the p nearest cluster centers of each data set and the query vector, q target cluster centers are determined in all the nearest cluster centers of the multiple data sets, and finally, according to the query vector, k nearest vectors are determined in the vector clusters to which the q target cluster centers belong, where k is an integer greater than 0. The method divides the data set into multiple vector clusters by clustering, and then the similarity with the query vector can be measured by the cluster centers of the vector clusters to screen the vector clusters, and then vector query is performed in the screened vector clusters, thereby reducing the query range and improving the query efficiency; and when the query vector targets multiple data sets, the method further screens the vector clusters screened out of each data set based on the weight of the data set, thereby further narrowing the query range and improving the query efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 It is an architectural diagram of a distributed database provided by an exemplary embodiment.

[0019] Figure 2 It is a flowchart of a vector query method provided by an exemplary embodiment.

[0020] Figure 3It is a flowchart of a vector cluster merging method provided by an exemplary embodiment.

[0021] Figure 4 It is a structural schematic diagram of a device provided by an exemplary embodiment.

[0022] Figure 5 It is a block diagram of a vector query device provided by an exemplary embodiment. DETAILED DESCRIPTION

[0023] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with one or more embodiments of this specification. Instead, they are merely examples of devices and methods consistent with some aspects of one or more embodiments of this specification as detailed in the appended claims.

[0024] It should be noted that: in other embodiments, the steps of the corresponding method are not necessarily performed in the order shown and described in this specification. In some other embodiments, the steps included in the method may be more or less than those described in this specification. In addition, a single step described in this specification may be decomposed into multiple steps for description in other embodiments; and multiple steps described in this specification may be combined into a single step for description in other embodiments.

[0025] First, some concepts involved in this specification are explained.

[0026] Vector database: refers to a database system specifically designed to store, manage, and retrieve vector data. Vector data usually refers to high-dimensional data, such as mathematical representations of images, audio, video, and text content, which are usually converted into high-dimensional vectors through deep learning models.

[0027] DB4AI (Database for Artificial Intelligence): is a field that involves the combination of database technology and artificial intelligence (AI) applications. It focuses on integrating artificial intelligence algorithms and machine learning (ML) models in the database to achieve data analysis, prediction, and automated decision-making directly within the database environment.

[0028] Embedding: In the process of AI training and reasoning, "embedding" refers to converting large-scale sparse data (such as words, phrases, or entire files in text) into a low-dimensional dense vector representation. This vectorized expression can capture and represent the important features of the data and is used for the subsequent training and reasoning of machine learning models.

[0029] ANNS (Approximate Nearest Neighbor Search): Compared with k-NNS (k Nearest Neighbor Search), ANNS sacrifices a certain degree of accuracy to obtain higher recall performance and accelerates the search by establishing indexes or quantization.

[0030] Based on the technical problems mentioned in the background technology, at least one embodiment of the present specification provides a vector query method, which can be applied to a database system that specifically processes vector data types or a traditional relational database with added plug-ins, and can efficiently complete vector queries, especially in scenarios where vector queries are performed on multiple data sets.

[0031] Next, we will give an example of vector query to describe the scenario more vividly. In addition, we can also combine this example to provide a more detailed and vivid description of the process of this method below. For the convenience of the following description, this example is called the product search example. The content of the product search example is as follows: The database has a data table for storing the image features of each product in the product set. The data table contains a key value column, a front view feature column, and a side view feature column. The key value column is used to store the product identification, such as the product ID, the front view feature column is used to store the front view features of the product, and the side view feature column is used to store the side view features of the product; that is, each row of records in the data table contains a product identification, and the front view features and side view features of the product corresponding to the product identification. The content of the vector query task is: based on the given product features (such as features extracted from the given product image), perform vector retrieval in the above database to determine the product to which the given product features belong, that is, the target product.

[0032] For example, the method can be applied to Figure 1 The distributed database shown, or other types of databases.

[0033] Please refer to the attached Figure 2 , which exemplarily shows a flowchart of a vector query method, including steps S201 to S204.

[0034] In step S201 , clustering is performed on multiple data sets targeted by the query vector to obtain multiple vector clusters for each data set, wherein the vector cluster has a cluster center.

[0035] In a vector query task, the query vector may target one data set or multiple data sets, and the objects to which the vectors in the multiple data sets belong may partially or completely overlap. For example, in a vector query task, the query vector may target one vector column in a data table or multiple vector columns in a data table, and the primary keys corresponding to the multiple vector columns may partially or completely overlap. For example, the vector query task in the above product retrieval example includes a query vector of a given product feature. The data set corresponding to the given product feature is two vector columns, the front view feature column and the side view feature column. The primary key corresponding to the front view feature in the front view feature column, the product identifier, overlaps with the primary key corresponding to the side view feature in the side view feature column, the product identifier.

[0036] Vector query tasks can be represented in the form of SQL statements. For example, the above product retrieval example can be represented by the following SQL statement: select c1 and c2<->"[1,2,3]" limit 5 from item; where [1,2,3] represents the given product features, column c1 is the front view feature column, and column c2 is the side view feature column. This SQL statement means: the product to which the given product features belong is locked by retrieving 5 features similar to the given product features in the front view feature set and the side view feature set. It should be understood that when executing the SQL statement involving multiple columns in this example, strategies such as IndexMerge can be used to improve query efficiency, such as integrating the vector-specific Index Merge logic in the ObIvfflatIndexSearchHelper function to optimize the cluster center selection process, or designing a new operator ObIvfflatMultiAnnScanOp to execute the query logic of this method.

[0037] Taking the above product search example as an example, the front view feature column and the side view feature column can be clustered to obtain multiple feature clusters of the front view feature column and multiple feature clusters of the side view feature column.

[0038] In step S202, for each of the multiple data sets, p neighbor cluster centers are determined in cluster centers of multiple vector clusters of the data set according to the query vector, as well as similarities between the neighbor cluster centers and the query vector, where p is an integer greater than 0.

[0039] Exemplarily, first, the similarity between the query vector and the cluster centers of the multiple vector clusters of the data set is determined respectively; then, among the cluster centers of the multiple vector clusters of the data set, the p cluster centers with the highest similarity to the query vector are determined as the neighboring cluster centers. The similarity can be represented by Euclidean distance or the like.

[0040] Among them, P can be preset and can be changed.

[0041] Taking the above-mentioned product retrieval example as an example, the cluster centers of multiple feature clusters of the front view feature column and the cluster centers of multiple feature clusters of the side view feature column can be screened respectively to obtain p neighbor cluster centers of the front view feature column and p neighbor cluster centers of the side view feature column.

[0042] In step S203, based on the weight of each data set in the multiple data sets and the similarity between the p nearest neighbor cluster centers of each data set and the query vector, q target cluster centers are determined from all the nearest neighbor cluster centers of the multiple data sets, where q is an integer greater than 0 and less than the number of all the nearest neighbor cluster centers of the multiple data sets.

[0043] Preferably, q is equal to p. In other words, after this step, the number of target cluster centers remaining is equal to the number of neighboring cluster centers of a single data set, which is equivalent to returning to the query range of a data set.

[0044] The weight of each data set may be pre-set on the data set; or may be dynamically set each time a vector query is executed. For example, when a vector query is defined using an SQL statement, the weight of each data set may be dynamically annotated in the SQL statement.

[0045] Exemplarily, q target cluster centers may be determined in the following manner: First, for each of the multiple data sets, the similarities between the p nearest cluster centers of the data set and the query vector are corrected according to the weight of the data set. For example, the similarity between each nearest cluster center and the query vector is multiplied by the weight of the data set to which the nearest cluster center belongs, so as to correct the similarity between the nearest cluster center and the query vector.

[0046] Next, among all the neighbor cluster centers of the multiple data sets, the q neighbor cluster centers with the highest similarity to the query vector are determined as the target cluster centers. For example, all the neighbor cluster centers of the multiple data sets are sorted from large to small according to the modified results of the similarity to the query vector, and the top q neighbor cluster centers in the sorting are used as the target cluster centers.

[0047] Taking the above-mentioned product retrieval example as an example, the similarities between the p nearest neighbor cluster centers of the front view feature column and the query vector can be corrected based on the weight of the front view feature column, and the similarities between the p nearest neighbor cluster centers of the side view feature column and the query vector can be corrected based on the weight of the side view feature column; the 2p nearest neighbor cluster centers of the two feature columns are sorted from large to small according to the modified results of the similarity, and the top q nearest neighbor cluster centers in the sort are determined as the target cluster centers.

[0048] In step S204, k nearest neighbor vectors are determined in the vector clusters to which the q target cluster centers belong according to the query vector, where k is an integer greater than 0.

[0049] Wherein, k can be a fixed value that can be preset in P; or its value can be dynamically set each time a vector query is executed. For example, when a vector query is defined in an SQL statement, the number of neighbor vectors, i.e., the value of k, can be dynamically marked in the SQL statement. For example, in the SQL statement mentioned in the above embodiment: select c1 and c2<->"[1,2,3]" limit 5 from item, the value of k is 5.

[0050] Exemplarily, this step can be performed in the following manner: First, the similarity between the query vector and each vector in the vector cluster to which the q target cluster centers belong is determined.

[0051] Next, among the vector clusters to which the centers of the q target clusters belong, k vectors having the highest similarity to the query vector are determined as neighbor vectors.

[0052] In other words, this step uses the vector clusters to which the centers of the q target clusters belong as the query range, and retrieves the k vectors with the highest similarity to the query vector as neighbor vectors.

[0053] It should be understood that the method may further include: determining n nearest neighbor primary keys according to the primary keys corresponding to the k nearest neighbor vectors.

[0054] For example: according to the primary key corresponding to the neighbor vector, k neighbor vectors are clustered to obtain a neighbor vector set of at least one primary key, wherein each neighbor vector set includes at least one neighbor vector; for each neighbor vector set in the neighbor vector set of the at least one primary key, at least one neighbor vector in the neighbor vector set is weightedly summed according to the weight of the data set to which it belongs, to obtain a weighted sum result of the neighbor vector set; in the neighbor vector set of the at least one primary key, the primary key to which the n neighbor vector sets with the highest weighted sum result belong is determined as the neighbor primary key.

[0055] Preferably, n is 1. This can lock a unique product query result.

[0056] Taking the above-mentioned product retrieval example as an example, the k nearest neighbor vectors with the highest similarity to the given product feature can be retrieved from all the features of the front view feature cluster / side view feature cluster to which the centers of q target clusters belong; and the k nearest neighbor vectors are clustered according to the product identification to obtain a nearest neighbor vector set of at least one product identification; the front view features and / or side view features in each nearest neighbor vector set are weightedly summed according to the weight of the front view feature column and the weight of the side view feature column to obtain a weighted summation result; the nearest neighbor vector set of each product identification is sorted based on the weighted summation result, and the top n product identifications in the sort are taken as the target products to which the given product feature belongs.

[0057] The vector query method provided in the embodiment of the specification first performs clustering processing on multiple data sets targeted by the query vector to obtain multiple vector clusters of each data set, then for each data set in the multiple data sets, according to the query vector, p nearest cluster centers are determined in the cluster centers of the multiple vector clusters of the data set, and the similarity between the nearest cluster centers and the query vector, then according to the weight of each data set in the multiple data sets and the similarity between the p nearest cluster centers of each data set and the query vector, q target cluster centers are determined in all the nearest cluster centers of the multiple data sets, and finally, according to the query vector, k nearest vectors are determined in the vector clusters to which the q target cluster centers belong, where k is an integer greater than 0. The method divides the data set into multiple vector clusters by clustering, and then the similarity with the query vector can be measured by the cluster centers of the vector clusters to screen the vector clusters, and then vector query is performed in the screened vector clusters, thereby reducing the query range and improving the query efficiency; and when the query vector targets multiple data sets, the method further screens the vector clusters screened out of each data set based on the weight of the data set, thereby further narrowing the query range and improving the query efficiency.

[0058] In some embodiments of the present disclosure, after obtaining p neighbor cluster centers of each data set and the similarity between the neighbor cluster centers and the query vector, the method may also include the following: Figure 3 The following steps are shown: Step S301: for each neighbor cluster center among all neighbor cluster centers of the multiple data sets, determine the primary key cluster corresponding to the vector cluster to which the neighbor cluster center belongs, and determine the cluster center of the primary key cluster.

[0059] For example, a cluster consisting of the primary key corresponding to each vector in the vector cluster to which the center of the neighbor cluster belongs (that is, the primary key in the data row where the vector is located) is determined as the corresponding primary key cluster.

[0060] In step S302: among all the neighbor cluster centers of the multiple data sets, if the similarity between the cluster centers of the primary key clusters corresponding to the vector clusters to which any two neighbor cluster centers belong is greater than a preset threshold, the vector clusters to which the two neighbor cluster centers belong are merged, and the two neighbor cluster centers are merged into multiple cluster centers.

[0061] If the similarity between the cluster centers of the primary key clusters corresponding to the vector clusters to which any two neighbor cluster centers belong is greater than the preset threshold, the vector primary keys in the vector clusters to which the two neighbor cluster centers belong overlap, so they are merged. In this way, all vectors with the same primary key can be retained when the target cluster center is subsequently screened to ensure the retrieval accuracy of neighbor vectors and neighbor primary keys.

[0062] Based on the operation of merging vector clusters shown in this embodiment, the following Figure 2 The step S203 shown can be performed in the following manner: First, for each neighbor cluster center that has not been merged among all the neighbor cluster centers of the multiple data sets, the similarity between the neighbor cluster center and the query vector is corrected according to the weight of the data set to which the neighbor cluster center belongs.

[0063] For example, the similarity between each neighbor cluster center that has not been merged and the query vector is multiplied by the weight of the data set to which the neighbor cluster center belongs, so as to correct the similarity between the neighbor cluster center and the query vector.

[0064] Next, for each multi-cluster center obtained by merging all the neighboring cluster centers of the multiple data sets, the similarity of the multi-cluster centers is determined according to the weights of the data sets to which the two neighboring cluster centers corresponding to the multi-cluster center belong, and the similarity between the two neighboring cluster centers corresponding to the multi-cluster center and the query vector.

[0065] For example, according to the weights of the data sets to which the two neighboring cluster centers corresponding to the multi-cluster center belong, a weighted sum is performed on the similarities between the two neighboring cluster centers corresponding to the multi-cluster center and the query vector, and the weighted sum result is used as the similarity between the multi-cluster center and the query vector.

[0066] Finally, among all the unmerged neighbor cluster centers of the multiple data sets and all the merged multi-cluster centers, q neighbor cluster centers and / or multi-cluster centers having the highest similarity to the query vector are determined as target cluster centers.

[0067] For example, all the neighbor cluster centers that have not been merged and all the multi-cluster centers that have been merged of the multiple data sets are sorted from large to small according to the similarity with the query vector, and the top q neighbor cluster centers / multi-cluster centers in the sorting are used as the target cluster centers. The similarity of the neighbor cluster centers that have not been merged during sorting uses the corrected result.

[0068] Taking the above-mentioned product retrieval example as an example, the product identification cluster and cluster center corresponding to the feature cluster to which the p nearest cluster centers of the front view feature column belong can be determined, and the product identification cluster and cluster center corresponding to the feature cluster to which the p nearest cluster centers of the side view feature column belong can be determined; if the similarity between the cluster center of the product identification cluster corresponding to the feature cluster to which a certain nearest cluster center of the front view feature column belongs and the product identification cluster and cluster center corresponding to the feature cluster to which a certain nearest cluster center of the side view feature column belongs is greater than a preset threshold, then the feature cluster of the front view feature is merged with the feature cluster of the side view feature, and the two cluster centers are merged into multiple cluster centers; based on the weight of the front view feature column, each nearest cluster center of the front view feature column that has not been merged is merged into multiple cluster centers. The similarity between the neighbor cluster center and the query vector is corrected, and the similarity between each unmerged neighbor cluster center of the side view feature column and the query vector is corrected based on the weight of the side view feature column. According to the weight of the front view feature column and the weight of the side view feature column, the cluster center of the front view feature cluster and the cluster center of the side view feature cluster corresponding to each multi-cluster center are weighted summed, and the weighted sum result is used as the similarity between the multi-cluster center and the query vector; all the unmerged neighbor cluster centers of the multiple data sets and all the merged multi-cluster centers are sorted from large to small according to the similarity with the query vector, and the top q neighbor cluster centers / multi-cluster centers in the sorting are used as the target cluster center. Among them, the similarity of the unmerged neighbor cluster centers during sorting adopts its correction result.

[0069] Figure 4 is a schematic structural diagram of a device provided by an exemplary embodiment. Figure 4 At the hardware level, the device includes a processor 402, an internal bus 404, a network interface 406, a memory 408, and a non-volatile memory 410, and may also include hardware required for other tasks. One or more embodiments of this specification may be implemented based on software, such as the processor 402 reading the corresponding computer program from the non-volatile memory 410 into the memory 408 and then running it. Of course, in addition to the software implementation, one or more embodiments of this specification do not exclude other implementations, such as logic devices or a combination of software and hardware, etc., that is, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0070] Please refer to Figure 5 , the vector query device can be applied to Figure 4 The device shown in the figure can realize the technical solution of this specification. The vector query device can include: A clustering module 501 is used to perform clustering processing on the multiple data sets targeted by the query vector respectively to obtain multiple vector clusters of each data set, wherein the vector cluster has a cluster center; A first screening module 502 is used to determine, for each of the multiple data sets, p neighboring cluster centers among cluster centers of multiple vector clusters of the data set according to the query vector, and similarities between the neighboring cluster centers and the query vector, wherein p is an integer greater than 0; A second screening module 503 is used to determine q target cluster centers from all the neighbor cluster centers of the multiple data sets according to the weight of each data set in the multiple data sets and the similarity between the p neighbor cluster centers of each data set and the query vector, where q is an integer greater than 0 and less than the number of all the neighbor cluster centers of the multiple data sets; The query module 504 is configured to determine, according to the query vector, k nearest neighbor vectors in the vector clusters to which the centers of the q target clusters belong, where k is an integer greater than 0.

[0071] In one or more embodiments of this specification, the first screening module is used to: respectively determining similarities between the query vector and cluster centers of a plurality of vector clusters of the data set; Among the cluster centers of the multiple vector clusters in the data set, p cluster centers having the highest similarity to the query vector are determined as neighbor cluster centers.

[0072] In one or more embodiments of this specification, the second screening module is used to: For each of the multiple data sets, respectively modifying the similarity between the p nearest neighbor cluster centers of the data set and the query vector according to the weight of the data set; Among all the nearest cluster centers of the multiple data sets, q nearest cluster centers having the highest similarity to the query vector are determined as target cluster centers.

[0073] In one or more embodiments of the present specification, q is equal to p.

[0074] In one or more embodiments of the present specification, the device further includes a merging module, configured to: For each neighbor cluster center among all neighbor cluster centers of the multiple data sets, determining a primary key cluster corresponding to the vector cluster to which the neighbor cluster center belongs, and determining a cluster center of the primary key cluster; Among all the neighbor cluster centers of the multiple data sets, if the similarity between the cluster centers of the primary key clusters corresponding to the vector clusters to which any two neighbor cluster centers belong is greater than a preset threshold, the vector clusters to which the two neighbor cluster centers belong are merged, and the two neighbor cluster centers are merged into multiple cluster centers.

[0075] In one or more embodiments of this specification, the second screening module is used to: For each neighbor cluster center that has not been merged among all the neighbor cluster centers of the multiple data sets, modifying the similarity between the neighbor cluster center and the query vector according to the weight of the data set to which the neighbor cluster center belongs; For each multi-cluster center obtained by merging all the nearest cluster centers of the multiple data sets, determining the similarity of the multi-cluster centers according to the weights of the data sets to which the two nearest cluster centers corresponding to the multi-cluster center belong and the similarity between the two nearest cluster centers corresponding to the multi-cluster center and the query vector; Among all the unmerged neighbor cluster centers and all the merged multi-cluster centers of the multiple data sets, q neighbor cluster centers and / or multi-cluster centers having the highest similarity to the query vector are determined as target cluster centers.

[0076] In one or more embodiments of this specification, the query module is used to: Determine the similarity between the query vector and each vector in the vector cluster to which the q target cluster centers belong; Among the vector clusters to which the centers of the q target clusters belong, k vectors having the highest similarity to the query vector are determined as neighbor vectors.

[0077] In one or more embodiments of the present specification, the device further includes a master key module, configured to: According to the primary keys corresponding to the k nearest neighbor vectors, n nearest neighbor primary keys are determined.

[0078] One or more embodiments of the present specification also propose a computer program product, including a computer program / instruction, which implements the steps of the method provided in the first aspect when executed by a processor.

[0079] One or more embodiments of the present specification also propose a computer-readable storage medium having computer instructions stored thereon, which, when executed by a processor, implement the steps of the method described in the first aspect.

[0080] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer, which may be in the form of a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email transceiver, a game console, a tablet computer, a wearable device or a combination of any of these devices.

[0081] In a typical configuration, a computer includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0082] Memory may include non-permanent storage in a computer-readable medium, in the form of random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0083] Computer readable media include permanent and non-permanent, removable and non-removable media that can be used to store information by any method or technology. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, disk storage, quantum memory, graphene-based storage media or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0084] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0085] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0086] The terms used in one or more embodiments of this specification are only for the purpose of describing specific embodiments, and are not intended to limit one or more embodiments of this specification. The singular forms of "a", "said" and "the" used in one or more embodiments of this specification and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0087] The user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this manual are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0088] It should be understood that although the terms first, second, third, etc. may be used to describe various information in one or more embodiments of this specification, these information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of one or more embodiments of this specification, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0089] The above description is merely a preferred embodiment of one or more embodiments of the present specification and is not intended to limit one or more embodiments of the present specification. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of one or more embodiments of the present specification shall be included in the scope of protection of one or more embodiments of the present specification.

Claims

1. A vector query method, the method comprising: Performing clustering processing on the multiple data sets targeted by the query vector respectively to obtain multiple vector clusters of each data set, wherein the vector cluster has a cluster center; For each of the multiple data sets, determining p neighboring cluster centers among cluster centers of multiple vector clusters of the data set according to the query vector, and similarities between the neighboring cluster centers and the query vector, where p is an integer greater than 0; According to the weight of each data set in the multiple data sets and the similarity between the p nearest neighbor cluster centers of each data set and the query vector, q target cluster centers are determined from all the nearest neighbor cluster centers of the multiple data sets, where q is an integer greater than 0 and less than the number of all the nearest neighbor cluster centers of the multiple data sets; According to the query vector, k nearest neighbor vectors are determined in the vector clusters to which the q target cluster centers belong, where k is an integer greater than 0.

2. The vector query method according to claim 1, wherein the determining, according to the query vector, p nearest neighbor cluster centers among the cluster centers of the plurality of vector clusters in the data set, and the similarity between the nearest neighbor cluster centers and the query vector comprises: respectively determining similarities between the query vector and cluster centers of a plurality of vector clusters of the data set; Among the cluster centers of the multiple vector clusters in the data set, p cluster centers having the highest similarity to the query vector are determined as neighbor cluster centers.

3. The vector query method according to claim 1, wherein determining q target cluster centers from all the neighbor cluster centers of the multiple data sets according to the weight of each data set in the multiple data sets and the similarity between the p neighbor cluster centers of each data set and the query vector comprises: For each of the multiple data sets, respectively modifying the similarity between the p nearest neighbor cluster centers of the data set and the query vector according to the weight of the data set; Among all the nearest cluster centers of the multiple data sets, q nearest cluster centers having the highest similarity to the query vector are determined as target cluster centers.

4. According to the vector query method described in claim 1 or 3, q ​​is equal to p.

5. The vector query method according to claim 1, further comprising: For each neighbor cluster center among all neighbor cluster centers of the multiple data sets, determining a primary key cluster corresponding to the vector cluster to which the neighbor cluster center belongs, and determining a cluster center of the primary key cluster; Among all the neighbor cluster centers of the multiple data sets, if the similarity between the cluster centers of the primary key clusters corresponding to the vector clusters to which any two neighbor cluster centers belong is greater than a preset threshold, the vector clusters to which the two neighbor cluster centers belong are merged, and the two neighbor cluster centers are merged into multiple cluster centers.

6. The vector query method according to claim 5, wherein the step of determining q target cluster centers from all the neighbor cluster centers of the multiple data sets according to the weight of each data set in the multiple data sets and the similarity between the p neighbor cluster centers of each data set and the query vector comprises: For each neighbor cluster center that has not been merged among all the neighbor cluster centers of the multiple data sets, modifying the similarity between the neighbor cluster center and the query vector according to the weight of the data set to which the neighbor cluster center belongs; For each multi-cluster center obtained by merging all the nearest cluster centers of the multiple data sets, determining the similarity of the multi-cluster centers according to the weights of the data sets to which the two nearest cluster centers corresponding to the multi-cluster center belong and the similarity between the two nearest cluster centers corresponding to the multi-cluster center and the query vector; Among all the unmerged neighbor cluster centers and all the merged multi-cluster centers of the multiple data sets, q neighbor cluster centers and / or multi-cluster centers having the highest similarity to the query vector are determined as target cluster centers.

7. The vector query method according to claim 1, wherein k nearest neighbor vectors are determined in the vector clusters to which the centers of the q target clusters belong based on the query vector, wherein: k is an integer greater than 0, including: Determine the similarity between the query vector and each vector in the vector cluster to which the q target cluster centers belong; Among the vector clusters to which the centers of the q target clusters belong, k vectors having the highest similarity to the query vector are determined as neighbor vectors.

8. The vector query method according to claim 1 or 7, further comprising: According to the primary keys corresponding to the k nearest neighbor vectors, n nearest neighbor primary keys are determined.

9. A vector query device, comprising: A clustering module, used for clustering the multiple data sets targeted by the query vector respectively to obtain multiple vector clusters of each data set, wherein the vector cluster has a cluster center; A first screening module is used to determine, for each of the multiple data sets, p nearest neighbor cluster centers among cluster centers of multiple vector clusters of the data set according to the query vector, and similarities between the nearest neighbor cluster centers and the query vector, wherein p is an integer greater than 0; A second screening module is used to determine q target cluster centers from all the neighbor cluster centers of the multiple data sets according to the weight of each data set in the multiple data sets and the similarity between the p neighbor cluster centers of each data set and the query vector, wherein q is an integer greater than 0 and less than the number of all the neighbor cluster centers of the multiple data sets; A query module is used to determine k nearest neighbor vectors in the vector clusters to which the centers of the q target clusters belong according to the query vector, wherein k is an integer greater than 0.

10. The vector query device according to claim 9, wherein the first screening module is used for: respectively determining similarities between the query vector and cluster centers of a plurality of vector clusters of the data set; Among the cluster centers of the multiple vector clusters in the data set, p cluster centers having the highest similarity to the query vector are determined as neighbor cluster centers.

11. The vector query device according to claim 9, wherein the second screening module is used for: For each of the multiple data sets, respectively modifying the similarity between the p nearest neighbor cluster centers of the data set and the query vector according to the weight of the data set; Among all the nearest cluster centers of the multiple data sets, q nearest cluster centers having the highest similarity to the query vector are determined as target cluster centers.

12. The vector query device according to claim 9, further comprising a merging module, configured to: For each neighbor cluster center among all neighbor cluster centers of the multiple data sets, determining a primary key cluster corresponding to the vector cluster to which the neighbor cluster center belongs, and determining a cluster center of the primary key cluster; Among all the neighbor cluster centers of the multiple data sets, if the similarity between the cluster centers of the primary key clusters corresponding to the vector clusters to which any two neighbor cluster centers belong is greater than a preset threshold, the vector clusters to which the two neighbor cluster centers belong are merged, and the two neighbor cluster centers are merged into multiple cluster centers.

13. The vector query device according to claim 12, wherein the second screening module is used for: For each neighbor cluster center that has not been merged among all the neighbor cluster centers of the multiple data sets, modifying the similarity between the neighbor cluster center and the query vector according to the weight of the data set to which the neighbor cluster center belongs; For each multi-cluster center obtained by merging all the nearest cluster centers of the multiple data sets, determining the similarity of the multi-cluster centers according to the weights of the data sets to which the two nearest cluster centers corresponding to the multi-cluster center belong and the similarity between the two nearest cluster centers corresponding to the multi-cluster center and the query vector; Among all the unmerged neighbor cluster centers and all the merged multi-cluster centers of the multiple data sets, q neighbor cluster centers and / or multi-cluster centers having the highest similarity to the query vector are determined as target cluster centers.

14. The vector query device according to claim 9, wherein the query module is used for: Determine the similarity between the query vector and each vector in the vector cluster to which the q target cluster centers belong; Among the vector clusters to which the centers of the q target clusters belong, k vectors having the highest similarity to the query vector are determined as neighbor vectors.

15. A computer program product, comprising a computer program / instruction, which, when executed by a processor, implements the steps of the method according to any one of claims 1 to 8.

16. An electronic device, comprising: processor; a memory for storing processor-executable instructions; The processor implements the method according to any one of claims 1 to 8 by running the executable instructions.

17. A computer-readable storage medium having computer instructions stored thereon, which, when executed by a processor, implement the steps of the method according to any one of claims 1 to 8.

Images