Vector retrieval method and device

Abstract

A vector retrieval method and device are used for solving the problems that when an existing index established based on an IVF is applied to a vector retrieval process, the calculation amount is large, and the retrieval speed is low. In the application, the method comprises the following steps: acquiring a vector to be queried; a first index corresponding to the vector to be queried is searched from an index database, the index database comprises a plurality of indexes, each index corresponds to a first cluster, the first cluster comprises a plurality of second clusters, each index records a first cluster center vector, a plurality of second cluster center vectors, feature vectors corresponding to the first clusters are clustered by the first cluster center vectors, and the feature vectors corresponding to the second clusters are clustered by the second cluster center vectors; the feature vector corresponding to each second cluster is clustered by each second cluster center vector; searching a second cluster center vector corresponding to the vector to be queried from the first index; finding a second cluster corresponding to the second cluster center vector according to the second cluster center vector; and searching a target feature vector related to the vector to be queried from a plurality of feature vectors included in the second cluster.

Description

[0001] This application is a divisional application. The application number of the original application is 202210546971.0, and the original application date is May 18, 2022. The entire contents of the original application are incorporated into this application by reference. Technical Field

[0002] The present application relates to the field of computing technology, and in particular to a vector retrieval method and device. Background Art

[0003] Vector retrieval technology is widely used in artificial intelligence (AI) scenarios. Specifically, the data collected from the outside world (such as pictures, videos, audio, text, etc.) can be analyzed and processed by feature extraction to obtain the feature vectors corresponding to each data, and then these feature vectors are stored in the feature library. Furthermore, based on these feature vectors in the feature library, an index of the feature library is created. When a feature vector needs to be retrieved, the search can be performed in the feature library based on the index of the created feature library.

[0004] Vector retrieval based on inverted file system (IVF) is one of the more common vector retrieval technologies currently. Specifically, an index of a feature library (i.e., IVF) can be established based on feature vectors in the feature library, and a feature vector can be retrieved from the feature library based on IVF during vector retrieval.

[0005] However, the IVF-based search process has the problems of large amount of calculation and slow search speed. And as the amount of data continues to increase, the speed of IVF-based search may not meet business needs. Summary of the invention

[0006] The present application provides a vector retrieval method and device, which are used to solve the problems of large amount of calculation and slow retrieval speed when the existing index established based on IVF is applied to the vector retrieval process.

[0007] In a first aspect, the present application provides a vector retrieval method, which can be specifically executed by a computing device or by a processor in a computing device. The method includes: obtaining a vector to be queried; searching for a first index corresponding to the vector to be queried from an index library, wherein the index library includes multiple indexes, each index corresponds to a first cluster, the first cluster includes multiple second clusters, each index records the first cluster center vector, and multiple second cluster center vectors corresponding to multiple second clusters in the first cluster, wherein the feature vector corresponding to the first cluster is obtained by clustering the first cluster center vector, and the feature vector corresponding to each second cluster is obtained by clustering each second cluster center vector; searching for the second cluster center vector corresponding to the vector to be queried from the first index; finding the second cluster corresponding to the second cluster center vector according to the second cluster center vector; and searching for a target feature vector related to the vector to be queried from the multiple feature vectors included in the found second cluster.

[0008] In the above technical solution, the index library includes multiple indexes corresponding to the first clusters, each index records the first cluster center vector and multiple second cluster center vectors. Accordingly, when performing vector retrieval, the computing device can first select the first index corresponding to the vector to be queried from the index library; then select the second cluster center vector corresponding to the vector to be queried from the multiple second cluster center vectors included in the first index; and then select the target feature vector related to the vector to be queried from the second cluster corresponding to the selected second cluster center vector. In this way, the target feature vector is determined through multi-level vector retrieval, which helps to reduce the amount of retrieval calculation and speed up the retrieval.

[0009] In a possible implementation, searching the index library for a first index corresponding to the vector to be queried includes: comparing the vector to be queried with first cluster center vectors of multiple index records in the index library, and the first index is the one whose comparison result satisfies a first preset condition.

[0010] In a possible implementation, the vector to be queried is of M dimensions, M is a positive integer, and the feature vectors in the second cluster are of N dimensions, N is a positive integer and N is less than M; searching for a target feature vector related to the vector to be queried from the multiple feature vectors included in the second cluster includes: reducing the dimension of the vector to be queried from M dimensions to N dimensions, that is, obtaining an N-dimensional query vector after dimension reduction; determining the similarity between the vector to be queried (that is, the N-dimensional query vector) and the multiple feature vectors in the second cluster (that is, the N-dimensional feature vector); and selecting the target feature vector from the multiple feature vectors according to the similarity.

[0011] In the above technical solution, the computing device may first perform dimensionality reduction processing on the M-dimensional feature vector to obtain an N-dimensional feature vector, and then store the N-dimensional feature vector after the dimensionality reduction processing, which helps to reduce the storage space occupied by the feature vector. Further, during the retrieval process, the computing device may first perform dimensionality reduction processing on the M-dimensional query vector to obtain an N-dimensional query vector, and then select a target feature vector from multiple N-dimensional feature vectors based on the similarity between the N-dimensional query vector and multiple N-dimensional feature vectors, which helps to reduce the amount of calculation during the retrieval process and further improve the retrieval speed.

[0012] In a possible implementation, X first cluster center vectors are determined, and the feature vectors in the feature library are clustered into X first clusters with each of the X first cluster center vectors as the center; for each of the X first clusters, Y second cluster center vectors are selected, and the feature vectors in the first cluster are clustered into Y second clusters with the Y second cluster center vectors as the center; for each first cluster, an index is generated, and the index is used to record the first cluster center vector and the second cluster center vector corresponding to the first cluster center vector. The number Y of second clusters included in each first cluster is the same or different. X and Y are both positive integers.

[0013] In the above technical solution, the index of the first cluster is determined by performing multi-level clustering on the feature vectors in the feature library. The index of the first cluster can be applied to the vector retrieval process, which helps to reduce the amount of retrieval calculation and speed up the retrieval.

[0014] In a possible implementation, the X first cluster center vectors are determined, specifically, by clustering a preset proportion of feature vectors (which may be referred to as training samples) in the feature library to obtain X first clusters, and determining the first cluster center vector corresponding to each first cluster according to the feature vectors included in the first cluster. Furthermore, with each of the X first cluster center vectors as the center, the remaining feature vectors in the feature library are clustered into the X first clusters.

[0015] In a possible implementation, the Y second cluster center vectors are determined by clustering a preset proportion of feature vectors in the first cluster to obtain Y second clusters, and determining the second cluster center vector corresponding to each second cluster according to the feature vectors included in each second cluster. Furthermore, the remaining feature vectors in the first cluster are clustered into Y second clusters with each second cluster center vector in the Y second cluster center vectors as the center.

[0016] In one possible implementation, after clustering the feature vectors in the first cluster into Y second clusters with Y second cluster center vectors as the center, the method also includes: selecting a third cluster from the Y second clusters, the number of feature vectors in the third cluster being less than a threshold; determining a second cluster center vector that is closest to the second cluster center vector of the third cluster; and merging the feature vector of the third cluster into the second cluster corresponding to the determined second cluster center vector.

[0017] In the above technical solution, after clustering multiple feature vectors in the first cluster to obtain multiple second clusters in the first cluster, the multiple second clusters can also be dynamically adjusted, that is, the second cluster containing a smaller number of feature vectors can be merged into the closest second cluster containing a larger number of feature vectors, so as to avoid querying (or recalling) the second cluster with a smaller number of features when querying the second cluster corresponding to the center vector of the second cluster, which helps to improve the accuracy of the determined second cluster.

[0018] In a possible implementation, after clustering the feature vectors in the feature library into X first clusters with each of the X first cluster center vectors as the center, the method further includes: executing on each feature vector in the feature library: selecting a redundant cluster corresponding to the feature vector from the X first clusters, wherein the similarity between the feature vector and the cluster center vector of the redundant cluster corresponding to the feature vector meets a preset redundancy condition; and selecting a redundant feature vector to be redundantly stored from a plurality of feature vectors according to the similarity between each feature vector in the feature library and the cluster center vector of the redundant cluster corresponding to the respective feature vector, so that the redundant cluster corresponding to the redundant feature vector includes the redundant feature vector.

[0019] In the above technical solution, the redundant feature vector is redundantly stored, so that the redundant feature vector can be obtained (or recalled) through multiple second clusters, which helps to improve the probability of obtaining (or recalling) the redundant feature vector.

[0020] In a possible implementation, the feature vector in the second cluster is an M-dimensional feature vector, where M is a positive integer; the method further includes: performing dimensionality reduction processing on each M-dimensional feature vector in the second cluster to obtain an N-dimensional feature vector corresponding to the M-dimensional feature vector, where N is a positive integer and less than M; and storing the N-dimensional feature vector corresponding to each M-dimensional feature vector in the second cluster.

[0021] In the above technical solution, the M-dimensional feature vector is first subjected to dimensionality reduction processing to obtain the N-dimensional feature vector corresponding to the M-dimensional feature vector, thereby helping to reduce the storage space occupied by the feature vector storage. And in the process of vector retrieval, it helps to reduce the amount of calculation of similarity calculation. In addition, clustering the feature vectors in the feature library first, and then performing dimensionality reduction processing on the M-dimensional feature vector, helps to ensure the accuracy of clustering, and the number of feature vectors in the second cluster after clustering is small, avoiding the problem of reduced retrieval accuracy due to vector dimensionality reduction.

[0022] In a second aspect, an embodiment of the present application provides a vector retrieval device, which has the function of implementing the method in the above-mentioned first aspect or any possible implementation manner of the first aspect. The device can be a computing device or a processor included in the computing device.

[0023] The functions of the above-mentioned vector search device can be implemented by hardware, or by hardware executing corresponding software. The hardware or software includes one or more modules or units or means corresponding to the above-mentioned functions.

[0024] In a possible implementation, the structure of the device includes a processing module and a transceiver module, wherein the processing module is configured to support the device to execute the method in the first aspect or any one of the implementations of the first aspect. The transceiver module is used to support communication between the device and other devices, for example, it can receive data from an acquisition device. The vector retrieval device may also include a storage module, which is coupled to the processing module and stores program instructions and data necessary for the device. As an example, the processing module may be a processor, the transceiver module may be a transceiver, and the storage module may be a memory. The memory may be integrated with the processor or may be set separately from the processor.

[0025] In another possible implementation, the structure of the device includes a processor and may also include a memory. The processor is coupled to the memory and may be used to execute computer program instructions stored in the memory so that the device performs the method in the first aspect or any possible implementation of the first aspect. Optionally, the device further includes a communication interface, and the processor is coupled to the communication interface. When the device is a computing device, the communication interface may be a transceiver or an input / output interface.

[0026] In a third aspect, an embodiment of the present application provides a chip system, comprising: a processor, the processor being coupled to a memory, the memory being used to store programs or instructions, and when the programs or instructions are executed by the processor, the chip system implements the method in the above-mentioned first aspect or any possible implementation method of the first aspect.

[0027] Optionally, the chip system also includes an interface circuit, which is used to interact with code instructions to the processor.

[0028] Optionally, there may be one or more processors in the chip system, and the processor may be implemented by hardware or software. When implemented by hardware, the processor may be a logic circuit, an integrated circuit, etc. When implemented by software, the processor may be a general-purpose processor implemented by reading software code stored in a memory.

[0029] Optionally, the memory in the chip system may be one or more. The memory may be integrated with the processor or may be separately provided with the processor. Exemplarily, the memory may be a non-transient processor, such as a read-only memory ROM, which may be integrated with the processor on the same chip or may be provided on different chips.

[0030] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program or instructions stored thereon. When the computer program or instructions are executed, the computer executes the method in the above-mentioned first aspect or any possible implementation of the first aspect.

[0031] In a fifth aspect, an embodiment of the present application provides a computer program product. When a computer reads and executes the computer program product, the computer executes the method in the above-mentioned first aspect or any possible implementation manner of the first aspect.

[0032] The technical effects that can be achieved in any of the second to fifth aspects mentioned above can refer to the description of the beneficial effects in the first aspect mentioned above, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 A schematic diagram of a system architecture provided for this application;

[0034] Figure 2 A schematic diagram of the structure of a computing device provided for this application;

[0035] Figure 3 A schematic diagram of the structure of a processor provided in this application;

[0036] Figure 4 A schematic diagram of a process flow of a vector retrieval technology provided in this application;

[0037] Figure 5 A flowchart of a method for establishing an index provided in this application;

[0038] Figure 6 A flowchart of a redundant storage method provided in this application;

[0039] Figure 7 A flowchart of a feature dimension reduction storage method provided in this application;

[0040] Figure 8 A schematic diagram of reducing the dimension of a 256-dimensional feature vector to obtain a 64-dimensional feature vector provided in this application;

[0041] Fig. 9 A schematic diagram of a flow chart of a vector retrieval method provided in this application;

[0042] Fig.10 A schematic diagram of a flow chart of another vector retrieval method provided in this application. DETAILED DESCRIPTION

[0043] In order to better explain the present application, the technologies or terms involved in the present application are explained as follows.

[0044] 1. Vector retrieval technology: In a given vector data set, vectors similar to the query vector are retrieved according to a certain metric.

[0045] 2. K-means clustering algorithm is an iterative clustering analysis algorithm. Specifically, given the number of categories k, the entire data set is clustered. The objective function is the minimum sum of the distances from all samples to the class center. The objective function is iteratively calculated and optimized to obtain k class centers and the category to which each sample belongs.

[0046] 3. K-nearest neighbor classification algorithm (KNN) is that a sample is most similar to K samples in the data set. If most of these K samples belong to a certain category, then the sample also belongs to this category.

[0047] 4. Feature vector dimensionality reduction. Since high-dimensional feature vectors bring about high computational costs and require a large sample size in feature training, which makes fitting difficult, feature vector dimensionality reduction can be used to improve the speed and load of feature training. For example, compared to representing each age in one dimension, the age can be represented as 0-10, 10-20..., >100 segments, that is, the age is mapped into 11 dimensions.

[0048] 5. FP32, FP16, INT8

[0049] In computer language, FP32 means single-precision floating point number, and correspondingly, FP16 means half-precision floating point number, where the memory access consumption of FP16 is only 1 / 2 of the memory access consumption of FP32.

[0050] INT8 is a fixed-point computing method that represents integer operations and is generally quantized from floating-point operations. In binary, a "0" or "1" is one bit, and INT8 means using 8 bits to represent a number. Therefore, although INT8 has lower precision than FP16, it has a smaller data volume, lower energy consumption, and a relatively faster computing speed.

[0051] 6. Inverted file system (IVF), for multiple feature vectors, all data are divided into nlist clusters (or buckets / groups / families) by clustering. For the search of a feature vector, nlist searches can be performed first, that is, the input feature vector (or query vector) is compared with the cluster center vectors of nlist clusters (also feature vectors), and the nprobe clusters with the highest similarity to the input feature vector are found, and then the similarity (or distance, similarity, etc.) comparison is performed for all feature vectors in these nprobe clusters.

[0052] 7. Affine transformation: In geometry, a space vector undergoes a linear transformation followed by a translation to transform it into another vector space.

[0053] 8. Inner product (IP): also known as scalar product or dot product. It is a vector operation, but its result is a numerical value, not a vector.

[0054] Figure 1 A possible system architecture diagram is provided, which includes a collection device 10 and a computing device 20. The collection device 10 may be one or more, the computing device 20 may be one or more, and the one or more collection devices 10 and the one or more computing devices 20 may be connected via a network.

[0055] The acquisition device 10 can be used to collect data and send the collected data to the computing device 20 through the network. The acquisition device 10 can be a camera, a mobile phone, a computer, etc., and the data collected by the acquisition device 10 can be pictures, videos, audio, text, etc. Exemplarily, in a video surveillance scenario, the acquisition device 10 can be a camera, and the data collected by the camera can be pictures and / or videos taken by the camera.

[0056] The computing device 20 can receive and store the data sent by the acquisition device 10. In addition, the computing device 20 can also perform feature extraction on the data to obtain a feature vector corresponding to the data, generate an index based on the feature vector corresponding to the data, and perform a data retrieval process based on the generated index to retrieve the corresponding data.

[0057] Further, such as Figure 2 2 is a schematic diagram of a possible structure of a computing device 20 , which includes a processor 201 , a memory 202 , and a communication interface 203 . Any two of the processor 201 , the memory 202 , and the communication interface 203 may be connected via a bus 204 .

[0058] The processor 201 may be a central processing unit (CPU), which may be used to execute software programs in the memory 202 to implement one or more functions, such as extracting features from data. In addition to the CPU, the processor 201 may also be an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), a system on chip (SoC) or a complex programmable logic device (CPLD), a graphics processing unit (GPU), a neural-network processing unit (NPU), etc.

[0059] It should be noted that, in actual applications, there may be multiple processors 201, and the multiple processors 201 may include multiple processors of the same type, or may include multiple processors of different types. For example, multiple processors 201 are multiple CPUs. For another example, the multiple processors 201 include one or more CPUs and one or more GPUs. For another example, the multiple processors 201 include one or more CPUs and one or more NPUs. Alternatively, the multiple processors 201 include one or more CPUs, one or more GPUs, and one or more NPUs, etc. Among them, the processor 201 (such as a CPU, an NPU, etc.) may include one core, or may include multiple cores.

[0060] The memory 202 refers to a device for storing data, which can be a memory or a hard disk.

[0061] Memory refers to an internal memory that directly exchanges data with the processor 201. It can read and write data at any time and at a very fast speed. It serves as a temporary data storage for the operating system or other running programs running on the processor 201. Memory includes volatile memory (volatile memory), for example, random access memory (RAM), dynamic random access memory (DRAM), etc., and may also include non-volatile memory (non-volatile memory), such as storage class memory (SCM), etc., or a combination of volatile memory and non-volatile memory. In practical applications, multiple memories can be configured in the computing device 20, and optionally, the multiple memories can be of different types. This embodiment does not limit the number and type of memory. In addition, the memory can be configured to have a power-saving function. The power-saving function means that when the system loses power and then powers on again, the data stored in the memory will not be lost. Memory with a power-saving function is called a non-volatile memory.

[0062] The hard disk is used to provide storage resources, such as for storing data such as pictures, videos, audio, text, etc. collected by the acquisition device 10. The hard disk includes, but is not limited to: non-volatile memory (non-volatile memory), such as read-only memory (ROM), hard disk drive (HDD) or solid state drive (SSD), etc. The difference from the memory is that the hard disk has a slower read and write speed and is usually used to store data persistently. In one embodiment, the data, program instructions, etc. in the hard disk need to be loaded into the memory first, and then the processor obtains these data and / or program instructions from the memory.

[0063] The communication interface 203 is used for communicating with other devices, such as for the computing device 20 to communicate with the acquisition device 10 .

[0064] In practical applications, the computing device 20 may include two processors 201, which may be a CPU and an NPU. The CPU may include 6 CPU cores, and the NPU may include 2 NPU cores, which may also be called AI cores. The computing power of the NPU is higher than that of the CPU. The CPU can be used to perform similarity sorting in the data retrieval process, and the NPU can be used to perform similarity calculation in the data retrieval process. For details, see Figure 3 The structure of a processor 201 in a computing device 20 is shown.

[0065] This application provides a flow chart of applying a vector retrieval technology to an AI scenario (specifically, a monitoring scenario). The flow chart can be found in Figure 4 Specifically, the process can be specifically Figures 1 to 3 The computing device 20 shown is executed, and the process can be divided into the following three stages:

[0066] 1. Feature extraction stage

[0067] The computing device 20 can obtain the acquisition device 10 ( Figure 4 The computing device 20 takes multiple pictures taken by multiple cameras (in the example), and inputs each picture into a preset convolutional neural network (CNN) model, so that the CNN model outputs a feature vector (or picture feature) corresponding to each picture. Subsequently, the computing device 20 stores the feature vector corresponding to each picture in a feature library, which can be specifically located in the memory 202.

[0068] 2. Index Generation Phase

[0069] The computing device 20 clusters (also referred to as aggregates) the feature vectors in the feature library using a k-means clustering algorithm to obtain nlist clusters, each of which corresponds to its own cluster center vector (also referred to as a cluster center).

[0070] In a specific implementation, the computing device 20 may first randomly select a preset proportion (e.g., about 10%) of feature vectors from the feature library as training samples, and cluster the training samples according to the k-means clustering algorithm to obtain nlist clusters. The computing device 20 then divides the feature vectors in the feature library other than the training samples into clusters corresponding to the closest cluster center vectors according to the cluster center vectors of the nlist clusters.

[0071] As described above, the computing device 20 can obtain which cluster each feature vector in the feature library belongs to, and generate the cluster index according to the cluster identifier, cluster center vector and identifier of the feature vector contained in the cluster. Further, the nlist clusters corresponding to the feature library correspond to nlist indexes respectively, and the computing device 20 can form the nlist indexes into the index of the feature library. The index of the feature library can be called the feature library index, or the index library.

[0072] 3. Vector Retrieval Stage

[0073] The computing device 20 selects a target feature vector from all feature vectors in the feature library according to the query vector and the index library. Specifically, the vector retrieval process can be divided into two parts: determining a cluster and determining a feature vector.

[0074] During the cluster determination process:

[0075] The computing device 20 compares the query vector with the cluster center vectors of nlist clusters respectively, and selects nprobe clusters from the nlist clusters according to the comparison results. For example, the computing device 20 determines the similarity between the query vector and the cluster center vectors of nlist clusters respectively, and obtains nlist similarities. The computing device 20 selects the first nprobe similarities in the nlist similarities sorted from high to low, and uses the clusters corresponding to the first nprobe similarities as nprobe clusters.

[0076] In the process of determining the eigenvectors:

[0077] The computing device 20 compares the query vector with all feature vectors in the nprobe clusters, and selects K nearest feature vectors from all feature vectors in the nprobe clusters according to the comparison result.

[0078] Each of the nprobe clusters may include multiple feature vectors, for example, the nprobe clusters include a total of mprobe feature vectors. The computing device 20 determines the similarity between the query vector and all feature vectors in the nprobe clusters, obtains mprobe similarities, and then selects the first K similarities in the mprobe similarities sorted from high to low, and then uses the feature vectors corresponding to the first K similarities as the K nearest neighbor feature vectors (i.e., KNN results), where K is a positive integer. The KNN result is returned as a search result.

[0079] Here, the vector to be queried (also referred to as the query vector) can be a feature vector input by the user, or any feature vector in the feature library. In the latter case, it can be understood that the computing device 20 can traverse each feature vector in the feature library and determine the K nearest feature vectors of each feature vector. In this way, the computing device 20 can obtain the KNN results of all feature vectors in the feature library.

[0080] It can be understood that, assuming that the number of feature vectors in each cluster is the same, the total amount of calculation by the computing device 20 in the vector retrieval stage can be expressed as Wherein, N is the total number of feature vectors in the feature library. Furthermore, as the total number of feature vectors in the feature library increases, the amount of calculation required by the computing device 20 in the vector retrieval phase will gradually increase, and the query speed will become slower and slower.

[0081] To this end, the present application provides a method for establishing an index, which is used to solve the problems of large amount of calculation and slow retrieval speed when the existing index established based on space partitioning (such as IVF) is applied to the vector retrieval stage.

[0082] In the indexing method provided in the present application, the computing device 20 can perform multi-level clustering on multiple feature vectors in the feature library, and each level of clustering can achieve clustering of multiple feature vectors in the current cluster into multiple next-level clusters.

[0083] For example, the computing device 20 performs first-level clustering on multiple feature vectors in the feature library to obtain multiple first-level clusters, each of which may include multiple feature vectors. Further, the computing device 20 may perform second-level clustering on multiple feature vectors in a first-level cluster (i.e., the current cluster) to obtain multiple second-level clusters.

[0084] For the convenience of description, the current cluster may be referred to as the first cluster, and the next-level cluster corresponding to the first cluster may be referred to as the second cluster. That is, the first cluster may correspond to multiple second clusters.

[0085] Reference Figure 5 A flowchart illustrating an exemplary method of indexing is shown below:

[0086] In step 501 , the computing device 20 determines X first clusters.

[0087] Specifically, the computing device 20 may determine X first clusters according to the first clustering algorithm and the plurality of feature vectors in the feature library, where X is a positive integer. Each of the X first clusters includes a plurality of feature vectors. The first clustering algorithm may be, for example, a k-means clustering algorithm, a fuzzy c-means algorithm (FCMA), or the like.

[0088] Exemplarily, the computing device 20 performs first-level clustering on multiple feature vectors in the feature library to obtain X' first-level clusters, where X' is a positive integer. Among the X' first-level clusters, there may be a part of the first-level clusters with a small number of feature vectors, and the computing device 20 does not need to perform the next-level clustering on this part of the first-level clusters; while for another part of the first-level clusters, since the number of feature vectors is large, the computing device 20 needs to perform the next-level clustering on this other part of the first-level clusters. Of course, the present application does not exclude the need for the computing device 20 to perform the next-level clustering on all of the above-mentioned first-level clusters. That is, the X first clusters may be all or part of the X' first-level clusters obtained after the first-level clustering.

[0089] As another example, a second-level clustering is performed on multiple feature vectors in the first-level clusters to obtain X' second-level clusters, wherein the X first clusters may be all or part of the X' second-level clusters. Of course, the X first clusters may also be part or all of the X' clusters obtained by another level of clustering, which will not be listed in this application.

[0090] The computing device 20 determines X' first clusters according to the first clustering algorithm and the multiple feature vectors in the feature library in the following two implementations:

[0091] In a first implementation method, the computing device 20 clusters multiple feature vectors in the feature library according to a first clustering algorithm to directly obtain X' first clusters. The computing device 20 then determines the cluster center vector (which may be referred to as the first cluster center vector) of each first cluster according to the feature vector in each of the X' first clusters.

[0092] In the second implementation method, the computing device 20 randomly selects (or samples) a preset proportion (e.g., about 10%) of feature vectors from the feature library as the first training sample, and clusters to obtain X' first clusters according to the first clustering algorithm and the first training sample. The computing device 20 then determines the first cluster center vector of each first cluster according to the feature vector in each of the X' first clusters. The computing device 20 clusters the other feature vectors in the feature library except the first training sample into the X first clusters, taking each of the X' first cluster center vectors as the center. In this way, the amount of calculation required by the computing device 20 to determine the first cluster center vector can be reduced, and the first cluster center vector can be determined more quickly.

[0093] It is understandable that when determining the X first clusters, the computing device 20 may refer to the above implementation mode 1 and implementation mode 2, and may replace "X' first clusters" with "X first clusters", which will not be described in detail.

[0094] In step 502 , the computing device 20 performs, for each of the X first clusters: determining a plurality of second clusters in the first cluster according to a plurality of feature vectors in the first cluster.

[0095] Take any one of the X first clusters as an example: the computing device 20 clusters multiple feature vectors in the first cluster according to the second clustering algorithm to obtain Y second clusters corresponding to the first cluster, where Y is a positive integer. The second clustering algorithm is, for example, a k-means clustering algorithm, a fuzzy c-means clustering algorithm, etc., and the first clustering algorithm is the same as or different from the second clustering algorithm. The number Y of second clusters obtained by dividing each first cluster can be the same or different, for example, among the X first clusters, one first cluster is divided into 5 second clusters, and another first cluster is divided into 10 second clusters.

[0096] The computing device 20 determines the implementation of the Y first clusters according to the second clustering algorithm and the multiple feature vectors in the first clusters, which may include the following implementation 1 and implementation 2:

[0097] Implementation method 1: The computing device 20 clusters multiple feature vectors in the first cluster according to the second clustering algorithm to directly obtain Y second clusters. The computing device 20 then determines the cluster center vector (which may be called the second cluster center vector) of each second cluster based on the feature vector in each of the Y second clusters.

[0098] Implementation method 2, the computing device 20 randomly selects a preset proportion (for example, about 10%) of feature vectors from the first cluster as the second training sample, and clusters to obtain Y second clusters according to the second clustering algorithm and the second training sample. The computing device 20 then determines the second cluster center vector corresponding to each second cluster according to the feature vector in each of the Y second clusters. The computing device 20 clusters the other feature vectors in the first cluster except the second training sample into the Y second clusters with each of the Y second cluster center vectors as the center. In this way, the amount of calculation for the computing device 20 to determine the second cluster center vector can be reduced, and the second cluster center vector can be determined more quickly.

[0099] Further, the computing device 20 may first determine the number Y of second clusters corresponding to the first cluster, and then cluster the plurality of feature vectors in the first cluster to obtain Y second clusters according to the number Y of second clusters and the second clustering algorithm.

[0100] When the computing device 20 determines the number Y of second clusters corresponding to the first cluster, it can be determined specifically according to the number of feature vectors in the first cluster. It can be considered that the more the number of feature vectors in the first cluster, the larger the number Y of second clusters corresponding to the first cluster. Alternatively, the computing device 20 can also determine according to the number of feature vectors in the first cluster and hardware conditions, wherein the hardware conditions include the computing power of the processor 201 in the computing device 20 that runs the algorithm, and the computing power of the processor 201 is, for example, the size and number of matrix operation units included in the processor 201.

[0101] For example, the processor 201 is an NPU, and the size of the matrix operation unit in the NPU is 16×16, that is, the matrix operation unit can calculate a 16×16 matrix at a time. The computing device 20 can determine that the number of eigenvectors included in the second cluster is a multiple of 16. For example, the computing device 20 determines that the number of eigenvectors included in the second cluster is 64. Further, for example, if the number of eigenvectors in the current first cluster is 640, the computing device 20 can assume that the 640 eigenvectors are evenly divided into each second cluster to obtain the number of second clusters Y, that is, Y is equal to 10.

[0102] As described above, determining the number of second clusters in the first cluster according to the number of feature vectors in the first cluster, or according to the number of feature vectors in the first cluster and hardware conditions, helps to improve the flexibility of clustering.

[0103] In addition, after the computing device 20 clusters the plurality of feature vectors in the first cluster to obtain Y second clusters according to the number Y of second clusters and the second clustering algorithm, the Y second clusters may be further dynamically adjusted.

[0104] Exemplarily, a threshold value (denoted as the quantity threshold) of the number of features in the second cluster is pre-set in the computing device 20. The computing device 20 selects one or more third clusters from Y second clusters according to the quantity threshold, wherein the number of feature vectors in the third cluster is less than the quantity threshold. For a third cluster, the computing device 20 may calculate the similarity between the cluster center vector of the third cluster and the cluster center vectors of other second clusters except the third cluster, and then select the second cluster with the highest similarity. The computing device 20 merges the feature vectors in the third cluster into the selected second cluster to form a new second cluster. In this way, the computing device 20 determines Y' second clusters, where Y' is a positive integer, and Y' is less than Y.

[0105] For example, when the computing device 20 clusters the first cluster (referred to as cluster 1), it determines that the value of the number Y of second clusters corresponding to cluster 1 is 5, and clusters multiple feature vectors in cluster 1 to obtain 5 second clusters, which can be respectively recorded as cluster 1-1 to cluster 1-5, wherein cluster 1-1 to cluster 1-5 include 300 feature vectors, 200 feature vectors, 200 feature vectors, 10 feature vectors, and 20 feature vectors, respectively. Assuming that the value of the quantity threshold is 150, clusters 1-4 and 1-5 are both the third clusters. Furthermore, cluster 1-4 is closest to cluster 1-1, and the computing device 20 can merge the 10 feature vectors in cluster 1-4 into cluster 1-1; cluster 1-5 is closest to cluster 1-2, and the computing device 20 can merge the 20 feature vectors in cluster 1-5 into cluster 1-2. That is, the value of Y' is 3, and the computing device 20 divides the first cluster into three second clusters, namely cluster 1-1 to cluster 1-3, wherein cluster 1-1 includes 310 feature vectors, cluster 1-2 includes 220 feature vectors, and cluster 1-3 includes 200 feature vectors.

[0106] In the above technical solution, after clustering multiple feature vectors in the first cluster to obtain multiple second clusters in the first cluster, the multiple second clusters can also be dynamically adjusted, that is, the second cluster containing a smaller number of feature vectors can be merged into the closest second cluster containing a larger number of feature vectors, which helps to improve the query accuracy.

[0107] It should be added that, in the present application, the computing device 20 can also dynamically adjust the clusters after other levels of clustering, such as dynamically adjusting the first-level clusters obtained after the first-level clustering, etc., which will not be elaborated herein.

[0108] In this way, the computing device 20 can complete the clustering of each of the X first clusters to obtain Y second clusters corresponding to each of the X first clusters. For example, the computing device 20 determines that the X first clusters are represented as cluster 1 to cluster X, wherein the three second clusters included in cluster 1 are represented as cluster 1-1 to cluster 1-3 respectively; the two second clusters included in cluster 2 are represented as cluster 2-1 and cluster 2-2 respectively; the four second clusters included in cluster 3 are represented as cluster 3-1 to cluster 3-4 respectively, and so on.

[0109] Step 503: The computing device 20 establishes an index library according to the X first clusters and a plurality of second clusters in each of the X first clusters.

[0110] It is pre-defined that the second cluster can be considered to be the cluster corresponding to the last level of clustering in the multi-level clustering. For example, if the computing device 20 performs three-level clustering on the feature vectors in the feature library, the second cluster is the cluster corresponding to the third level of clustering in the three-level clustering, and the first cluster is the cluster corresponding to the second level of clustering in the three-level clustering; for another example, if the computing device 20 performs two-level clustering on the feature vectors in the feature library, the second cluster is the cluster corresponding to the second level of clustering in the two-level clustering, and the first cluster is the cluster corresponding to the first level of clustering in the two-level clustering. Among them, the number of levels of clustering of the feature vectors in the feature library by the computing device 20 can be greater than or equal to 2, and the specific number of levels can be determined according to the total number of feature vectors in the feature library and / or hardware conditions.

[0111] After determining multiple second clusters of each of the X first clusters, the computing device 20 may generate an index library according to the cluster identifiers and cluster center vectors (i.e., first cluster center vectors) of the X first clusters, the cluster identifiers and cluster center vectors (i.e., second cluster center vectors) of the multiple second clusters in each first cluster, and the feature vectors in each second cluster.

[0112] The following is an explanation of the level of clustering of feature vectors in the feature library by the computing device 20 in two cases:

[0113] Case 1: The computing device 20 performs two-level clustering on the feature vectors in the feature library.

[0114] It can be understood that the first cluster here is the first-level cluster, and the second cluster is the second-level cluster.

[0115] The computing device 20 may first perform, for each of the X first-level clusters, the following steps: determine the index of the first-level cluster according to the cluster identifier and cluster center vector of the first-level cluster, the cluster identifiers and cluster center vectors of multiple second-level clusters in the first-level cluster, and the identifier of the feature vector in each second-level cluster, thereby obtaining the indexes of the X first-level clusters. The computing device 20 then forms an index library with the indexes of the X first-level clusters.

[0116] Take the example that the computing device 20 determines clusters 1 to X (i.e., X first-level clusters) as follows:

[0117] The index of cluster 1 generated by the computing device 20 can be seen in Table 1. The cluster identifier of cluster 1 is represented as "cluster 1", the cluster center vector of cluster 1 is represented as "cluster center vector 1", cluster 1 includes clusters 1-1 to cluster 1-3, the cluster identifier of cluster 1-1 is represented as "cluster 1-1", the cluster center vector of cluster 1-1 is represented as "cluster center vector 1-1", and cluster 1-1 includes 100 feature vectors, represented as "feature vector 1-100", etc.

[0118] Table 1

[0119]

[0120] The computing device 20 may also generate indexes of other first clusters (ie, cluster 2 to cluster X), and form an index library with the indexes of cluster 1 to cluster X, as shown in Table 2 for details.

[0121] Table 2

[0122]

[0123] Case 2: The computing device 20 performs multi-level (greater than two levels) clustering on the feature vectors in the feature library.

[0124] It can be understood that the first cluster is obtained by clustering the feature vectors in a certain cluster by the computing device 20 according to the first clustering algorithm. For example, when the computing device 20 performs three-level clustering on the feature vectors in the feature library, the computing device 20 first performs first-level clustering on the feature vectors in the feature library to obtain multiple first-level clusters; clusters one of the first-level clusters to obtain multiple second-level clusters (i.e., X first clusters); and clusters one of the second-level clusters (i.e., the first cluster) to obtain multiple third-level clusters (i.e., Y second clusters).

[0125] In this case, the computing device 20 may first perform, for each first-level cluster, the following steps: determine the index of the first-level cluster based on the cluster identifier and cluster center vector of the first-level cluster, the cluster identifiers and cluster center vectors (i.e., first cluster center vectors) of multiple second-level clusters in the first-level cluster, the cluster identifiers and cluster center vectors (i.e., second cluster center vectors) of multiple third-level clusters in each second-level cluster, and the identifier of the feature vector in each third-level cluster. Then, determine the index library based on the indexes of the multiple first-level clusters. The specific method is similar to that of case 1.

[0126] Still taking the three first clusters determined by the above-mentioned computing device 20 as clusters 1 to 3 as an example, wherein clusters 1 to 3 are three second-level clusters, and the upper-level cluster of clusters 1 to 3 is the first-level cluster, for example, the first-level cluster is cluster 0, and the index library generated by the computing device 20 is shown in Table 3. Table 3 adds one level of index compared to Table 2.

[0127] Table 3

[0128]

[0129] Furthermore, when the computing device 20 performs four-level, five-level or more-level clustering on the feature vectors in the feature library, the index library is generated in a manner similar to that in the first case and will not be described in detail.

[0130] It should be added that, in the above step 501, in addition to determining X first clusters, the computing device 20 may also determine the first clusters that do not need to be aggregated at the next level. The computing device 20 may also generate indexes of these first clusters based on the cluster center vectors of these first clusters and the feature vectors in the first clusters, and then generate an index library.

[0131] It should also be added that, in any level of the above multi-level clustering, the similarity between most feature vectors and the cluster center vector of the cluster to which they belong is significantly higher than the similarity between the feature vector and the cluster center vectors of other clusters. This shows that the cluster to which the feature vector belongs can express the feature vector more accurately.

[0132] However, for some feature vectors, the similarity between the cluster center vector of the cluster to which they belong and the cluster center vector of other clusters is not much different. In this case, it is necessary to bind the feature vector to multiple clusters to increase the probability of the feature vector being selected (or called recalled) during the vector retrieval process, which helps to improve the accuracy of the vector retrieval process. For the convenience of description, the feature vector is called a redundant feature vector, which means that the feature vector needs to be stored redundantly.

[0133] The computing device 20 may select redundant feature vectors from all feature vectors included in the feature library, and then determine redundant clusters corresponding to the redundant feature vectors. This process may be referred to as a redundant storage process.

[0134] Furthermore, the computing device 20 may perform the redundant storage process after performing any level of clustering in the multi-level clustering. As follows, the redundant storage process is performed by the computing device 20 after performing the clustering corresponding to the first cluster.

[0135] like Figure 6 An implementation method of a redundant storage process is provided as an example in this application.

[0136] In step 601 , the computing device 20 may execute, for each feature vector in the feature library, the following steps: selecting a redundant cluster corresponding to the feature vector from among the X first clusters.

[0137] The similarity between the feature vector and the cluster center vector of the redundant cluster corresponding to the feature vector meets the preset redundancy condition.

[0138] In a specific implementation, after determining the X first clusters, the computing device 20 may perform, for each feature vector, the following steps: determining the similarity between the feature vector and the cluster center vectors of the other (X-1) first clusters except the cluster to which it belongs, thereby obtaining the (X-1) similarity. Based on the (X-1) similarity, the computing device 20 selects a redundant cluster corresponding to the feature vector from the other (X-1) first clusters except the cluster to which it belongs.

[0139] The following two examples are provided based on the specific manifestations of the preset redundancy conditions.

[0140] Example 1, the preset redundancy condition is specifically that the ratio of the similarity between the feature vector and the cluster center vector of the redundant cluster to which it corresponds to and the similarity between the feature vector and the cluster center vector of the cluster to which it belongs (recorded as the baseline similarity) is greater than the first redundancy threshold. It can be understood that the larger the similarity ratio is, the greater the similarity between the feature vector and the cluster center vector of the redundant cluster to which it corresponds, that is, the more the feature vector should be redundantly stored in its corresponding redundant cluster.

[0141] Accordingly, the computing device 20 may determine the ratios between the (X-1) similarities and the reference similarities to obtain (X-1) ratios. The computing device 20 then selects ratios greater than the first redundancy threshold from the (X-1) ratios, and determines the first clusters corresponding to the selected ratios as the redundant clusters corresponding to the feature vector.

[0142] Example 2, the preset redundancy condition is specifically the ratio of the similarity between the feature vector and the cluster center vector of the corresponding redundant cluster to the similarity between the feature vector and the cluster center vector of the cluster to which it belongs, which is the first h1 in the order of the ratio from high to low. Among them, h1 can also be considered as a preset threshold, h1 is a positive integer, and the explanation of the ratio can be referred to in Example 1.

[0143] Accordingly, the computing device 20 may determine the ratios between the (X-1) similarities and the reference similarities to obtain (X-1) ratios. The computing device 20 then selects the first h1 ratios in descending order from the (X-1) ratios, and determines the h1 first clusters corresponding to the h1 ratios as the redundant clusters corresponding to the feature vector.

[0144] In addition, the preset redundancy condition may specifically be that the similarity between the feature vector and the cluster center vector of the redundant cluster corresponding to the feature vector is greater than the second redundancy threshold, so that the computing device 20 determines the h1 redundant clusters corresponding to the feature vector according to the second redundancy threshold. Alternatively, the preset redundancy condition may specifically be that the similarity between the feature vector and the cluster center vector of the redundant cluster corresponding to the feature vector is the first h1 in the order of similarity from high to low, and the computing device 20 determines the h1 redundant clusters corresponding to the feature vector according to the order of (X-1) similarities from high to low. For specific implementation, please refer to the above-mentioned Examples 1 and 2.

[0145] In this way, the computing device 20 can determine the redundant cluster corresponding to each feature vector in the feature library, and the similarity between each feature vector and the cluster center vector of the corresponding redundant cluster.

[0146] Continuing to give examples in conjunction with Table 2 and Example 2, for example, h1 is equal to 2, and the computing device 20 determines that the redundant clusters corresponding to feature vector 1 are cluster 2 and cluster 3, and the redundant clusters corresponding to feature vector 2 are cluster 3 and cluster 4, etc. Further, assuming that there are a total of 20,000 feature vectors in the feature library, the computing device 20 can determine the two redundant clusters corresponding to the 20,000 feature vectors, and the two similarities between the 20,000 feature vectors and the cluster center vectors of the two redundant clusters, that is, the computing device 20 determines a total of 40,000 similarities.

[0147] In step 602, the computing device 20 selects redundant feature vectors to be redundantly stored from a plurality of feature vectors according to the similarity between each feature vector in the feature library and the cluster center vector of the redundant cluster corresponding to the respective feature vector.

[0148] After obtaining the similarity between each feature vector in the feature library and the cluster center vector of the redundant cluster corresponding to each feature vector, the computing device 20 sorts these similarities from high to low. In one example, the computing device 20 selects the first h2 similarities from the sorting of these similarities from high to low, and determines the feature vectors corresponding to the first h2 similarities as redundant feature vectors. Among them, h2 can be considered as a preset threshold, and h2 is a positive integer. In another example, the computing device 200 selects the similarities with a ratio of the first p% from the sorting of these similarities from high to low, and determines the feature vectors corresponding to the selected similarities as redundant feature vectors, wherein p is a preset threshold, and p is a positive integer. Accordingly, these redundant feature vectors need to be redundantly stored in their corresponding redundant clusters.

[0149] Continuing with the example in step 601, the computing device 20 can sort the 40,000 similarities from high to low, and select the first h2 similarities therefrom. For example, if the value of h2 is 5000, the computing device 20 can determine 5000 similarities, and then determine the feature vectors (i.e., redundant feature vectors) corresponding to the 5000 similarities, and the redundant clusters corresponding to the feature vectors. For example, if the computing device 20 determines that the redundant feature vector and the redundant cluster are feature vector 1 and cluster 2, respectively, it means that feature vector 1 should not only be stored in cluster 1, but also needs to be redundantly stored in cluster 2.

[0150] It should be explained that the redundant feature vector is redundantly stored in the redundant cluster, which can be considered that the computing device 20 not only divides the redundant feature vector into the cluster to which it belongs, but also divides the redundant feature vector into its corresponding redundant cluster. Therefore, when the computing device 20 generates the index library, it can generate the relationship between the redundant feature vector and its corresponding redundant cluster. For example, the computing device 20 determines that the redundant feature vector and its corresponding redundant cluster are feature vector 1 and cluster 2, respectively. The index of feature vector 1 in the index library generated by the computing device 20 can be seen in Table 4.

[0151] Of course, Table 4 is only an example to illustrate the role of redundant storage. In this application, the form of the index library also needs to be understood in combination with Table 4 and Tables 1 to 3 above.

[0152] Table 4

[0153]

[0154] It is also necessary to explain that the above X first clusters can be stored in the memory 202 of the computing device 20, that is, each first cluster can occupy a part of the total storage space. When a certain feature vector is redundantly stored, the feature vector needs to be stored in the cluster to which it belongs, that is, stored in a part of the storage space corresponding to the cluster to which it belongs; the feature vector also needs to be stored in the redundant cluster corresponding to it, that is, stored in a part of the storage space corresponding to the redundant cluster corresponding to it. It can be understood that in one possible way, the computing device 20 can determine the total number of redundant feature vectors and the total number of redundant clusters corresponding to the redundant feature vectors based on the total storage space and / or the remaining storage space, that is, the computing device 20 can determine one or more of the above first redundant threshold, h1, h2, p%, and the second redundant threshold based on the total storage space and / or the remaining storage space. Among them, the storage space is, for example, the memory in the memory 202.

[0155] Through the redundant storage process, the feature vectors that most need to be redundantly stored are determined from the feature library, which can maximize the accuracy of the index library while making the most reasonable use of storage space.

[0156] It should be added that the above is explained by taking the example of the computing device 20 executing the redundant storage process after executing the clustering corresponding to the first cluster. The present application is also applicable to the computing device 20 executing the redundant storage process after completing the clustering of other levels. The "first cluster" in the above embodiment can be replaced by the cluster obtained after clustering of other levels, and no further details will be given.

[0157] In addition, after the computing device 20 completes the redundant storage process, the next level of clustering may be further performed. For example, after the computing device 20 divides the redundant feature vectors into corresponding redundant clusters, the next level of clustering is performed on the redundant clusters containing the redundant feature vectors, and then an index library is further generated.

[0158] The index library generation process and the redundant storage process are explained above. In addition, the present application also provides a feature storage method. Specifically, the computing device 20 can perform dimensionality reduction processing on the feature vectors in each second cluster after multi-level clustering, that is, after the above step 502, and then store the feature vectors after dimensionality reduction processing into a corresponding part of the storage space of the cluster to which it belongs. This helps to save storage space and reduce the time spent on data transfer.

[0159] For details, please refer to Figure 7 A flowchart of a feature storage method is shown as an example.

[0160] Step 701: The computing device 20 performs dimensionality reduction processing on each M-dimensional feature vector in the second cluster to obtain an N-dimensional feature vector corresponding to the M-dimensional feature vector.

[0161] Each feature vector in the second cluster is specifically an M-dimensional feature vector (i.e., a high-dimensional feature vector), where M is, for example, equal to 256. The computing device 20 performs dimensionality reduction processing on each M-dimensional feature vector to obtain an N-dimensional feature vector (i.e., a low-dimensional feature vector) corresponding to each M-dimensional feature vector, where N is, for example, equal to 64 dimensions.

[0162] Wherein, M and N are both positive integers, and N is less than M, or it is considered that M is an integer multiple of N.

[0163] The following explains the process of dimensionality reduction:

[0164] In step a, the computing device 20 determines a vector matrix according to a plurality of M-dimensional feature vectors in the second cluster; and determines the similarity between every two dimensions in the vector matrix.

[0165] For example, the second cluster includes a total of J M-dimensional feature vectors, and the computing device 20 can form the J M-dimensional feature vectors into a vector matrix with J rows and M columns. The computing device 20 determines the inner product between the two dimensions for every two columns (that is, every two dimensions) in the vector matrix according to the J values ​​respectively included in the two dimensions. The inner product can be used to measure the similarity between the two dimensions. The greater the similarity, the closer the two dimensions are. In this way, the computing device 20 can determine the similarity between every two dimensions in the vector matrix.

[0166] In step b, the computing device 20 divides the M dimensions in the vector matrix into N feature dimension groups on an even basis according to the similarity between every two dimensions in the vector matrix.

[0167] That is, each feature dimension group may include M / N dimensions. Combining the above-mentioned dimensionality reduction of the 256-dimensional feature vector to obtain a 64-dimensional feature vector, it can be known that each feature dimension group includes 4 dimensions.

[0168] The computing device 20 may traverse each dimension of the M dimensions. If the dimension is not grouped, the computing device 20 may search for three other dimensions closest to the dimension in all other ungrouped dimensions to form a group. If the dimension has been grouped, the computing device 20 may skip the dimension until all dimensions are grouped into feature dimension groups, thus obtaining N feature dimension groups.

[0169] It can be understood that for each M-dimensional feature vector, the M elements (i.e., numerical values) in the M-dimensional feature vector are also respectively located in their respective feature dimension groups. For example, the 1st to 4th dimensions in a 256-dimensional feature vector are located in the 1st feature dimension group, the 5th to 8th dimensions are located in the 2nd feature dimension group, and so on.

[0170] In step c, the computing device 20 maps the dimensions in the same feature dimension group in any M-dimensional feature vector into one element, thereby mapping the N feature dimension groups into N elements respectively to form an N-dimensional feature vector.

[0171] Specifically, the computing device 20 performs, for one of the N feature dimension groups: M / N dimensions in the feature dimension group are calculated by affine transformation to obtain an element. Subsequently, the computing device 20 can obtain N elements corresponding to the N feature dimension groups, respectively, and form the N elements into an N-dimensional feature vector. Thus, the M-dimensional feature vector is reduced in dimension to obtain an N-dimensional feature vector.

[0172] In addition, the data format of each M-dimensional feature vector in the second cluster is FP32, and the computing device 20 also obtains an N-dimensional feature vector of FP32 after reducing the dimension of each M-dimensional feature vector. In order to further save storage space and reduce the time consumption of data transfer, the computing device 20 can quantize the original FP32 data format to the INT8 data format, that is, obtain an N-dimensional feature vector of INT8, and store the N-dimensional feature vector of INT8.

[0173] like Figure 8 This is a schematic diagram of a 256-dimensional feature vector dimensionality reduction to obtain a 64-dimensional feature vector provided by the present application, wherein the 256-dimensional feature vector can be found in Figure 8 As shown in (a), each square represents a value in the 256-dimensional feature vector.

[0174] The computing device 20 may select four dimensions with higher similarity to form a feature dimension group, for example, Figure 8 The four shaded blocks in (a) represent the four dimensions with higher similarity, namely x hi ,x hj ,x hm ,x hn The computing device 20 has the hi ,x hj ,x hm ,x hn Form a feature dimension group, and perform affine transformation on the feature dimension group to obtain the element x lk , that is, Figure 8 Similarly, the computing device 20 reduces the dimension of the 256-dimensional feature vector to obtain a 64-dimensional feature vector as shown in FIG. Figure 8 As shown in (b).

[0175] The computing device 20 quantizes the 64-dimensional feature vector of FP32. lk For example, for element x lk Quantization is performed to obtain q j2 , that is, Figure 8 Similarly, the computing device 20 quantizes the 64-dimensional feature vector of FP32 to obtain the N-dimensional feature vector of INT8, which can be seen in Figure 8 As shown in (c).

[0176] Step 702: The computing device 20 stores the N-dimensional feature vector corresponding to each M-dimensional feature vector in the second cluster.

[0177] The computing device 20 will change from storing M-dimensional feature vectors to storing N-dimensional feature vectors, which helps save storage space and reduce data transfer time. Further, the computing device 20 determines the vector matrix based on the multiple M-dimensional feature vectors in the second cluster, and realizes dimensionality reduction processing for each M-dimensional feature vector by calculating the inner product between every two dimensions in the vector matrix. The dimensionality reduction method is simple and has a small amount of calculation.

[0178] In addition, based on the index library generated above, the present application also provides a vector retrieval method for solving the problem of large amount of calculation and slow retrieval speed when the existing index established based on space division (such as IVF) is applied to the vector retrieval process. In the vector retrieval method, the computing device 20 can search for the target feature vector from the feature library according to the query vector and the index library. For details, see Fig. 9 Flowchart shown as an example.

[0179] Step 901: The computing device 20 obtains a vector to be queried.

[0180] The vector to be queried obtained by the computing device 20 may specifically be a feature vector input by the user to the computing device 20 , so that the computing device 20 may return a target feature vector corresponding to the feature vector to the user.

[0181] Alternatively, the vector to be queried obtained by the computing device 20 may specifically be any feature vector in the feature library. That is, the computing device 20 may traverse each feature vector in the feature library, determine the target feature vector corresponding to each feature vector, and record the target feature vector corresponding to the feature vector. Exemplarily, the computing device 20 may determine the K nearest feature vectors (i.e., KNN results) of each feature vector, and record the KNN results of all feature vectors in the feature library. In this way, when the user needs to query the target feature vector corresponding to a feature vector, the target feature vector corresponding to the feature vector (such as the KNN result) can be directly returned based on the record, thereby helping to improve the efficiency of vector retrieval.

[0182] Step 902: The computing device 20 selects a target feature vector related to the query vector from the feature library according to the query vector and the index library.

[0183] The index library is specifically the above Figure 5 and Figure 6 The index library generated in the relevant embodiment. The index library includes multiple indexes of the first cluster, and each index of the first cluster further includes a cluster identifier and a cluster center vector of the first cluster (i.e., a first cluster center vector), cluster identifiers and cluster center vectors of multiple second clusters in the first cluster (i.e., a second cluster center vector), and a feature vector in each second cluster. The specific expression of the index library can be described in Tables 1 to 4.

[0184] The following is an explanation of how the computing device 20 selects the target feature vector, referring to steps (1) to (4):

[0185] Step (1): the computing device 20 searches the index library for a first index corresponding to the vector to be queried.

[0186] Optionally, the computing device 20 compares the query vector with the first cluster center vectors of multiple index records in the index library, and the comparison result that satisfies the first preset condition is the first index.

[0187] In a specific implementation, the computing device 20 may obtain n first cluster center vectors respectively recorded in n indexes in the index library, and determine n similarities between the query vector and the n first cluster center vectors respectively. The computing device 20 selects a first index whose similarity meets a first preset condition from the n indexes according to the n similarities between the query vector and the n first cluster center vectors respectively, and the first cluster corresponding to the first index may be referred to as a first candidate cluster.

[0188] The first preset condition may specifically be that the similarity between the first cluster center vector and the query vector is greater than a first similarity threshold. Alternatively, the first preset condition may specifically be that the query vector and the first K1 similarities between the n first cluster center vectors are sorted from high to low, where K1 is a positive integer.

[0189] In step (2), the computing device 20 searches the first index for a second cluster center vector corresponding to the query vector.

[0190] The first index may be one or more, and each first index may include multiple cluster center vectors of the second clusters (i.e., second cluster center vectors). Exemplarily, after determining one or more first indexes, the computing device 20 may determine m second cluster center vectors from the one or more first indexes. In combination with the example in Table 2 above, for example, the indexes of cluster 1 and cluster 2 are both first indexes, the computing device 20 may determine 3 second cluster center vectors from the index of cluster 1, namely, cluster center vector 1-1, cluster center vector 1-2, and cluster center vector 1-3; the computing device 20 may also determine 2 second cluster center vectors from the index of cluster 2, namely, cluster center vector 2-1 and cluster center vector 2-2, where m=5.

[0191] Further, the computing device 20 determines the similarity between the query vector and the m second cluster center vectors. The computing device 20 selects a second cluster center vector that meets the second preset condition from the m second cluster center vectors according to the similarity between the query vector and the m second cluster center vectors as the second cluster center vector corresponding to the query vector.

[0192] The second preset condition may specifically be that the similarity between the second cluster center vector and the query vector is greater than a second similarity threshold, and illustratively, the second similarity threshold is greater than the first similarity threshold. Alternatively, the second preset condition may also be that the query vector and the m second cluster center vectors are respectively ranked from high to low in terms of similarity, where K2 is a positive integer.

[0193] In step (3), the computing device 20 finds the second cluster (which may be referred to as the second candidate cluster) corresponding to the second cluster center vector corresponding to the query vector based on the second cluster center vector corresponding to the query vector.

[0194] In step (4), the computing device 20 searches for a target feature vector from a plurality of feature vectors included in the second candidate cluster.

[0195] Optionally, the similarity between the target feature vector and the vector to be queried meets a third preset condition. Specifically, the third preset condition may be that the similarity between the target feature vector and the vector to be queried is greater than a third similarity threshold, and illustratively, the third similarity threshold is greater than the second similarity threshold. Alternatively, the third preset condition may also be that the similarity between the target feature vector and the vector to be queried is the first K3 of the similarities between the vector to be queried and all feature vectors in the second candidate cluster, respectively, in a descending order, where K3 is a positive integer.

[0196] Specifically, after determining the second candidate cluster, the computing device 20 may obtain multiple feature vectors contained in the second candidate cluster from the index library. The second candidate cluster may be one or more, and each second candidate cluster may include multiple feature vectors of its own. Exemplarily, after determining one or more second candidate clusters, the computing device 20 obtains multiple feature vectors corresponding to the one or more second candidate clusters from the index library. Still combining the example in Table 2 above, for example, cluster 1-1 and cluster 2-1 are both second candidate clusters, wherein cluster 1-1 includes feature vectors 1-100, and cluster 2-1 includes feature vectors 301-400, and the computing device 20 can obtain 200 feature vectors. The computing device 20 determines the similarity between the query vector and the above multiple feature vectors respectively, and selects a feature vector whose similarity meets the third preset condition from the multiple feature vectors according to the similarity between the query vector and the multiple feature vectors respectively, as the target feature vector.

[0197] Optionally, the query vector may have M dimensions, which may be referred to as an M-dimensional query vector. The computing device 20 may first perform dimensionality reduction processing on the M-dimensional query vector to obtain an N-dimensional query vector. Specifically, the computing device 20 may group the M-dimensional query vector according to the dimensionality reduction grouping method in the storage process (for details, see the above Figure 7 and Figure 8Described in the relevant embodiments), the M elements corresponding to the M-dimensional vector to be queried are evenly divided into N feature dimension groups, and then the N feature dimension groups are respectively mapped to N elements to form an N-dimensional vector to be queried. After the computing device 20 performs dimensionality reduction processing on the M-dimensional vector to be queried to obtain the N-dimensional vector to be queried, the computing device 20 can obtain multiple N-dimensional feature vectors in the second candidate cluster, and then determine the similarity between the N-dimensional vector to be queried and the multiple N-dimensional feature vectors in the second candidate cluster. In this way, when calculating the similarity between two vectors, it helps to reduce the amount of calculation of the computing device 20, thereby improving the retrieval efficiency.

[0198] It should be added that the representation of the index library depends on the number of levels of clustering of the feature vectors in the feature library by the computing device 20. When the computing device 20 performs two-level clustering on the feature vectors in the feature library, the representation of the index library can be shown in Table 2. When the computing device 20 performs multi-level (for example, three-level) clustering on the feature vectors in the feature library, the representation of the index library can be shown in Table 3.

[0199] The following is an explanation of the level of clustering of feature vectors in the feature library by the computing device 20 in two cases:

[0200] Case 1: The computing device 20 performs two-level clustering on the feature vectors in the feature library.

[0201] It can be understood that the first cluster in the feature library is the first-level cluster, and the second cluster in the feature library is the second-level cluster.

[0202] When the computing device 20 determines the target feature vector based on the query vector and the index library, the "first cluster" in the above steps (1) to (4) can be replaced by "first-level cluster", the "second cluster" can be replaced by "second-level cluster", the "first candidate cluster" can be replaced by "first-level candidate cluster", and the "second candidate cluster" can be replaced by "second-level candidate cluster".

[0203] Exemplarily, the computing device 20 searches for a first index corresponding to the query vector from the index library, the first index corresponding to the first-level candidate cluster, and the first index includes cluster center vectors of multiple second-level clusters included in the first-level candidate cluster (i.e., second-level cluster center vectors). The computing device 20 searches for the second-level cluster center vector corresponding to the query vector from the first index, and finds the second-level candidate cluster corresponding to the second-level cluster center vector based on the second-level cluster center vector. The target feature vector is searched from the multiple feature vectors included in the selected second-level candidate cluster.

[0204] Case 2: The computing device 20 performs multi-level clustering on the feature vectors in the feature library.

[0205] Taking three-level clustering as an example, when the computing device 20 determines the target feature vector based on the query vector and the index library, it can replace the "first cluster" in the above steps (1) to (4) with the "second-level cluster", the "second cluster" with the "third-level cluster", the "first candidate cluster" with the "second-level candidate cluster", and the "second candidate cluster" with the "third-level candidate cluster".

[0206] Before step (1), the computing device selects a first-level candidate cluster from multiple first-level clusters, wherein the first-level candidate cluster includes a total of multiple second-level clusters, and the index library includes indexes corresponding to the multiple second-level clusters. Further, the computing device 20 searches for a first index corresponding to the vector to be queried from the indexes corresponding to the multiple second-level clusters according to the vector to be queried, and the first index corresponds to the second candidate cluster. The computing device 20 searches for a third-level cluster center vector corresponding to the vector to be queried from the first index, finds a third-level candidate cluster corresponding to the third-level cluster center vector according to the third-level cluster center vector, and searches for a target feature vector from the multiple feature vectors included in the third-level candidate cluster.

[0207] When the computing device 20 performs more-level clustering on the feature vectors in the feature library, the determination method is similar to that in Case 1 and Case 2, and will not be described in detail.

[0208] Optionally, in the above step 902, the data format of the target feature vector obtained by the computing device 20 is INT8, for details, see Figure 7 and Figure 8 The relevant embodiments are described. Subsequently, the computing device 20 may first perform a dequantization process on the INT8 target feature vector to obtain an N-dimensional feature vector in a data format of FP32.

[0209] It should be noted that the steps in the above method embodiment are all described by taking the computing device 20 as an example. In addition, the steps in the above method embodiment can also be executed by the processor 201 in the computing device 20. Fig.10 This is a schematic diagram of a flow chart of a processor 201 in a computing device 20 performing vector retrieval in a two-level clustering case provided by the present application. The structure of the processor 201 in the computing device 20 can be found in Figure 3 shown.

[0210] Among them, the CPU can be used to execute operation instructions, similarity sorting, vector dimension reduction, etc., and the NPU can be used to perform similarity calculations between vectors. For details, please refer to Fig.10 As shown:

[0211] The CPU can obtain the vector to be queried input by the user through the communication interface 203 .

[0212] In the first-level vector retrieval: the CPU sends the first-level operation instruction to the NPU, wherein the first-level operation instruction includes a similarity operator, a vector to be queried, and cluster center vectors of multiple first-level clusters (i.e., first cluster center vectors). Here, the similarity operator is used to instruct the NPU to calculate the similarity between the vector to be queried and the cluster center vectors of multiple first-level clusters. Correspondingly, the NPU performs the first-level similarity calculation, i.e., the NPU calculates the similarity between the vector to be queried and the cluster center vectors of multiple first-level clusters, and reports the similarity calculation results to the CPU. The CPU performs the first-level similarity sorting according to the similarity calculation results, and then selects one or more first-level candidate clusters (i.e., first candidate clusters) from the multiple first-level clusters. Among them, the one or more first-level candidate clusters each correspond to their own multiple second-level clusters, i.e., the CPU can further determine multiple second-level clusters.

[0213] In the second-level vector retrieval: the CPU sends the second-level operation instruction to the NPU, wherein the second-level operation instruction includes a similarity operator, a vector to be queried, and cluster center vectors of multiple second-level clusters (i.e., second cluster center vectors). Here, the similarity operator is used to instruct the NPU to calculate the similarity between the vector to be queried and the cluster center vectors of multiple second-level clusters. Correspondingly, the NPU performs the second-level similarity calculation, i.e., the NPU calculates the similarity between the vector to be queried and the cluster center vectors of multiple second-level clusters, and then reports the similarity calculation result to the CPU. The CPU performs the second-level similarity sorting according to the similarity calculation result, and then selects one or more second-level candidate clusters (i.e., second candidate clusters) from the multiple second-level clusters. Among them, the one or more second-level candidate clusters each correspond to their own multiple feature vectors, i.e., the CPU can further determine multiple feature vectors. In addition, in the second-level vector retrieval, the CPU can also perform dimensionality reduction processing on the vector to be queried, i.e., reduce the M-dimensional vector to be queried to an N-dimensional vector to be queried.

[0214] In the third-level vector retrieval: the CPU sends the third-level operation instruction to the NPU, wherein the third-level operation instruction includes a similarity operator, a vector to be queried, and multiple feature vectors determined in the second-level vector retrieval. Here, the similarity operator is used to instruct the NPU to calculate the similarity between the vector to be queried and the multiple feature vectors. Accordingly, the NPU performs the third-level similarity calculation, that is, the NPU calculates the similarity between the vector to be queried and the multiple feature vectors, and then reports the similarity calculation result to the CPU. The CPU performs the third-level similarity sorting according to the similarity calculation result, and then selects one or more feature vectors from the multiple feature vectors as the target feature vector.

[0215] The CPU then feeds back the determined target feature vector to the user via the communication interface 203 .

[0216] Further, the CPU may include 6 CPU cores, which are respectively represented as CPU core 0 to CPU core 5, wherein CPU core 0 may be used to issue first-level operation instructions, issue second-level operation instructions, and issue third-level operation instructions, and obtain the query vector input by the user and feed back the target feature vector to the user through the communication interface 203. CPU core 0 to CPU core 5 may be used to jointly perform first-level similarity sorting, second-level similarity sorting, third-level similarity sorting, and vector dimensionality reduction.

[0217] The NPU may include two NPU cores, which are respectively represented as NPU core 0 and NPU core 1, wherein NPU core 0 and NPU core 1 can be used to jointly perform the first-level similarity calculation, the second-level similarity calculation and the third-level similarity calculation.

[0218] For the above first-level vector search to third-level vector search, please refer to Fig. 9 It has been described in the relevant embodiments and will not be repeated here.

[0219] Based on the above content and the same concept, an embodiment of the present application provides a computer-readable storage medium, on which a computer program or instruction is stored. When the computer program or instruction is executed, the computer executes the method in the above method embodiment.

[0220] Based on the above content and the same concept, an embodiment of the present application provides a computer program product. When a computer reads and executes the computer program product, the computer executes the method in the above method embodiment.

[0221] It is understood that the various numbers involved in the embodiments of the present application are only for the convenience of description and are not used to limit the scope of the embodiments of the present application. The size of the sequence number of the above-mentioned processes does not mean the order of execution, and the execution order of each process should be determined by its function and internal logic.

[0222] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the scope of protection of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A vector retrieval method, characterized in that: include: Searching for a first index corresponding to the query vector from an index library, wherein the index library includes multiple indexes, each index corresponds to multiple second clusters, each index records multiple second cluster center vectors, and the feature vectors corresponding to each second cluster are clustered with each second cluster center vector; Determine a second cluster center vector corresponding to the query vector from a plurality of second cluster center vectors recorded in the first index; determining a second cluster corresponding to the center vector of the second cluster; A target feature vector related to the query vector is searched from the plurality of feature vectors included in the second cluster.

2. The method according to claim 1, characterized in that Each of the indexes corresponds to a first cluster, the first cluster includes a first cluster center vector, the feature vectors corresponding to the first cluster are clustered with the first cluster center vector, the first cluster includes the plurality of second clusters, and searching the index library for a first index corresponding to the query vector includes: The query vector is compared with first cluster center vectors of a plurality of index records in the index library, and the first index is the one whose comparison result satisfies a first preset condition.

3. The method according to claim 1, characterized in that The query vector is of M dimension, where M is a positive integer, and the feature vector in the second cluster is of N dimension, where N is a positive integer and is smaller than M; The step of searching for a target feature vector related to the query vector from the plurality of feature vectors included in the second cluster includes: Reducing the dimension of the query vector from M dimension to N dimension; Determine the similarity between the query vector and the plurality of feature vectors in the second cluster respectively; The target feature vector is selected from the plurality of feature vectors according to the similarity.

4. The method according to any one of claims 1 to 3, characterized in that Also includes: Determine X first cluster center vectors, and cluster the feature vectors in the feature library into X first clusters with each of the X first cluster center vectors as the center; For each first cluster in the X first clusters, determine Y second cluster center vectors, and cluster the feature vectors in the first cluster into Y second clusters with the Y second cluster center vectors as the center; For each first cluster, generate an index, where the index is used to record the first cluster center vector and the second cluster center vector corresponding to the first cluster center vector; Wherein, X and Y are both positive integers.

5. The method according to claim 4, characterized in that After clustering the feature vectors in the first cluster into Y second clusters with the Y second cluster center vectors as the center, the method further includes: Selecting a third cluster from the Y second clusters, wherein the number of feature vectors in the third cluster is less than a threshold; Determine a second cluster center vector that is closest to the second cluster center vector of the third cluster; The feature vector of the third cluster is merged into the second cluster corresponding to the determined center vector of the second cluster.

6. A vector search device, characterized in that: The invention comprises a processor, wherein the processor is connected to a memory, wherein the memory is used to store a computer program, and when the processor executes the computer program stored in the memory, it is used to: Searching for a first index corresponding to the query vector from an index library, wherein the index library includes a plurality of indexes, each index corresponds to a plurality of second clusters, each index records a plurality of second cluster center vectors, and the feature vectors corresponding to each second cluster are clustered with each second cluster center vector; Determine a second cluster center vector corresponding to the query vector from a plurality of second cluster center vectors recorded in the first index; determining a second cluster corresponding to the center vector of the second cluster; A target feature vector related to the query vector is searched from the plurality of feature vectors included in the second cluster.

7. The device according to claim 6, characterized in that Each of the indexes corresponds to a first cluster, the first cluster includes a first cluster center vector, the feature vectors corresponding to the first cluster are clustered with the first cluster center vector, the first cluster includes the plurality of second clusters, and when the processor searches the index library for the first index corresponding to the vector to be queried, is specifically configured to: The query vector is compared with first cluster center vectors of a plurality of index records in the index library, and the first index is the one whose comparison result satisfies a first preset condition.

8. The device according to claim 6, characterized in that The vector to be queried is of M dimension, where M is a positive integer, and the feature vectors in the second cluster are of N dimension, where N is a positive integer and N is less than M; when the processor searches for a target feature vector related to the vector to be queried from the plurality of feature vectors included in the second cluster, the processor is specifically configured to: Reducing the dimension of the query vector from M dimension to N dimension; Determine the similarity between the query vector and the plurality of feature vectors in the second cluster respectively; The target feature vector is selected from the plurality of feature vectors according to the similarity.

9. The device according to any one of claims 6 to 8, characterized in that The processor is further configured to: Determine X first cluster center vectors, and cluster the feature vectors in the feature library into X first clusters with each of the X first cluster center vectors as the center; For each first cluster in the X first clusters, determine Y second cluster center vectors, and cluster the feature vectors in the first cluster into Y second clusters with the Y second cluster center vectors as the center; For each first cluster, generate an index, where the index is used to record the first cluster center vector and the second cluster center vector corresponding to the first cluster center vector; Wherein, X and Y are both positive integers.

10. The device according to claim 9, characterized in that After clustering the feature vectors in the first cluster into Y second clusters with the Y second cluster center vectors as the center, the processor is further configured to: Selecting a third cluster from the Y second clusters, wherein the number of feature vectors in the third cluster is less than a threshold; Determine a second cluster center vector that is closest to the second cluster center vector of the third cluster; The feature vector of the third cluster is merged into the second cluster corresponding to the determined center vector of the second cluster.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program or instruction. When the computer program or instruction is executed by the vector search device, the method according to any one of claims 1 to 5 is implemented.

12. A chip, characterized in that: The system comprises at least one processor and an interface; the interface is used to provide program instructions or data to the at least one processor; the at least one processor is used to execute the program line instructions to implement the method as claimed in any one of claims 1 to 5.

Images