Inverted index construction method of vector database and related product

Abstract

The invention provides a method for constructing a reverse index of a vector database and related products, and the method comprises the following steps: constructing a reverse index, which at least comprises the following steps: clustering each vector to obtain a set number of clusters; judging whether vectors in the vector database change or not; if yes, obtaining a vector variable quantity after the inverted index is constructed, and judging whether a preset index updating condition is triggered or not according to the vector variable quantity; and under the condition that the index updating condition is triggered, returning to the step of constructing the reverse index. According to the technical scheme, the reverse index of the vector database can be automatically updated, and the problem that the reliability of the database is affected due to frequent construction of the reverse index of the vector database is solved.

Description

Technical Field

[0001] The present invention relates to the field of database technology, and in particular to a method for constructing an inverted index of a vector database and related products. Background Art

[0002] The working principle of the vector database is to convert unstructured data such as images and texts into structured vector data, and store the vector data in the database. After receiving the user's image or text query instruction, the image or text to be queried will also be converted into vector data, and the vector data closest to the vector data will be found in the database and returned to the user, thereby improving the efficiency of the database in executing data queries.

[0003] In the process of searching for nearest neighbor vector data in a vector database in the prior art, a query method based on an IVF (Inverted FileSystem) algorithm is a commonly used vector query method. This query method needs to first cluster each vector in the vector database to obtain multiple clusters, and obtain the centroid vector of each cluster to establish an inverted index; then, after receiving a vector query instruction, calculate the distance between each centroid vector and the query vector to obtain the centroid vector with the smallest distance to the query vector, and find the neighbor vector of the query vector in the cluster corresponding to the centroid vector.

[0004] According to the above content, if the accuracy of the query method based on the IVF algorithm is to be guaranteed, the accuracy of the clustering of each vector in the vector database needs to be guaranteed. After clustering each vector in the vector database to obtain multiple clusters, if a vector in the vector database is modified or a new vector is added to the vector database, the staff needs to delete the original inverted index and build a new inverted index. Frequent construction of inverted indexes will consume a lot of time of the vector database, and during the process of inverted index construction, the vector database cannot provide vector query services, which will cause the problem of reduced reliability of the vector database. Summary of the invention

[0005] The purpose of the present invention is to provide an inverted index construction method and related products for a vector database, which can automatically update the inverted index of the vector database and solve the problem that the reliability of the database is affected by frequent construction of the inverted index of the vector database.

[0006] In particular, the present invention provides a method for constructing an inverted index of a vector database, comprising:

[0007] Constructing an inverted index, which at least includes: clustering each vector to obtain a set number of clusters;

[0008] Determining whether a vector in the vector database has changed;

[0009] If yes, then obtaining the vector change amount after the inverted index is constructed, and judging whether a preset index update condition is triggered according to the vector change amount; and

[0010] When the index update condition is triggered, return to the step of building the inverted index.

[0011] Furthermore, the step of judging whether a preset index update condition is triggered according to the vector change amount includes:

[0012] Obtaining the total amount of vectors after constructing the inverted index, and calculating the ratio of the vector change to the total amount of vectors;

[0013] Determining whether the ratio is greater than or equal to a set threshold;

[0014] If so, it is determined that the index update condition is triggered.

[0015] Further, the vector database is provided with a vector update parameter and / or a total vector parameter, wherein the vector update parameter is used to record the vector change amount, and the total vector parameter is used to record the vector total amount; and

[0016] After the step of constructing the inverted index, the method further includes: updating the total vector parameter, and / or resetting the vector update parameter; and / or

[0017] After the step of determining whether the vector in the vector database has changed, the method further includes: if the vector in the vector database has changed, updating the vector update parameter.

[0018] Furthermore, the step of clustering each vector to obtain a set number of clusters includes:

[0019] Obtaining a set number of initial centroid vectors, respectively calculating the distance between each of the vectors and each of the initial centroid vectors, and assigning each of the vectors to an initial cluster corresponding to the initial centroid vector closest to the vector;

[0020] The centroid vector of each of the initial clusters is obtained, the distance between each of the vectors and each of the centroid vectors is calculated respectively, and each of the vectors is assigned to the cluster corresponding to the centroid vector closest to the vector.

[0021] Furthermore, the step of obtaining a set number of initial centroid vectors includes:

[0022] Selecting a vector from each of the vectors and storing it in a centroid vector set; and

[0023] When the number of vectors in the mass centroid vector set is less than the set number:

[0024] Calculating the distance between each of the vectors and each of the vectors in the mass center vector set to obtain the minimum distance between each of the vectors and each of the vectors in the mass center vector set, and storing the vector with the largest minimum distance among the vectors into the mass center vector set;

[0025] When the number of vectors in the mass center vector set is equal to the set number:

[0026] The vector in the centroid vector set is used as the initial centroid vector.

[0027] Furthermore, the step of calculating the centroid vector of each of the initial clusters includes:

[0028] The average vector of each vector in each of the initial clusters is calculated respectively, and each of the average vectors is used as the centroid vector of the corresponding initial cluster.

[0029] Furthermore, after the step of constructing the inverted index of the vector database, the method further includes:

[0030] Determine whether a vector insertion request is received;

[0031] If yes, then obtaining a newly added vector and inserting the newly added vector into the vector database; and

[0032] After the step of determining whether a preset index update condition is triggered according to the vector change amount, the method further includes:

[0033] If the index update condition is not triggered, the cluster corresponding to the newly added vector is obtained, and the newly added vector is allocated to the corresponding cluster.

[0034] Furthermore, after the step of constructing the inverted index, the method further includes:

[0035] Determine whether an index update request instruction is received;

[0036] If yes, return to the step of constructing the inverted index.

[0037] According to another aspect of the present invention, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of any of the above-mentioned methods for constructing an inverted index are implemented.

[0038] According to yet another aspect of the present invention, a computer program product is provided, comprising a computer program, which implements the steps of any of the above-mentioned methods for constructing an inverted index when executed by a processor.

[0039] The technical solution provided by the present invention determines whether a preset index update condition is triggered according to the vector change amount of the vector database after the vector database constructs the inverted index, and rebuilds the inverted index when the index update condition is triggered. Due to the technical solution of the present invention, the vector database will rebuild the inverted index only after the vector change amount triggers the preset index update condition. Therefore, not only can the inverted index of the vector database be automatically updated, but also the frequency of the vector database constructing the inverted index can be reduced, avoiding the problem that the vector database consumes a lot of time due to frequent construction of the inverted index, so as to achieve the purpose of improving the reliability of the vector database.

[0040] Based on the following detailed description of specific embodiments of the present invention in conjunction with the accompanying drawings, those skilled in the art will become more aware of the above and other objects, advantages and features of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Hereinafter, some specific embodiments of the present invention will be described in detail in an exemplary and non-limiting manner with reference to the accompanying drawings. The same reference numerals in the accompanying drawings indicate the same or similar components or parts. It should be understood by those skilled in the art that these drawings are not necessarily drawn to scale. In the accompanying drawings:

[0042] Figure 1 is a schematic flow chart of a method for constructing an inverted index of a vector database according to an embodiment of the present invention;

[0043] Figure 2 is a schematic flow chart of determining whether a preset index update condition is triggered in a method for constructing an inverted index according to an embodiment of the present invention;

[0044] Figure 3 is a schematic flow chart of clustering vectors in an inverted index construction method according to an embodiment of the present invention;

[0045] Figure 4 is a schematic diagram of clustering vectors in an inverted index construction method according to an embodiment of the present invention;

[0046] Figure 5 is a schematic flow chart of obtaining multiple initial centroid vectors in an inverted index construction method according to an embodiment of the present invention;

[0047] Figure 6 is a schematic diagram of allocating a newly added vector to its corresponding cluster in an inverted index construction method according to an embodiment of the present invention;

[0048] Figure 7 is a schematic flow chart of a method for constructing an inverted index of a vector database according to another embodiment of the present invention;

[0049] Figure 8 is a schematic diagram of a computer program product according to an embodiment of the present invention;

[0050] Fig. 9 is a schematic diagram of a computer-readable storage medium according to an embodiment of the present invention. DETAILED DESCRIPTION

[0051] Refer to the following Figures 1 to 9 To describe an inverted index construction method for a vector database and related products according to an embodiment of the present invention. In the description of this embodiment, it should be understood that the terms "first" and "second" are used for descriptive purposes only, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features, that is, include one or more of the features. In the description of the present invention, the meaning of "multiple" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined. When a feature "includes or contains" one or some of the features it covers, unless otherwise specifically described, this indicates that other features are not excluded and may further include other features.

[0052] This embodiment provides a method for constructing an inverted index of a vector database, which can ensure automatic updating of the inverted index of the vector database and solve the problem that a large amount of time is consumed in constructing the inverted index of the vector database due to frequent construction of the inverted index.

[0053] Figure 1 is a schematic flow chart of a method for constructing an inverted index of a vector database according to an embodiment of the present invention. The method may generally include:

[0054] Step S102: constructing an inverted index of the vector database;

[0055] Step S104: determining whether the vector in the vector database has changed;

[0056] If yes, execute step S106;

[0057] Step S106: obtaining the vector change amount of the vector database after the inverted index is constructed;

[0058] Step S108: judging whether a preset index update condition is triggered according to the above vector change amount;

[0059] If yes, return to step S102 to rebuild the inverted index of the vector database.

[0060] In the above step S102, the method for constructing an inverted index of a vector database includes:

[0061] First, a preset clustering method is used to cluster each vector according to the distance between each vector in the vector database, so as to cluster each vector into a first set number of clusters, and obtain the centroid vector of each cluster to complete the construction of the inverted index of the vector database.

[0062] In this embodiment, after constructing an inverted index of a vector database, a method for performing vector query according to the inverted index includes:

[0063] After receiving the vector query instruction, the query vector is first obtained, and the distance between each centroid vector and the query vector is calculated; then the centroid vector with the smallest distance to the query vector is used as the target centroid vector, and the cluster corresponding to the target centroid vector is used as the target cluster;

[0064] Finally, the distance between each vector in the target cluster and the query vector is calculated, and the vectors in the target cluster are arranged in descending order according to the distance, and the last second set number of vectors in the sorting are used as the neighbor vectors of the query vector to complete the vector query.

[0065] In the above step S104, the vector in the vector database changes, which means that the vector in the vector database is modified, or a new vector is inserted into the vector database. Therefore, when it is detected that there is a vector modification operation in the vector database, or when it is detected that there is a new vector inserted into the vector database, it is determined that the vector in the vector database has changed.

[0066] In the above step S106, the vector change amount of the vector database refers to the number of vectors that have changed in the vector database, that is, the vector change amount of the vector database is the number of vectors modified in the vector database, or the number of vectors inserted in the vector database.

[0067] In the above step S108, if the vector change in the vector database is small, it will not affect the vector query; if the vector change in the vector database is large, it can be considered that there are many vectors that have changed in the vector database, and the accuracy of the established inverted index is poor, so return to step S102 to rebuild the inverted index of the vector database to update the inverted index of the vector database.

[0068] According to the above content, it can be known that in the technical solution of this embodiment, after the vector database constructs the inverted index, it is determined whether the preset index update condition is triggered according to the vector change amount of the vector database, and the inverted index is rebuilt when the index update condition is triggered. Due to the embodiment, the vector database will only rebuild the inverted index after the vector change amount triggers the preset index update condition, so not only can the inverted index of the vector database be automatically updated, but also the frequency of the vector database constructing the inverted index can be reduced, avoiding the problem of reduced reliability of the vector database due to spending a lot of time frequently constructing the inverted index.

[0069] In some embodiments of the present invention, in the above step S108, the method for determining whether to trigger the preset index update condition according to the vector change amount of the vector database is as follows: Figure 2 As shown, the following steps are included:

[0070] Step S112: Obtain the total amount of vectors in the vector database after building the inverted index;

[0071] Step S114: Calculate the ratio between the vector change amount and the vector total amount in the vector database;

[0072] Step S116: determining whether the above ratio is greater than or equal to a set threshold;

[0073] If yes, execute step S118;

[0074] Step S118: determining that a preset index update condition is triggered.

[0075] In this embodiment, the total amount of vectors in the vector database is the total number of vectors in the vector database. Assume that the total amount of vectors in the vector database after building the inverted index is N, and the vector change is M. Therefore, in the above step S114, assume that the ratio of the vector change to the total amount of vectors in the vector database is P, then:

[0076]

[0077] The above-mentioned threshold value can be set according to the requirements. In this embodiment, the threshold value is preferably set to 20%, that is, when the value of P is greater than or equal to 20%, it is determined to trigger the preset index update condition.

[0078] Since a change in a vector in a vector database means that a vector in the vector database is modified, or a new vector is inserted into the vector database, in this embodiment, if after building an inverted index, the proportion of modified vectors in the vector database is greater than or equal to a set threshold, or the proportion of inserted vectors in the vector database is greater than or equal to a set threshold, it is determined to trigger a preset index update condition to ensure the timeliness of updating the inverted index of the vector database.

[0079] In some embodiments of the present invention, a vector update parameter and a total vector parameter are set in the vector database, wherein the vector update parameter is used to record the vector change amount of the vector database, and the total vector parameter is used to record the total amount of vectors in the vector database.

[0080] After the step of constructing the inverted index of the vector database in the above step S102, the step of updating the total vector parameter and resetting the vector update parameter is also included, so that the total vector parameter can record the total amount of vectors in the vector database after the inverted index is constructed, and the vector update parameter starts to record the vector change amount in the vector database.

[0081] Accordingly, in the above step S112, the total amount of vectors in the vector database after the inverted index is constructed can be quickly obtained according to the record of the total vector parameter.

[0082] After the step of determining whether the vector in the vector database has changed in the above step S104, the method further includes: if the vector in the vector database has changed, updating the vector update parameter.

[0083] For example, if it is determined that the vectors in the vector database have changed, the number of changed vectors is obtained, and then the number is added to the vector update parameter to obtain the updated vector update parameter, so that the vector update parameter can record the vector change amount of the vector database after the inverted index is built.

[0084] Accordingly, in the above step S106, the vector change amount of the vector database after the inverted index is constructed can be quickly obtained according to the record of the vector update parameter.

[0085] Through the technical solution of this embodiment, the vector change amount and the total vector amount of the vector database can be quickly obtained according to the update parameters and the total vector parameters in the vector database, thereby improving the work efficiency of determining whether the preset index update condition is triggered.

[0086] In some embodiments of the present invention, in the process of constructing the inverted index of the vector database in the above step S102, the kmeans++ algorithm is used to cluster the vectors in the vector database. The specific clustering method is as follows: Figure 3As shown, the following steps are included:

[0087] Step S202: obtaining a first set number of initial centroid vectors, and respectively calculating the distance between each vector in the vector database and each initial centroid vector;

[0088] Step S204: assigning each vector in the vector database to the initial cluster corresponding to the nearest initial centroid vector, and calculating the centroid vector of each initial cluster;

[0089] Step S206: determining whether the obtained centroid vector satisfies a preset update stop condition;

[0090] If yes, execute step S208; if no, return to step S202;

[0091] Step S208: Calculate the distance between each vector in the vector database and the initial centroid vector, and assign each vector in the vector database to the cluster corresponding to the centroid vector with the closest distance.

[0092] In the above step S202, a first set number of initial centroid vectors may be determined first, and then a first set number of initial centroid vectors may be selected from each vector in the vector database according to a preset centroid determination method. The distance between each vector in the vector database and each initial centroid vector calculated may be a Euclidean distance, a negative inner product distance, a cosine distance, or a user-defined distance.

[0093] In the above step S204, a preset centroid vector calculation method may be used to calculate the centroid vector of each initial cluster. The centroid vector may be a vector existing in the corresponding initial cluster or may not be a vector existing in the corresponding initial cluster.

[0094] In the above step S206, the preset update stop condition includes: the acquired centroid vector no longer changes, or the number of iterative updates of the centroid vector reaches a preset number of iterations.

[0095] In this embodiment, the centroid vector no longer changes, which means that the centroid vectors obtained by at least two consecutive iterative updates remain consistent. At this time, it can be considered that the clustering of each vector in the vector database has reached a convergence effect, so the iterative update of the centroid vectors of each initial clustering cluster is stopped.

[0096] In this embodiment, the above-mentioned preset number of iterations is 40 times. When the number of iterations of the centroid vector reaches the preset number of iterations, it can be considered that the clustering of each vector in the vector database has achieved a good convergence effect. Therefore, in order to avoid consuming a long time for clustering each vector, the iterative update of the centroid vector of each initial clustering cluster is stopped.

[0097] In the above step S208, the cluster center of each initial cluster can be calculated using a preset cluster center calculation method to obtain the centroid vector of each initial cluster. Then, the preset distance calculation method is used again to calculate the distance between each vector in the vector database and each centroid vector.

[0098] Take one of the vectors as an example, after calculating the distance between the vector and each centroid vector, obtain the centroid vector with the smallest distance to the vector, and use the cluster corresponding to the centroid vector as the cluster of the vector. And so on, until each vector in the vector database is assigned to the corresponding cluster, and the clustering of each vector in the vector database is completed.

[0099] Taking an application scenario as an example, the clustering of each vector in the vector database is as follows: Figure 4 As shown, the vectors in the vector database are Figure 4 The circular points in the figure represent the clusters formed after clustering according to the distance between the vectors, and the square points represent the centroid vectors of the corresponding clusters.

[0100] The original distribution of each vector in the vector database is as follows Figure 4 After obtaining each initial centroid vector and assigning each vector to the corresponding initial cluster, the clustering result is as follows: Figure 4 As shown in part b of . Then calculate the centroid vector in each initial cluster, as Figure 4 Finally, the vectors in the vector database are assigned to the corresponding clusters, as shown in part c of Figure 4 As shown in part d in , the clustering results of each vector in the vector database are obtained.

[0101] Through the technical solution of this embodiment, the initial clustering center can be obtained first to perform initial clustering on each vector in the vector database to obtain multiple initial clustering clusters, and then the vectors in the vector database can be accurately clustered according to the centroid vector of each initial clustering cluster to improve the accuracy of clustering each vector in the vector database, thereby improving the accuracy of establishing an inverted index of the vector database.

[0102] In some embodiments of the present invention, in the above step S202, the method for obtaining the first set number of initial centroid vectors is as follows: Figure 5 As shown, the following steps are included:

[0103] Step S222: Select a vector from the vector database and store it in the centroid vector set;

[0104] Step S224: Calculate the distance between each vector in the vector database and each vector in the centroid vector set to obtain the minimum distance between each vector in the vector database and each vector in the centroid vector set;

[0105] Step S226: obtaining the vector with the largest minimum distance among the vectors in the vector database, and adding the vector to the centroid vector set;

[0106] Step S228: determining whether the number of vectors in the centroid vector concentration is less than a first set number;

[0107] If not, execute step S230; if yes, return to step S224;

[0108] Step S230: taking the vector in the centroid vector set as the initial centroid vector.

[0109] Taking an application scenario as an example, first a vector is randomly selected from a vector database and stored in a centroid vector set. At this time, the selected vector can be used as the first initial centroid vector; then the vector with the largest distance to the first initial centroid vector is selected from the remaining vectors in the vector database and stored in the centroid vector set. At this time, the vector selected this time can be used as the second initial centroid vector.

[0110] Next, the distances between the remaining vectors in the vector database and the first initial centroid vector and the distances between the vectors and the second initial centroid vector are calculated respectively, and the minimum distance is used as the minimum distance between the corresponding vector and each vector in the centroid vector set. Taking one of the vectors as an example, let the distance between the vector and the first initial centroid vector be the first distance, and the distance between the vector and the second initial centroid vector be the second distance, and let the minimum value between the first distance and the second distance be the minimum distance between the vector and each vector in the centroid vector set. Finally, the vector with the largest minimum distance among the remaining vectors in the vector database is stored in the centroid vector set.

[0111] And so on, until a first set number of vectors are stored in the centroid vector set to obtain a first set number of initial centroid vectors.

[0112] According to the above content, it can be seen that through the technical solution of this embodiment, the distance between each initial centroid vector can be maximized, preventing the distance between each initial centroid vector from being small and affecting the clustering effect of each vector, so as to ensure the reliability of the selected initial centroid vector.

[0113] In other embodiments, if the preset index update condition is triggered in the above step S108 and the process returns to step S102 to reconstruct the inverted index of the vector database, the existing centroid vector can be used as the initial centroid vector to improve the work efficiency of updating the inverted index of the vector database.

[0114] In some embodiments of the present invention, the method of calculating the centroid vector of each initial cluster in step S208 includes:

[0115] The average vector of each vector in each initial cluster is calculated respectively, and each average vector is used as the centroid vector of the corresponding initial cluster.

[0116] In this embodiment, it is assumed that there are n vectors in one of the initial clusters, and the jth component vector of the i-th vector is W ij , the jth component vector of the centroid vector of the initial cluster is but

[0117]

[0118] The component vectors of the centroid vector of the initial clustering cluster are calculated according to the above formula to obtain the centroid vector of the initial clustering cluster.

[0119] The technical solution of this embodiment can ensure the accuracy of calculating the centroid vector of each initial cluster, so as to improve the reliability of clustering each vector in the vector database.

[0120] In some embodiments of the present invention, after the step of constructing an inverted index of the vector database in step S102, the following steps are further included:

[0121] After receiving the vector insertion request, the newly added vector to be inserted is obtained, and the newly added vector is inserted into the vector database.

[0122] After determining whether a preset index update condition is triggered according to the vector change amount of the vector database in the above step S108, the following further includes:

[0123] If the preset index update condition is not triggered, the cluster corresponding to the newly added vector is obtained, and the newly added vector is assigned to the corresponding cluster.

[0124] In this embodiment, the method of allocating the newly added vector to its corresponding cluster includes:

[0125] First, calculate the distance between the newly added vector and the centroid vector of each cluster;

[0126] Then, each vector in the vector database is assigned to the cluster corresponding to the nearest centroid vector.

[0127] Take a practical application scenario as an example. Figure 6 As shown, after constructing the inverted index of the vector database in the above step S102, the clustering results of each vector in the vector database are as follows: Figure 6 As shown in part a of FIG; after the newly added vector is inserted into the vector database, the distribution of each vector in the vector database is as follows Figure 6 As shown in part b of ; after assigning the newly added vectors to the corresponding clusters, the clustering results of each vector in the vector database are as follows Figure 6 As shown in part c.

[0128] Through the technical solution of this embodiment, after receiving a vector insertion request, the vector database can insert the newly added vector to be inserted into the vector database and allocate the newly added vector to the corresponding cluster to improve the reliability of the inverted index.

[0129] In some embodiments of the present invention, after constructing the inverted index of the vector database in the above step S102, the following is further included:

[0130] Determine whether an index update request instruction is received;

[0131] If yes, the process returns to step S102 to execute the step of constructing an inverted index of the vector database again.

[0132] In this embodiment, if the staff believes that the inverted index of the vector database needs to be updated, for example, the inverted index of the vector database has not been updated within a set time period, an index update request instruction can be input into the vector database. After receiving the index update request instruction input by the user, the vector database reconstructs the inverted index of the vector database to update the inverted index of the vector database.

[0133] It can be seen from the above content that the technical solution of this embodiment can improve the flexibility of updating the inverted index of the vector database, so as to achieve the purpose of improving the reliability of the vector database.

[0134] In some embodiments of the present invention, the process of the inverted index construction method of the vector database of the present invention is as follows: Figure 7 As shown, the following steps are included:

[0135] Step S302: obtaining a first set number of initial centroid vectors, and respectively calculating the distance between each vector in the vector database and each initial centroid vector;

[0136] Step S304: assigning each vector in the vector database to the initial cluster corresponding to the nearest initial centroid vector, and calculating the centroid vector of each initial cluster;

[0137] Step S306: determining whether the obtained centroid vector satisfies a preset update stop condition;

[0138] If yes, execute step S308; if no, return to step S302;

[0139] Step S308: Calculate the distance between each vector in the vector database and the initial centroid vector, and assign each vector in the vector database to the cluster corresponding to the centroid vector with the closest distance, so as to construct an inverted index of the vector database;

[0140] Step S310: updating the total vector parameters of the vector database, and resetting the vector update parameters of the vector database;

[0141] Step S312: Determine whether an index update request instruction is received, or whether a new vector is inserted;

[0142] If an index update instruction is received, return to step S302;

[0143] If a new vector is inserted, execute step S314;

[0144] Step S314: updating the vector update parameter, and calculating the ratio between the vector change amount and the total vector amount of the vector database after the inverted index is constructed according to the vector update parameter and the total vector parameter;

[0145] Step S316: Determine whether the above ratio is greater than or equal to a set threshold;

[0146] If yes, then return to step S302; if no, then execute step S318;

[0147] Step S318: Obtain the cluster corresponding to the newly added vector, and assign the newly added vector to the corresponding cluster.

[0148] In the above step S302, if the inverted index of the vector database is constructed for the first time, a first set number of initial centroid vectors are selected from each vector in the vector database; if the inverted index of the vector database has been constructed, the centroid vectors of the existing clusters are used as the initial centroid vectors.

[0149] In the above step S306, the preset update stop condition includes that the acquired centroid vector no longer changes, or the number of iterative updates of the centroid vector reaches a preset number of iterations.

[0150] The technical solution of this embodiment can ensure that the inverted index of the vector database is automatically updated, and can solve the problem that the vector database consumes a lot of time due to frequent construction of the inverted index.

[0151] The flow chart provided by the present embodiment is not intended to indicate that the operation of the method will be performed in any particular order, or that all operations of the method are included in all every case. In addition, the method may include additional operations. Within the scope of the technical ideas provided by the present embodiment method, additional changes can be made to the above method.

[0152] It should be understood that in some embodiments, each part can be implemented by hardware, software, firmware or a combination thereof. In the above embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system.

[0153] This embodiment also provides a computer program product 10 , a computer readable storage medium 20 , and a computer device 30 . Figure 8 is a schematic diagram of a computer program product 10 according to an embodiment of the present invention, Fig. 9 is a schematic diagram of a computer-readable storage medium 20 according to an embodiment of the present invention. The computer program product 10 includes a computer program 11, which, when executed by a processor 32, implements the steps of any of the above-mentioned methods for constructing an inverted index of a vector database. The computer-readable storage medium 20 stores the above-mentioned computer program 11 thereon, which, when executed by a processor 32, implements the steps of any of the above-mentioned methods for constructing an inverted index of a vector database.

[0154] The computer program 11 for performing the operation of the present invention may be an assembly instruction, an instruction set architecture (ISA) instruction, a machine instruction, a machine-related instruction, a microcode, a firmware instruction, a state setting data, a configuration data of an integrated circuit, or a source code or an object code written in any combination of one or more programming languages ​​and process programming languages. The computer program 11 may be executed entirely on the user's computer, partially on the user's computer, as an independent software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the latter case, the remote computer may be connected to the user's computer via any type of network (including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., using an Internet service provider via the Internet). In some embodiments, in order to perform various aspects of the present invention, an electronic circuit including, for example, a programmable logic circuit, a field programmable gate array (FPGA) or a programmable logic array (PLA) may execute computer-readable program instructions by utilizing the state information of the computer-readable program instructions to personalize the electronic circuit.

[0155] In the description of this embodiment, the computer program product 10 is a related product including the computer program 11 .

[0156] For the purpose of the description of the present embodiment, the computer readable storage medium 20 is a tangible device capable of retaining and storing the computer program 11, which can be any device that can contain, store, communicate, propagate or use the computer program 11 for an instruction execution system, device or apparatus or in conjunction with these instruction execution systems, devices or apparatuses. More specific examples (a non-exhaustive list) of the computer readable storage medium 20 include the following: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, and any suitable combination of the above.

[0157] At this point, those skilled in the art should recognize that, although multiple exemplary embodiments of the present invention have been shown and described in detail herein, many other variations or modifications that conform to the principles of the present invention can still be directly determined or derived based on the content disclosed in the present invention without departing from the spirit and scope of the present invention. Therefore, the scope of the present invention should be understood and recognized as covering all these other variations or modifications.

Claims

1. A method for constructing an inverted index of a vector database, characterized in that: include: Constructing an inverted index, which at least includes: clustering each vector to obtain a set number of clusters; Determining whether a vector in the vector database has changed; If yes, then obtaining the vector change amount after the inverted index is constructed, and judging whether a preset index update condition is triggered according to the vector change amount; and When the index update condition is triggered, return to the step of building the inverted index.

2. The inverted index construction method according to claim 1, characterized in that: The step of judging whether a preset index update condition is triggered according to the vector change amount includes: Obtaining the total amount of vectors after constructing the inverted index, and calculating the ratio of the vector change to the total amount of vectors; Determining whether the ratio is greater than or equal to a set threshold; If so, it is determined that the index update condition is triggered.

3. The inverted index construction method according to claim 2, characterized in that: The vector database is provided with a vector update parameter and / or a total vector parameter, wherein the vector update parameter is used to record the vector change amount, and the total vector parameter is used to record the vector total amount; as well as After the step of constructing the inverted index, the method further includes: updating the total vector parameter, and / or resetting the vector update parameter; and / or After the step of determining whether the vector in the vector database has changed, the method further includes: if the vector in the vector database has changed, updating the vector update parameter.

4. The inverted index construction method according to claim 1, characterized in that: The step of clustering each vector to obtain a set number of clusters includes: Obtaining a set number of initial centroid vectors, respectively calculating the distance between each of the vectors and each of the initial centroid vectors, and assigning each of the vectors to an initial cluster corresponding to the initial centroid vector closest to the vector; The centroid vector of each of the initial clusters is obtained, the distance between each of the vectors and each of the centroid vectors is calculated respectively, and each of the vectors is assigned to the cluster corresponding to the centroid vector closest to the vector.

5. The inverted index construction method according to claim 4, characterized in that: The step of obtaining a set number of initial centroid vectors includes: Selecting a vector from each of the vectors and storing it in a centroid vector set; and When the number of vectors in the mass centroid vector set is less than the set number: Calculating the distance between each of the vectors and each of the vectors in the mass center vector set to obtain the minimum distance between each of the vectors and each of the vectors in the mass center vector set, and storing the vector with the largest minimum distance among the vectors into the mass center vector set; When the number of vectors in the mass center vector set is equal to the set number: The vector in the centroid vector set is used as the initial centroid vector.

6. The inverted index construction method according to claim 4, characterized in that: The step of calculating the centroid vector of each of the initial clusters comprises: The average vector of each vector in each of the initial clusters is calculated respectively, and each of the average vectors is used as the centroid vector of the corresponding initial cluster.

7. The inverted index construction method according to claim 1, characterized in that: After the step of constructing the inverted index of the vector database, the method further includes: Determine whether a vector insertion request is received; If yes, then obtaining a newly added vector and inserting the newly added vector into the vector database; and After the step of determining whether a preset index update condition is triggered according to the vector change amount, the method further includes: If the index update condition is not triggered, the cluster corresponding to the newly added vector is obtained, and the newly added vector is allocated to the corresponding cluster.

8. The inverted index construction method according to claim 1, characterized in that: After the step of constructing the inverted index, the method further includes: Determine whether an index update request instruction is received; If yes, return to the step of constructing the inverted index.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the inverted index construction method according to any one of claims 1 to 8 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the inverted index construction method according to any one of claims 1 to 8 are implemented.

Images