A vector encoding learning method and device for neighbor graph vector retrieval

Abstract

The application relates to a vector coding learning method and device for neighbor graph vector retrieval. After an original vector is acquired, a neighbor graph of the original vector is constructed; then, coding model parameters are initialized to complete preparation work; in a training process, the original vector is converted into transition coding through the coding model; the transition coding corresponds to replacing the original vector in the neighbor graph of the original vector to obtain a transition coding neighbor graph; through performing neighbor search on the transition coding neighbor graph, routing data is acquired and screened to adjust the coding model parameters, so that the coding model is adapted to the search characteristics of the neighbor graph; iterative training is performed until a training termination condition is met, and finally, compressed coding is output. In the vector coding learning process, a search process based on the neighbor graph is introduced, the retrieval performance of the compressed coding under the neighbor graph is directly optimized, and the search precision and efficiency can be improved while the memory overhead is reduced.

Description

Technical Field

[0001] This invention belongs to the field of vector retrieval, specifically relating to a vector encoding learning method and apparatus for nearest-neighbor graph vector retrieval. Background Technology

[0002] With the development of artificial intelligence, vector retrieval technology has been widely used in applications such as facial recognition, file search, speech recognition, and nucleic acid sequence retrieval. Among them, vector retrieval based on nearest neighbor graphs has become a research hotspot in academia and industry due to its excellent search and navigation capabilities.

[0003] However, with the rapid increase in data volume, the excessive memory and computational overhead poses a challenge to the widespread application of nearest-neighbor graph vector retrieval. To address this issue, one current solution is to reduce the vector dimensionality through vector encoding before performing nearest-neighbor graph vector retrieval. Existing vector encoding methods, such as LSH, PQ, OPQ, and Link&Code, employ value-preserving or order-preserving strategies to transform lossless original vectors into compressed codes. However, because these methods do not effectively integrate the encoding process with the nearest-neighbor graph vector retrieval task, they significantly reduce the accuracy and efficiency of nearest-neighbor graph vector retrieval. To address this problem, this invention proposes a vector encoding learning method and apparatus for nearest-neighbor graph vector retrieval. Summary of the Invention

[0004] This invention proposes a vector encoding learning method and apparatus for nearest-neighbor graph vector retrieval. By introducing a nearest-neighbor graph search process during training, it directly optimizes retrieval performance. Compared to existing encoding methods, it is better suited for nearest-neighbor graph vector retrieval tasks, improving search accuracy and efficiency. This invention can meet the vector retrieval requirements of related applications in large-scale scenarios, optimizing the user experience.

[0005] According to a first aspect of one or more embodiments of this specification, the specific content of the vector encoding learning method for nearest-neighbor graph vector retrieval proposed by the present invention is as follows:

[0006] (1) Obtain several original vectors;

[0007] (2) Construct an original vector nearest neighbor graph based on several original vectors. Each node in the original vector nearest neighbor graph corresponds to an original vector, and the edges represent the nearest neighbor relationship between the original vectors.

[0008] (3) Load the encoding model and initialize the encoding model parameters;

[0009] (4) Use the coding model to generate the transition code corresponding to the original vector, and use the transition code to replace the original vector in the nearest neighbor graph of the original vector, thereby obtaining the transition code nearest neighbor graph;

[0010] (5) Perform a nearest neighbor search on the transition coding nearest neighbor graph, record and filter the set of routing data to adjust the coding model parameters;

[0011] (6) Repeat steps (4) to (5) to output the final compressed code.

[0012] The transition coding nearest neighbor graph described in step (4) uses transition coding to replace the original vector in the original vector nearest neighbor graph. The nearest neighbor graphs of the transition coding nearest neighbor graph and the original vector nearest neighbor graph are the same. The difference between the original vector nearest neighbor graph and the transition coding nearest neighbor graph is that the corresponding data in the nearest neighbor graph nodes are different.

[0013] Preferably, the specific process for obtaining routing data in step (5) is as follows:

[0014] (5-1) Obtain the training vector, where the training vector may or may not be one of several original vectors;

[0015] (5-2) The training code corresponding to the training vector is generated by the coding model as the query point. The query point initializes the candidate node set according to the entry node of the transition coding nearest neighbor graph, calculates the predicted selection probability of each node in the candidate node set, and marks these nodes as unvisited. The maximum capacity of the candidate node set is fixed at L.

[0016] The predicted selection probability refers to the probability that, given the current encoding model parameters, a given query point will select one of the candidate nodes to visit during the search process based on the nearest neighbor graph.

[0017] (5-3) Select the unvisited state node v with the highest predicted selection probability from the candidate node set. j Using routing data ψ for group structure j Record the candidate node set and the unvisited state node v with the highest predicted selection probability. j ;

[0018] (5-4) Obtain node υ j In the nearest neighbor set of the transition coding nearest neighbor graph, calculate the predicted selection probability of each node in the nearest neighbor set corresponding to the transition coding, update the candidate node set H according to the predicted selection probability, and mark the node as visited.

[0019] (5-5) Repeat steps (5-3) and (5-4) until the true nearest neighbor v of the query point is visited. gt If there are no unvisited nodes in the candidate node set H, then the routing data described in step (3) constitutes a set Ψ of routing data, denoted as:

[0020] Ψ={ψ j |j=1,2,…,Z}

[0021] Z represents the number of nodes visited during the entire search process.

[0022] Step (5) filters the routing data in the set Ψ of routing data. Based on the phased characteristics of the nearest neighbor graph search process, routing data with guidance priority is ignored, and routing data with accuracy priority is selected to form the training routing dataset Ψ′ and train the encoding model M. ω The training routing dataset Ψ′ is represented as:

[0023] Ψ′=(ψ j |j=k,k+1,…,Z}

[0024] The encoding model M trained in steps (1)-(6) is applied. ω The original vector is input into the encoding model to obtain a compressed code that embeds the nearest neighbor graph search features.

[0025] A vector encoding learning method for nearest-neighbor graph vector retrieval, characterized in that the compression encoding includes at least one of the following: quantization encoding and hash encoding.

[0026] Preferably, in step (6), the condition for ending the repetition of steps (4) to (5) is that the number of training iterations exceeds a fixed threshold.

[0027] Preferably, in step (6), the condition for ending the repetition of steps (4) to (5) can also be: the loss calculated by the loss function of the encoding model is lower than the target value.

[0028] The present invention also provides a vector encoding apparatus for nearest-neighbor graph vector retrieval, comprising:

[0029] The original vector input module is used to obtain the original vector;

[0030] The nearest neighbor graph construction module is used to generate the nearest neighbor graph of the original vector corresponding to the original vector.

[0031] The training vector input module is used to obtain training vectors;

[0032] The encoding model training module is used to optimize the encoding model parameters to adapt to the search process of the nearest neighbor graph;

[0033] The compression encoding output module is used to output compressed encoding.

[0034] Compared with the prior art, the beneficial effects of this invention are:

[0035] The present invention provides a vector encoding learning method and apparatus for vector retrieval of nearest neighbor graphs. During the training process, the compressed encoding combined with the nearest neighbor graph directly optimizes the search performance, so that the compressed encoding applied to the nearest neighbor graph can improve search accuracy and search efficiency while reducing overhead. Attached Figure Description

[0036] Figure 1 This is a flowchart illustrating a vector encoding learning method for nearest-neighbor graph vector retrieval according to an embodiment of the present invention;

[0037] Figure 2 This is an example flowchart illustrating a vector encoding learning method for nearest neighbor graph vector retrieval according to an embodiment of the present invention;

[0038] Figure 3 This is a block diagram of a vector encoding device for nearest neighbor graph vector retrieval according to an embodiment of the present invention; Detailed Implementation

[0039] To make the technical solutions and advantages of the present invention clearer, the present invention will be further described below with reference to the accompanying drawings. Obviously, the following description is merely some embodiments of the present invention, and those skilled in the art can obtain other implementation methods without creative effort.

[0040] Figure 1 This is a flowchart of the vector encoding learning method of the present invention. To further illustrate the specific application and execution flow of the present invention, a flowchart of the vector encoding software and retrieval software including the vector encoding learning method is shown below. Figure 2 As shown, a vector encoding learning method for nearest-neighbor graph vector retrieval mainly includes the following steps:

[0041] (1) First, obtain the raw data. In this embodiment, the raw data is nucleic acid sequence data.

[0042] (2) This invention can be applied to any device system consisting of one or more devices that includes computing power and storage capacity. Since the databases (face images, nucleic acid sequences, files and voice, etc.) used by most applications such as face recognition, nucleic acid sequence retrieval, file search, and speech recognition are large in scale, they have certain requirements for the storage capacity of the device or device system. Therefore, this specification uses a device with appropriate storage capacity that matches the nucleic acid sequence dataset in this embodiment as an example. In addition, since this invention includes a model learning step, the device in this embodiment has a certain computing power. Finally, the device in this embodiment has the ability to receive the nucleic acid sequences exemplified in the embodiment.

[0043] (3) Obtain the original vectors, which are generated from the original data through a pre-model; for example, in this embodiment, the vector encoding software reads the nucleic acid sequence dataset, preprocesses the DNA sequences using k-mer, and vectorizes them using CountVectorizer, thereby generating the original nucleic acid sequence vector set, wherein the original nucleic acid sequence vector set S is abstractly represented as:

[0044]

[0045] Where, α i Let d be the original vector of the i-th nucleic acid sequence, and N be the number of original vectors of nucleic acid sequences in the set of original vectors of nucleic acid sequences. Each original vector of nucleic acid sequence is d. in dimension, This represents the data distribution space corresponding to the original vector of the nucleic acid sequence.

[0046] (4) Constructing a nearest neighbor graph of the original vectors based on the original vectors; The nearest neighbor graph constructed in the vector encoding software of this embodiment is an HNSW graph, and the HNSW graph and the original vector set S of the nucleic acid sequence are combined to form the original vector HNSW graph;

[0047] (5) Load the encoding model M ω The encoding model parameters are initialized. In this embodiment, the vector encoding software uses quantization encoding. Specifically, the encoding model included in the vector encoding software initializes the data distribution space where the original nucleic acid sequence vector is located. Mapped to P learning spaces The product of P, each learning space has a product of P. A codebook composed of codewords, M ω Abstract representation:

[0048] M ω = [M1, ..., M P ]

[0049] The e-th codeword from the p-th codebook is represented as c. pe The input is the original vector α of the nucleic acid sequences in the original nucleic acid sequence vector set S. i Output the nucleic acid sequence transition coding β in the nucleic acid sequence transition coding set S′. i Where S′ is abstractly represented as:

[0050]

[0051] Nucleic acid sequence transition coding β i It consists of P codewords, which can be abstractly represented as:

[0052] β i ={e1, ...,e P}

[0053] The data distribution space of the original vector set S of nucleic acid sequences and the data distribution space of the nucleic acid sequence transition coding set S′ The relationship is:

[0054]

[0055] (6) Using coding model M ω The transition codes for nucleic acid sequences corresponding to the original vectors of nucleic acid sequences are generated, and the transition code set S′ of the nucleic acid sequences is used to replace the original vector set S of the original vector HNSW graph, thus obtaining the transition code HNSW graph I′. It should be emphasized that the nearest neighbor graph of the transition code HNSW graph I′ is the same as that of the original vector HNSW graph I, the only difference being the data corresponding to the nodes.

[0056] (7) The query point performs a nearest neighbor search on the transition-coded nearest neighbor graph, records the routing data during the search process, and the specific process of obtaining the routing data is as follows:

[0057] (7-1) The vector encoding software reads the nucleic acid sequence training data and vectorizes the nucleic acid sequence training data into nucleic acid sequence training vectors q. The encoding model generates the nucleic acid sequence training code q′ corresponding to the nucleic acid sequence training vector as a simulated query point. A pre-defined entry node of the HNSW graph is added to the min-heap H, where the maximum capacity of the min-heap H is fixed at L.

[0058] (7-2) Using routing data ψ for group structures j Record the state of the min-heap H, and select the unvisited node v with the highest predicted selection probability from the min-heap H. j Also stored in routing data ψ j In this context, the predicted selection probability represents the probability that, given the current encoding model parameters, a given query point will select one node from the candidate node set for access during a search based on a nearest neighbor graph. The calculation process for the predicted selection probability in this embodiment is as follows:

[0059]

[0060]

[0061] (7-3) Obtain node v j The nearest neighbor node N(v) j ), and calculate N(v) at the simulated query point. j The predicted selection probability corresponding to the transition coding is used to update the min-heap H;

[0062] (7-4) Repeat steps (7-3) and (7-4) until the nearest neighbor v is visited.gt Or, there are no more accessible nodes in the min-heap H. In this case, all the routing data described in step (7-3) constitutes the set Ψ of routing data, which can be abstractly represented as:

[0063] Ψ={ψ j |j=1,2,…,Z}

[0064] Where Z is the number of nodes accessed in the entire search process;

[0065] (8) Select a portion of precision-priority routing data to train the coding model. The routing data in the set Ψ is filtered, and based on the phased characteristics of the nearest neighbor graph search process, the routing data prioritizing guidance is ignored, and the routing data prioritizing precision is selected to form the training routing dataset Ψ′. The phased nature of the nearest neighbor graph search process refers to: based on its contribution to search performance, the search process of nearest neighbor graph vector retrieval can be divided into two phases. The initial phase has little impact on search precision but a significant impact on search efficiency, mainly playing a guiding role; the nodes visited in the final phase have the opposite effect, prioritizing search precision. In this embodiment, the first... The routing data from item Z to item Z is used as the training routing dataset Ψ′, which is abstractly represented as follows:

[0066]

[0067] (9) Training the coding model M based on the training routing dataset Ψ′ ω For each route data ψ in the training route dataset j Obtain all nodes in the min-heap H, and recalculate the nucleic acid sequence training vector q corresponding to the simulated query point and the original nucleic acid sequence vector α corresponding to these nodes. i The actual distance dist(q, α) i Record the node with the smallest actual distance, where the actual distance is dist(q, α). i Abstract representation:

[0068]

[0069] (10) Input the training routing dataset Ψ′ and the corresponding set of nodes with the smallest actual distance into the encoding model M. ω Optimize the coding model, coding model M ω The loss function is:

[0070]

[0071] (11) Repeat steps (6) to (10) until the loss calculated by the loss function is less than Ω or the number of training iterations is greater than or equal to Θ, and obtain the final nucleic acid sequence transition coding set as the nucleic acid sequence compressed coding set, as well as the codebook in the HNSW diagram and coding model;

[0072] (12) Before online querying, the retrieval software loads the codebook and combines the HNSW diagram and the nucleic acid sequence compressed coding set to form a compressed coding HNSW diagram;

[0073] (13) When querying online, the retrieval software first obtains nucleic acid sequence query data; then it vectorizes the nucleic acid sequence query data into nucleic acid sequence query vectors and performs nearest neighbor search on the compressed HNSW graph to return a certain number of results r.

[0074] (14) The above method fully considers the search process of the nearest neighbor graph during the model training process, directly optimizes the retrieval performance, thereby improving the search accuracy and search efficiency of the nearest neighbor graph while reducing the overhead.

[0075] This specification illustrates a vector encoding device such as... Figure 3 As shown, the device takes an original vector as input and outputs a corresponding vector code; the device includes:

[0076] The original vector input module is used to obtain the original vector;

[0077] The nearest neighbor graph construction module is used to generate the nearest neighbor graph of the original vector corresponding to the original vector.

[0078] The training vector input module is used to obtain training vectors;

[0079] The encoding model training module is used to optimize the encoding model parameters to adapt to the search process of the nearest neighbor graph;

[0080] The compression encoding output module is used to output compressed encoding.

[0081] It is understood that since the vector encoding device basically corresponds to the vector encoding method embodiment, the vector encoding device embodiment can be found in the description of the vector encoding method embodiment. It should be noted that the vector encoding device described above is illustrative, and the multiple modules may or may not be physically separate. Those skilled in the art can understand and implement this without any creative effort.

[0082] It should be noted that the above is only a detailed description of the preferred embodiments and principles of the present invention. For those skilled in the art, there may be changes in the specific implementation based on the ideas provided by the present invention, and these changes should also be considered within the scope of protection of the present invention.

Claims

1. A vector encoding learning method for nearest-neighbor graph vector retrieval, characterized in that... Includes the following steps: (1) Obtain several original vectors; (2) Construct an original vector nearest neighbor graph based on several original vectors. Each node in the original vector nearest neighbor graph corresponds to an original vector, and the edges represent the nearest neighbor relationship between the original vectors. (3) Load the encoding model and initialize the encoding model parameters; (4) Use the coding model to generate the transition code corresponding to the original vector, and use the transition code to replace the original vector in the nearest neighbor graph of the original vector to obtain the transition code nearest neighbor graph; (5) Perform a nearest neighbor search on the transition coding nearest neighbor graph, obtain and filter the set of routing data, and adjust the coding model parameters; (6) Repeat steps (4) to (5) to output the final compressed code; In step (5), the specific process of obtaining the set of routing data is as follows: (5-1) Obtain training vectors, wherein the training vectors may be one of the plurality of original vectors; (5-2) The training code corresponding to the training vector generated by the coding model is used as the query point. The query point initializes the candidate node set according to the entry node of the transition coding nearest neighbor graph, calculates the predicted selection probability of each node in the candidate node set, and marks these nodes as unvisited. The maximum capacity of the candidate node set is fixed at [value missing]. ; The predicted selection probability refers to the probability that, given the current encoding model parameters, a given query point will select one of the candidate nodes to visit during the search process based on the nearest neighbor graph. (5-3) Select the unvisited node with the highest predicted selection probability from the candidate node set. Using routing data for group structures Record the candidate node set and the unvisited state node with the highest predicted selection probability. ; (5-4) Obtaining Nodes In the nearest neighbor set of the transition coding nearest neighbor graph, calculate the predicted selection probability of the transition coding for each node in the nearest neighbor set, and update the candidate node set according to the predicted selection probability. The node is marked as visited; (5-5) Repeat steps (5-3) and (5-4) until the nearest neighbor node of the query point is visited. or candidate node set There are no longer any unvisited nodes in the list. The entire set of routing data recorded during the search process constitutes the routing data set. , represented as: ； in, This represents the number of nodes visited during the entire search process. In step (5), the set of routing data The specific method for filtering routing data includes the following steps: Based on the phased characteristics of the nearest neighbor graph search process, route data with guidance priority is ignored, and route data with accuracy priority is selected to form the training route dataset. Training the encoding model The training routing dataset Represented as: 。 2. The vector encoding learning method for nearest-neighbor graph vector retrieval as described in claim 1, characterized in that, The compression encoding includes at least one of the following: quantization encoding and hash encoding.

3. The vector encoding learning method for nearest-neighbor graph vector retrieval as described in claim 1, characterized in that, In step (6), the condition for ending the repetition of steps (4) to (5) is that the number of training iterations exceeds a fixed threshold.

4. The vector encoding learning method for nearest-neighbor graph vector retrieval as described in claim 1, characterized in that, In step (6), the condition for ending the repetition of steps (4) to (5) is that the loss calculated by the loss function of the encoding model is lower than the target value.

5. A vector encoding apparatus for nearest-neighbor graph vector retrieval, characterized in that, For performing the vector encoding learning method as described in claim 1, including: The original vector input module is used to obtain the original vector; The nearest neighbor graph construction module is used to generate the nearest neighbor graph of the original vector corresponding to the original vector. The training vector input module is used to obtain training vectors; The encoding model training module is used to perform step (5); The compression encoding output module is used to output compressed encoding.

Images