Large model vector similarity search optimization method and related equipment

Abstract

The invention discloses a large model vector similarity search optimization method and related equipment, and relates to the technical field of large language models.The method comprises the steps that a query vector is obtained; determining a first candidate vector set based on the query vector and a preset primary index; performing approximate distance calculation on the first candidate vector set based on the query vector and secondary index data of each candidate vector in the first candidate vector set to obtain an approximate distance calculation result; based on the approximate distance calculation result, filtering the first candidate vector set to determine a second candidate vector set; performing accurate distance calculation on each candidate vector in the second candidate vector set to obtain an accurate distance calculation result; and determining a target retrieval result based on the accurate distance calculation result.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of large language model, in particular to a large model vector similarity search optimization method and related equipment. BACKGROUND

[0002] With the rapid development of artificial intelligence technology, especially the wide application of deep learning and large language model, unstructured data such as text and image are often represented as high-dimensional vectors, and vector similarity search is used to realize functions such as semantic retrieval, personalized recommendation, and intelligent question answering. Such search usually relies on approximate nearest neighbor (ANN) algorithm, which first uses a primary index structure to quickly recall a candidate set, and then completes the precise sorting through precise distance calculation. However, as the data size continues to expand and real-time requirements continue to improve, the traditional method needs to calculate the high-dimensional vector distance of a large number of candidates in the precise sorting stage, which brings significant computational overhead and delay, and has become the main bottleneck of system performance. Therefore, there is an urgent need for a large model vector similarity search optimization method to solve the above problems. SUMMARY

[0003] A series of simplified concepts are introduced in the summary section, which will be further described in detail in the detailed description section. The summary section of the present application does not mean to attempt to limit the key features and necessary technical features of the claimed technical solution, nor does it attempt to determine the protection scope of the claimed technical solution.

[0004] In a first aspect, the present application provides a large model vector similarity search optimization method, comprising:

[0005] obtaining a query vector;

[0006] determining a first candidate vector set based on the query vector and a preset primary index;

[0007] performing approximate distance calculation on the first candidate vector set based on the query vector and the secondary index data of each candidate vector in the first candidate vector set, to obtain an approximate distance calculation result;

[0008] filtering the first candidate vector set based on the approximate distance calculation result to determine a second candidate vector set;

[0009] performing precise distance calculation on each candidate vector in the second candidate vector set to obtain a precise distance calculation result;

[0010] determining a target retrieval result based on the precise distance calculation result.

[0011] In some embodiments, the determining a first candidate vector set based on the query vector and a preset primary index comprises:

[0012] determine a corresponding retrieval strategy based on the index type of the preset first-level index;

[0013] perform approximate nearest neighbor retrieval from the vector library based on the retrieval strategy and the query vector;

[0014] determine the first candidate vector set based on a return result of the approximate nearest neighbor retrieval.

[0015] In some embodiments, the approximate distance calculation on the first candidate vector set based on the query vector and the second-level index data of each candidate vector in the first candidate vector set to obtain an approximate distance calculation result comprises:

[0016] loading the second-level index data corresponding to each candidate vector from the second-level index storage module based on the identifier of each candidate vector in the first candidate vector set;

[0017] calculating the approximate distance between the query vector and each candidate vector based on the second-level index data of the query vector and each candidate vector, respectively;

[0018] generating the approximate distance calculation result based on the approximate distance of each candidate vector.

[0019] In some embodiments, the filtering on the first candidate vector set based on the approximate distance calculation result to determine a second candidate vector set comprises:

[0020] performing filtering judgment on each candidate vector in the first candidate vector set based on a preset filtering threshold and the approximate distance calculation result to determine the retention state of each candidate vector;

[0021] selecting the candidate vector to be retained from the first candidate vector set based on the retention state of each candidate vector;

[0022] determining the second candidate vector set based on the selected candidate vector to be retained.

[0023] In some embodiments, the exact distance calculation on each candidate vector in the second candidate vector set to obtain an exact distance calculation result comprises:

[0024] loading the original high-dimensional vector data of each candidate vector from the vector storage module based on the identifier of each candidate vector in the second candidate vector set;

[0025] calculating the exact distance between the query vector and each candidate vector based on the original high-dimensional vector data of the query vector and each candidate vector, respectively;

[0026] The precise distance calculation result is generated based on the precise distance of each candidate vector.

[0027] In some implementations, determining the target retrieval result based on the precise distance calculation result includes:

[0028] Based on the precise distance calculation results, the candidate vectors in the second candidate vector set are sorted.

[0029] The target retrieval result is determined based on the sorting results and a preset quantity threshold.

[0030] In some implementations, before determining the first candidate vector set based on the query vector and a preset first-level index, the method further includes:

[0031] Based on each original high-dimensional vector in the vector library, pre-calculate the secondary index data corresponding to each original high-dimensional vector;

[0032] Based on the identifiers of each original high-dimensional vector, the secondary index data is associated with and stored with the corresponding original high-dimensional vector;

[0033] In response to the update operation of the vector library, the secondary index data is dynamically updated based on the added and / or deleted vector data.

[0034] Secondly, this application provides an optimization apparatus for large-scale model vector similarity search, comprising:

[0035] The query vector acquisition unit is used to acquire the query vector.

[0036] The first vector determination unit is used to determine a first candidate vector set based on the query vector and a preset first-level index;

[0037] An approximate distance calculation unit is used to perform approximate distance calculation on the first candidate vector set based on the query vector and the secondary index data of each candidate vector in the first candidate vector set, and obtain the approximate distance calculation result;

[0038] The second vector determination unit is used to filter the first candidate vector set based on the approximate distance calculation result to determine the second candidate vector set;

[0039] The precise distance calculation unit is used to perform precise distance calculation on each candidate vector in the second candidate vector set to obtain the precise distance calculation result.

[0040] The target retrieval and determination unit is used to determine the target retrieval result based on the accurate distance calculation result.

[0041] Thirdly, an electronic device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program stored in the memory to implement the steps of the optimization method for large model vector similarity search of any of the first aspects.

[0042] Fourthly, this application also proposes a computer-readable storage medium storing a computer program thereon, wherein when the computer program is executed by a processor, it implements the steps of the optimization method for large model vector similarity search of any one of the first aspects.

[0043] In summary, the optimization method for large-scale vector similarity search proposed in this application reduces the number of candidates requiring precise distance calculation by introducing a secondary index structure to perform approximate distance calculation and filtering on the candidate set recalled by the primary index. This effectively reduces computational load and memory access pressure, and improves system throughput and response speed. This application achieves efficient search without sacrificing retrieval accuracy, and is particularly suitable for latency-sensitive large-scale vector retrieval scenarios, exhibiting good versatility and integrability. Attached Figure Description

[0044] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit this specification. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings:

[0045] Figure 1 A schematic flowchart of an optimization method for large model vector similarity search provided in an embodiment of this application;

[0046] Figure 2 A schematic diagram of an optimization device for large model vector similarity search provided in an embodiment of this application;

[0047] Figure 3 This is a schematic diagram of an optimized electronic device structure for large model vector similarity search provided in an embodiment of this application. Detailed Implementation

[0048] The terms "first," "second," "third," "fourth," etc. (if present) in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments described herein can be implemented in a sequence other than that illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus. The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them.

[0049] Please see Figure 1 This is a flowchart illustrating an optimization method for large model vector similarity search provided in an embodiment of this application, which may specifically include:

[0050] S110, Obtain the query vector;

[0051] For example, in step S110, a natural language query request is received from user input or the upper-layer system. This request is usually in natural language form and is converted into a high-dimensional vector representation using a pre-trained embedding model to obtain a query vector. This query vector serves as the input to the entire retrieval process, and its quality directly affects the accuracy of subsequent index retrieval and similarity calculation.

[0052] S120. Based on the query vector and the preset first-level index, determine the first candidate vector set;

[0053] For example, in step S120, a pre-built first-level index structure is used to perform a fast approximate nearest neighbor retrieval of the query vector. This index structure can adopt various general index types based on graphs, clustering, hashing, or trees. Through an efficient retrieval strategy, a batch of candidate vectors most relevant to the query vector are quickly recalled from a large-scale vector library to form a preliminary first candidate vector set, providing a candidate basis for the subsequent fine ranking stage. This process reduces the complexity of the initial retrieval stage while ensuring a high recall rate.

[0054] S130. Based on the secondary index data of the query vector and each candidate vector in the first candidate vector set, perform approximate distance calculation on the first candidate vector set to obtain the approximate distance calculation result;

[0055] For example, in step S130, the approximate distance between the query vector and each candidate vector is quickly calculated in a low-dimensional or compressed feature space by using lightweight secondary index data pre-constructed for each candidate vector. This process avoids the computational overhead caused by directly manipulating the original high-dimensional vectors. It uses methods such as dimensionality reduction projection, hash signature, or distance boundary estimation to achieve a preliminary assessment of the similarity of the candidate set with low computational cost, providing a quantitative basis for subsequent filtering.

[0056] S140. Based on the approximate distance calculation results, filter the first candidate vector set to determine the second candidate vector set;

[0057] For example, in step S140, the approximate distance calculation results of candidate vectors are compared and judged according to a preset filtering threshold, and candidate vectors with smaller distances and higher correlations are selected to form a second candidate vector set with a significantly reduced size. This effectively reduces the computational load and processing time of the subsequent accurate distance calculation stage while retaining highly correlated candidates as much as possible.

[0058] S150. Perform precise distance calculation on each candidate vector in the second candidate vector set to obtain the precise distance calculation result;

[0059] For example, in step S150, the original high-dimensional vector data corresponding to each vector in the second candidate vector set is loaded from the underlying storage according to the identifier of each vector. Then, the query vector and each original vector are calculated accurately one by one. This calculation process can use various measurement methods such as Euclidean distance, cosine similarity or inner product according to actual application needs, and improves the computational efficiency through hardware instruction-level optimization, so as to finally obtain an accurate distance result that can truly reflect the semantic correlation between vectors.

[0060] S160. Based on the accurate distance calculation results, determine the target retrieval results.

[0061] For example, in step S160, the candidate set after screening is sorted and selected based on the accurately calculated distance or similarity score between vectors, and the target result most relevant to the query vector is determined and returned, thereby completing the entire retrieval process and ensuring that the output results meet the actual application requirements in terms of accuracy and relevance.

[0062] In summary, this application's embodiments introduce a secondary index structure to perform approximate distance calculation and effective filtering on the candidate vector set recalled by the primary index. This reduces the number of candidates requiring precise distance calculation in the subsequent stages, thereby significantly reducing the computational overhead and memory access pressure in the high-dimensional vector similarity search process while maintaining basic retrieval accuracy. This application's embodiments effectively improve the overall system throughput and response speed, making them suitable for large-scale vector database retrieval, high-concurrency recommendation systems, and applications with strict real-time requirements such as Retrieval Augmentation (RAG). They also possess good versatility, scalability, and ease of integration.

[0063] In some instances, the first candidate vector set is determined based on the query vector and a preset first-level index, including:

[0064] Based on the index type of the preset first-level index, determine the corresponding retrieval strategy;

[0065] Based on the retrieval strategy and query vector, an approximate nearest neighbor retrieval is performed from the vector database;

[0066] Based on the results of the approximate nearest neighbor retrieval, the first candidate vector set is determined.

[0067] For example, the corresponding retrieval strategy is determined based on the specific type of the preset primary index. This primary index can be a graph-based HNSW index, a clustering-based IVF index, a hash-based LSH index, or a tree-based KD-Tree or Annoy index, among other common types. Different types of indexes have their own applicable retrieval algorithms and parameter configurations. The process of determining the retrieval strategy includes selecting appropriate search parameters based on the characteristics of the index structure. For example, for an HNSW index, the efSearch parameter needs to be set to control the balance between search scope and precision; for an IVF index, the number of nearest clusters to be searched, nprobe, needs to be determined to achieve a balance between recall and retrieval efficiency.

[0068] Based on the determined retrieval strategy and query vector, an approximate nearest neighbor retrieval is performed from the vector database. This process utilizes the data organization and fast retrieval capabilities established by the first-level index structure to efficiently find candidate vectors most similar to the query vector in the vector database. For example, when using the HNSW index, a greedy search is performed through a hierarchical navigation graph to quickly approximate the nearest neighbor. If the IVF index is used, the nearest clusters are located first, and then a detailed search is performed within them. This reduces the complexity of recalling potentially relevant candidates from massive vectors, forming preliminary approximate nearest neighbor results.

[0069] Based on the results returned by the approximate nearest neighbor retrieval, a first candidate vector set is determined. This set typically includes identifiers for a series of candidate vectors and their preliminary similarity scores or distances to the query vector. The required number of candidates, M, is preset or dynamically calculated according to the actual application scenario. Based on this, the Top-M candidate vectors are selected from the returned results, or all candidates whose similarity exceeds a certain threshold, forming the first candidate vector set. This sets the data foundation for subsequent secondary index filtering and ranking stages, while ensuring a high recall rate during the recall stage to avoid missing relevant results. It should be noted that M is a preset value, representing the size of the first candidate vector set, such as between 100 and 10000.

[0070] In summary, the embodiments of this application determine the first candidate vector set through the above method, which takes advantage of the first-level index in fast approximate retrieval. It can recall a large number of potentially relevant candidates from a large-scale vector library in a very short time, providing a rich and high-quality candidate foundation for subsequent processing stages. This step effectively controls the complexity of the initial retrieval while ensuring a high recall rate.

[0071] In some instances, based on the secondary index data of the query vector and each candidate vector in the first candidate vector set, an approximate distance calculation is performed on the first candidate vector set to obtain the approximate distance calculation result, including:

[0072] Based on the identifiers of each candidate vector in the first candidate vector set, the secondary index data corresponding to each candidate vector is loaded from the secondary index storage module;

[0073] Based on the secondary index data of the query vector and each candidate vector, the approximate distance between the query vector and each candidate vector is calculated respectively.

[0074] Based on the approximate distance of each candidate vector, the approximate distance calculation result is generated.

[0075] For example, based on the identifier of each candidate vector in the first candidate vector set, the secondary index data corresponding to each candidate vector is loaded from the secondary index storage module. The secondary index data is a lightweight feature representation extracted from the original high-dimensional vector in advance through methods such as principal component analysis, random projection, SimHash hashing, or cluster boundary calculation. For example, low-dimensional projection vectors, binary hash signatures, distance upper bounds, or block quantization codes are stored in memory or high-speed storage devices in the form of key-value pairs associated with the vector identifier to ensure fast loading and access within milliseconds.

[0076] Based on the secondary index data of the query vector and each candidate vector, approximate distances between the query vector and each candidate vector are calculated. During this process, the query vector itself undergoes appropriate transformations depending on the type of secondary index used, such as dimensionality reduction using the same projection matrix or signature generation using the same hash function. Subsequently, based on different types of secondary index data, corresponding lightweight distance metrics are used for calculation. For example, if the secondary index is a low-dimensional projection (LPI), the Euclidean distance between the projections of the query vector and the candidate vectors is calculated; if it is a hash signature (HSI), the Hamming distance between the two signatures is calculated; if it is a distance upper bound cache (UBC), the distance upper bound is estimated based on the distance from the query vector to the cluster center and the pre-stored maximum offset. This calculation process is performed entirely in a low-dimensional or compressed space, avoiding direct operations on high-dimensional vectors, and its single computational cost is far lower than the precise distance calculation in the original high-dimensional space.

[0077] Based on the approximate distances of each candidate vector, approximate distance calculation results are generated. After calculating the approximate distances of all candidate vectors, a list or array containing the identifier of each candidate vector and its corresponding approximate distance value is obtained, which constitutes the approximate distance calculation result. This result quantifies the correlation or distance between the query vector and each candidate vector in the approximate feature space. The smaller the distance value, the higher the similarity, providing a numerical basis for the next filtering judgment.

[0078] In summary, the embodiments of this application, through the above steps, utilize pre-computed lightweight secondary index data to complete a fast and low-cost preliminary similarity assessment before performing the costly precise distance calculation. This reduces the size of the candidate set requiring subsequent precise calculations from a large M to a smaller N (N is much smaller than M), thereby reducing the computational overhead and memory bandwidth pressure of the most computationally demanding and time-consuming fine-ranking stage in the entire retrieval process. This lays a crucial foundation for improving overall throughput and response speed.

[0079] In some instances, based on the approximate distance calculation results, the first candidate vector set is filtered to determine the second candidate vector set, including:

[0080] Based on the preset filtering threshold and the approximate distance calculation results, the candidate vectors in the first candidate vector set are filtered and judged to determine the retention status of each candidate vector.

[0081] Based on the retention status of each candidate vector, select the candidate vectors to be retained from the first candidate vector set;

[0082] Based on the selected candidate vectors that need to be retained, a second set of candidate vectors is determined.

[0083] For example, based on a preset filtering threshold and the approximate distance calculation result, each candidate vector in the first candidate vector set is filtered to determine its retention status. Specifically, this process involves comparing the approximate distance value corresponding to each candidate vector with the preset filtering threshold. If the approximate distance of a candidate vector is less than or equal to the threshold, it is determined that the candidate vector has a high correlation with the query vector in the approximate feature space, and its retention status is marked as needing to be retained. Conversely, if its approximate distance is greater than the threshold, it is determined that its correlation is low, and it is marked as needing to be filtered. The preset filtering threshold can be set according to the accuracy and efficiency requirements of the actual business scenario. It can be a fixed empirical value or dynamically calculated based on the current query or data distribution. For example, in a financial policy retrieval scenario, a more lenient threshold is set to ensure compliance and a higher recall rate is required, while a stricter threshold can be set in scenarios such as product recommendations where extremely high accuracy is required.

[0084] Based on the retention status of each candidate vector, candidate vectors to be retained are selected from the first candidate vector set. This process iterates through the retention status markers of all candidate vectors in the first candidate vector set, and gathers all candidate vectors marked as needing to be retained according to their original identifier order or approximate distance, forming an intermediate candidate list smaller than the original first candidate vector set. The candidate vectors in this list are considered to be candidates that still have high potential to enter the final Top-K results after secondary index approximation evaluation, thus completing the transition from coarse selection to fine selection.

[0085] Based on the selected candidate vectors that need to be retained, a second candidate vector set is determined. This step defines the intermediate set obtained from the above filtering operation as the second candidate vector set. This second candidate vector set is a high-quality subset of the original first candidate vector set, and its size N is usually much smaller than the initial recall size M of the first candidate set. However, it includes candidate vectors that are considered most likely to become the final Top-K results after approximate distance evaluation, thereby reducing the load of subsequent calculation stages while retaining relevant results to the maximum extent. It should be noted that N is the size of the second candidate vector set, and K is the number of target retrieval results.

[0086] In summary, the embodiments of this application, through the aforementioned filtering mechanism, add an approximate evaluation and screening step based on a lightweight secondary index to the retrieval process. This effectively eliminates a large number of obviously irrelevant candidates, avoiding the need to perform high-dimensional precise distance calculations. While maintaining overall recall, this process reduces the number of computations and memory accesses required in the fine-ranking stage, thus becoming an optimization step that reduces overall system latency and improves throughput. It is particularly suitable for application scenarios with large candidate sets or high costs associated with precise distance calculations.

[0087] In some instances, precise distance calculations are performed on each candidate vector in the second candidate vector set to obtain precise distance calculation results, including:

[0088] Based on the identifiers of each candidate vector in the second candidate vector set, the original high-dimensional vector data of each candidate vector is loaded from the vector storage module;

[0089] Based on the original high-dimensional vector data of the query vector and each candidate vector, calculate the precise distance between the query vector and each candidate vector respectively;

[0090] Based on the precise distance between each candidate vector, the precise distance calculation result is generated.

[0091] For example, based on the identifiers of each candidate vector in the second candidate vector set, the original high-dimensional vector data of each candidate vector is loaded from the vector storage module. This vector storage module is used to persistently store the original, uncompressed or undimension-reduced high-dimensional vector data, and is typically implemented using distributed storage or a high-performance local storage system to ensure efficient access to large-scale vector data. The loading process accurately reads the corresponding complete vector data from the storage module through a data query interface (such as direct database key-value query) based on the unique identifier of each vector in the second candidate vector set.

[0092] Based on the original high-dimensional vector data of the query vector and each candidate vector, the precise distance between the query vector and each candidate vector is calculated. This calculation is performed in the original high-dimensional space, using a specific similarity metric required by the business scenario. For example, cosine similarity is often used in semantic matching tasks that need to measure directional similarity, while Euclidean distance or inner product operations may be used in retrieval tasks that need to measure absolute distance. To improve computational efficiency, this process is accelerated using the single instruction multiple data (SIMD) instruction set of modern processors (such as Intel AVX2 or ARM Neon) to achieve parallel processing of floating-point operations. For each pair of query vectors and candidate vectors, a complete distance or similarity calculation is performed independently, ensuring that the accuracy of the calculation results is not affected by other candidate or approximate calculations. The accuracy of the calculation results directly determines the quality of the final retrieval results.

[0093] Based on the precise distances of each candidate vector, precise distance calculation results are generated. After calculating the precise distances of all candidate vectors, the identifier of each candidate vector is associated with its corresponding precise distance value to form a structured list or array of distance results, i.e., the precise distance calculation results. This result set accurately records the true distances or similarity scores between the filtered candidate vectors and the query vector in high-dimensional space. Its data format is usually designed to facilitate subsequent sorting operations, such as a list of tuples containing vector IDs and distance values, providing direct and reliable data input for determining the final target retrieval results.

[0094] In summary, the embodiments of this application, through the aforementioned precise distance calculation process, perform a final refined evaluation of the high-quality candidate set after secondary index filtering, ensuring the accuracy and reliability of the output results. This process benefits from an effective filtering mechanism in the early stages, concentrating the computational load on a small number of high-potential candidates, thereby significantly reducing overall computational overhead and response time while maintaining high accuracy. It is particularly suitable for real-time retrieval scenarios with large-scale, high-dimensional vector libraries, effectively improving the overall performance and practicality of the system.

[0095] In some instances, target retrieval results are determined based on precise distance calculations, including:

[0096] Based on the accurate distance calculation results, the candidate vectors in the second candidate vector set are sorted.

[0097] Based on the ranking results and a preset quantity threshold, the target retrieval results are determined.

[0098] For example, based on the precise distance calculation results, the candidate vectors in the second candidate vector set are sorted. This process uses an efficient sorting algorithm (such as quickselect, heapsort, or mergesort) to sequentially arrange the entire candidate set according to the precise distance value or similarity score between each candidate vector and the query vector recorded in the precise distance calculation results. If the distance value is an indicator representing difference such as Euclidean distance or Manhattan distance, it is sorted in ascending order, with smaller values ​​indicating greater similarity to the query vector; if it is an indicator representing similarity such as cosine similarity or inner product, it is sorted in descending order, with larger values ​​indicating higher relevance. The sorting algorithm can be chosen based on the size of the candidate set. For example, quicksort can be used for smaller sets, while heapsort or quickselect can be used for larger sets to quickly determine the Top-K candidates, ensuring that the sorting process itself does not introduce significant performance overhead.

[0099] Based on the ranking results and a preset threshold, the target retrieval results are determined. The preset threshold K represents the number of Top-K most similar results that the system ultimately needs to return. Its value is set according to specific application requirements, usually between 5 and 20. For example, in the RAG system, K=5 is often used when generating answers for large language models. This step selects the top K candidate vectors from the ranked candidate list to form the final target retrieval results; if the number of ranked candidates is less than K, all candidates are returned. The target retrieval results are usually output in structured data form (such as JSON objects), containing the identifiers of the Top-K vectors and their corresponding precise distances or similarity scores. If necessary, it may also include reference information pointing to the original data (such as text fragments, image metadata) for use by subsequent processing modules (such as the Reranker re-ranking model or LLM generation module).

[0100] In summary, the embodiments of this application perform final ranking and selection of candidates based on the true relevance calculated with precise distance, ensuring high accuracy and reliability of the returned results. Its effectiveness is based on the high-quality candidate subset provided by the secondary index filtering in the early stage. As the final step in the retrieval process, this process determines the quality of the output results and is suitable for various application scenarios that require high-precision retrieval results.

[0101] In some instances, before determining the first candidate vector set based on the query vector and a preset first-level index, the following steps are also included:

[0102] Based on each original high-dimensional vector in the vector library, pre-calculate the secondary index data corresponding to each original high-dimensional vector;

[0103] Based on the identifiers of each original high-dimensional vector, the secondary index data is associated with and stored with the corresponding original high-dimensional vector;

[0104] In response to updates to the vector library, the secondary index data is dynamically updated based on newly added and / or deleted vector data.

[0105] For example, for each original high-dimensional vector stored in the vector library, a feature extraction or compression algorithm is used to generate its corresponding lightweight secondary index data. The secondary index data may include, but is not limited to, a Low-Dimensional Projected Index (LPI), which is a low-dimensional vector representation obtained by reducing the original high-dimensional vector to a lower dimension (e.g., 16 to 64 dimensions) using techniques such as Principal Component Analysis (PCA), random projection, or autoencoders; a Hash Signature Index (HSI), which generates a fixed-length binary signature (e.g., 64-bit or 128-bit) using SimHash, Locality Sensitive Hashing (LSH), or learned hashing methods; a Distance Upper Bound Cache Index (UBC), which pre-calculates the maximum offset distance based on the cluster center to which the vector belongs; or a Blocked Bit Compression Index (BCI), which divides the vector into blocks by dimension and performs scalarization processing. This pre-calculation process is typically performed offline in batches during data entry or system idle periods to ensure that the pre-generated index data can be directly used during online retrieval, avoiding the latency caused by real-time computation.

[0106] Based on the identifiers of each original high-dimensional vector, the secondary index data is associated with and stored in relation to the corresponding original high-dimensional vector. This step aims to establish and maintain the mapping relationship between the original vector and its secondary index data. A unique identifier (such as a numerical ID or hash value) is assigned to each original high-dimensional vector, and this identifier is used as the key, with its corresponding secondary index data as the value. These key-value pairs are persistently stored in a dedicated secondary index storage module. This storage module can be implemented based on an in-memory database (such as Redis), a high-performance cache, or an optimized disk storage structure to ensure data read speeds at the millisecond or even microsecond level. The associated storage design ensures that during subsequent retrieval processes, the corresponding secondary index data can be quickly located and loaded based on the identifier of the candidate vector, providing data support for approximate distance calculations.

[0107] In response to updates to the vector library, the secondary index data is dynamically updated based on newly added and / or deleted vector data. This step ensures data consistency and real-time performance between the secondary index and the original vector library. The specific process is as follows: when a data change is detected in the vector library (such as inserting a new vector or deleting an existing vector), the incremental update mechanism of the secondary index is automatically triggered. For newly added vectors, their corresponding secondary index data is calculated immediately or according to a predetermined strategy (such as batch processing after accumulating a certain number), and stored in the secondary index storage module using the identifier as the key. For deleted vectors, all secondary index data corresponding to their identifier are removed from the storage module. This dynamic update mechanism supports online operation without downtime or rebuilding the entire index. For example, after adding a regulatory policy to the knowledge base of a financial customer service system, its vectorized representation and corresponding SimHash signature can be calculated and take effect within one second, meeting the high real-time requirements of enterprise applications.

[0108] In summary, the secondary index pre-computation, associated storage, and dynamic update mechanism implemented through the above steps in this application provides an efficient, consistent, and sustainably maintainable auxiliary data foundation for the entire retrieval process. By pre-processing computationally intensive feature extraction and index building offline or when data changes, the real-time computational overhead during online retrieval is avoided. Lightweight index data storage and rapid loading enable efficient approximate filtering of large-scale candidate sets before fine-grained ranking. Ultimately, this lays a data foundation for reducing the number of precise distance calculations and improving system throughput and response speed, making it particularly suitable for large-scale vector retrieval scenarios with frequent data updates and stringent timeliness requirements.

[0109] Please see Figure 2 This application provides a schematic diagram of an optimization device for large model vector similarity search, comprising:

[0110] The query vector acquisition unit 21 is used to acquire the query vector;

[0111] The first vector determination unit 22 is used to determine the first candidate vector set based on the query vector and the preset first-level index;

[0112] The approximate distance calculation unit 23 is used to perform approximate distance calculation on the first candidate vector set based on the secondary index data of the query vector and each candidate vector in the first candidate vector set, and obtain the approximate distance calculation result;

[0113] The second vector determination unit 24 is used to filter the first candidate vector set based on the approximate distance calculation result and determine the second candidate vector set;

[0114] The precise distance calculation unit 25 is used to perform precise distance calculation on each candidate vector in the second candidate vector set to obtain the precise distance calculation result.

[0115] The target retrieval determination unit 26 is used to determine the target retrieval result based on the accurate distance calculation result.

[0116] Please see Figure 3 This application also provides an electronic device 300, including a memory 310, a processor 320, and a computer program 311 stored in the memory 310 and executable on the processor. When the processor 320 executes the computer program 311, it implements the steps of an optimization method for large model vector similarity search.

[0117] Since the electronic device described in this embodiment is the device used to implement the optimization device for large model vector similarity search in the embodiments of this application, those skilled in the art can understand the specific implementation method and various variations of the electronic device in this embodiment based on the method described in the embodiments of this application. Therefore, how the electronic device implements the method in the embodiments of this application will not be described in detail here. Any device used by those skilled in the art to implement the method in the embodiments of this application falls within the scope of protection of this application.

[0118] In practice, when the computer program 311 is executed by the processor, it can implement any of the embodiments corresponding to the first aspect.

[0119] It should be noted that the descriptions of each embodiment in the above embodiments have different focuses. For parts that are not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0120] Those skilled in the art will understand that embodiments of this application can provide methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-readable storage media containing computer-readable program code.

[0121] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create a machine for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0122] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0123] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0124] This application also provides a computer program product, which includes computer software instructions that, when executed on a processing device, cause the processing device to perform... Figure 1 The flowchart of an optimization method for large model vector similarity search in the corresponding embodiment.

[0125] A computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the flow or function according to the embodiments of this application is generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transferred from one computer-readable storage medium to another. For example, computer instructions may be transferred from one website, computer, server, or data center to another website, computer, server, or data center via wired or wireless means. The computer-readable storage medium may be any usable medium that a computer can store or a data storage device such as a server or data center that integrates one or more usable media. The usable medium may be a magnetic medium, an optical medium, or a semiconductor medium, etc.

[0126] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0127] In the several embodiments provided in this application, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; multiple units or components may be combined or integrated into another system, or some features may be omitted or not performed. Furthermore, the mutual couplings or direct couplings or communication connections shown or discussed may be indirect couplings or communication connections through some interfaces, devices, or units, and may be electrical, mechanical, or other forms.

[0128] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0129] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated units described above can be implemented in the form of hardware and / or software functional units.

[0130] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device to execute all or part of the steps of the methods in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory, magnetic disks, or optical disks.

[0131] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

[0132] Although preferred embodiments have been described in this specification, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications that fall outside the scope of this specification.

[0133] Obviously, those skilled in the art can make various modifications to this specification without departing from its spirit and scope. Therefore, this specification also intends to include any modifications that fall within the scope of the claims and their equivalents.

Claims

1. An optimization method for large-scale model vector similarity search, characterized in that, include: Retrieve the query vector; Based on the query vector and the preset first-level index, a first candidate vector set is determined; Based on the query vector and the secondary index data of each candidate vector in the first candidate vector set, an approximate distance calculation is performed on the first candidate vector set to obtain the approximate distance calculation result; Based on the approximate distance calculation results, the first candidate vector set is filtered to determine the second candidate vector set; Perform precise distance calculation on each candidate vector in the second candidate vector set to obtain the precise distance calculation result; Based on the accurate distance calculation results, the target retrieval results are determined.

2. The method according to claim 1, characterized in that, The step of determining the first candidate vector set based on the query vector and the preset first-level index includes: Based on the index type of the preset first-level index, determine the corresponding retrieval strategy; Based on the retrieval strategy and the query vector, an approximate nearest neighbor retrieval is performed from the vector database; Based on the results returned by the approximate nearest neighbor retrieval, the first candidate vector set is determined.

3. The method according to claim 1, characterized in that, The step of performing an approximate distance calculation on the first candidate vector set based on the query vector and the secondary index data of each candidate vector in the first candidate vector set, to obtain the approximate distance calculation result, includes: Based on the identifiers of each candidate vector in the first candidate vector set, the secondary index data corresponding to each candidate vector is loaded from the secondary index storage module; Based on the secondary index data of the query vector and each candidate vector, the approximate distance between the query vector and each candidate vector is calculated respectively. The approximate distance calculation result is generated based on the approximate distance of each candidate vector.

4. The method according to claim 1, characterized in that, The step of filtering the first candidate vector set based on the approximate distance calculation result to determine the second candidate vector set includes: Based on the preset filtering threshold and the approximate distance calculation result, each candidate vector in the first candidate vector set is filtered and judged to determine the retention status of each candidate vector. Based on the retention status of each candidate vector, candidate vectors that need to be retained are selected from the first candidate vector set; Based on the selected candidate vectors that need to be retained, the second candidate vector set is determined.

5. The method according to claim 1, characterized in that, The step of performing precise distance calculation on each candidate vector in the second candidate vector set to obtain precise distance calculation results includes: Based on the identifiers of each candidate vector in the second candidate vector set, the original high-dimensional vector data of each candidate vector is loaded from the vector storage module; Based on the original high-dimensional vector data of the query vector and each candidate vector, calculate the precise distance between the query vector and each candidate vector respectively; The precise distance calculation result is generated based on the precise distance of each candidate vector.

6. The method according to claim 1, characterized in that, The determination of the target retrieval result based on the accurate distance calculation result includes: Based on the precise distance calculation results, the candidate vectors in the second candidate vector set are sorted. The target retrieval result is determined based on the sorting results and a preset quantity threshold.

7. The method according to claim 1, characterized in that, Before determining the first candidate vector set based on the query vector and the preset first-level index, the method further includes: Based on each original high-dimensional vector in the vector library, pre-calculate the secondary index data corresponding to each original high-dimensional vector; Based on the identifiers of each original high-dimensional vector, the secondary index data is associated with and stored with the corresponding original high-dimensional vector; In response to the update operation of the vector library, the secondary index data is dynamically updated based on the added and / or deleted vector data.

8. An optimization device for large-scale model vector similarity search, characterized in that, include: The query vector acquisition unit is used to acquire the query vector. The first vector determination unit is used to determine a first candidate vector set based on the query vector and a preset first-level index; An approximate distance calculation unit is used to perform approximate distance calculation on the first candidate vector set based on the query vector and the secondary index data of each candidate vector in the first candidate vector set, and obtain the approximate distance calculation result; The second vector determination unit is used to filter the first candidate vector set based on the approximate distance calculation result to determine the second candidate vector set; The precise distance calculation unit is used to perform precise distance calculation on each candidate vector in the second candidate vector set to obtain the precise distance calculation result. The target retrieval and determination unit is used to determine the target retrieval result based on the accurate distance calculation result.

9. An electronic device, comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor executes the computer program stored in the memory to implement the steps of the optimization method for large model vector similarity search as claimed in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the optimization method for large model vector similarity search as described in any one of claims 1 to 7.

Images