Mass data vector index recall rate estimation method and device, medium and equipment

Abstract

The application discloses a recall rate estimation method and device for a mass data vector index, a medium and equipment. The method comprises the following steps: extracting a first preset number of vectors from a first online vector index structure to form a test query data set; based on the first online vector index structure, using a first recall query parameter and a second recall query parameter to query each vector in the test query data set, and calculating a first overall recall rate estimation value; extracting a second preset number of vectors from a full data set to form a verification data set, taking the verification data set as a query reference range, using an accurate index method and an online index method to query each vector in the test query data set, and calculating a second overall recall rate estimation value; and calculating a final recall rate estimation value based on the first overall recall rate estimation value and the second overall recall rate estimation value. The application improves the recall rate estimation efficiency and ensures the accuracy of the estimation result.

Description

Technical Field

[0001] This application relates to the field of information retrieval technology, and in particular to a method, apparatus, medium and device for estimating the recall rate of a massive data vector index. Background Technology

[0002] With the development of artificial intelligence and big data technologies, vector retrieval (or approximate nearest neighbor search, ANN) has been widely applied in recommender systems, image retrieval, natural language processing, and other fields. To efficiently manage massive amounts of vector data, the industry has proposed various vector indexing structures. Their core objective is to return the most similar item to the query vector with a high recall rate within an acceptable timeframe. In this process, recall rate is the core metric for measuring vector indexing performance, typically calculated by statistically analyzing the overlap between the indexed results and the actual nearest neighbor results.

[0003] Current recall estimation methods primarily rely on brute-force computation across the entire dataset. This approach requires performing linear scans and other brute-force calculations on each test vector across the complete vector dataset to obtain its true nearest neighbor as the baseline. However, when dealing with high-dimensional data in the tens or even hundreds of millions, brute-force computation is not only extremely time-consuming but also consumes significant amounts of CPU and memory resources, resulting in extremely low recall estimation efficiency.

[0004] Therefore, existing technologies are insufficient to meet the demand for rapid recall estimation of vector indexes in massive data scenarios while ensuring the accuracy of recall estimation. Summary of the Invention

[0005] In view of this, this application provides a method, apparatus, medium and device for estimating the recall rate of massive data vector indexes, which solves the problem that existing methods for estimating the recall rate of massive data vector indexes are difficult to achieve both high accuracy and high efficiency in recall rate estimation.

[0006] According to one aspect of this application, a method for estimating the recall rate of a massive data vector index is provided, the method comprising: Obtain the full dataset corresponding to the first online vector index structure, and extract a first preset number of vectors from the full dataset to form a test query dataset; Based on the first online vector index structure, each vector in the test query dataset is queried using the first recall query parameter and the second recall query parameter respectively to obtain the first query result. A first overall recall rate estimate is generated based on the first query result, wherein the first recall query parameter is greater than the second recall query parameter. A second preset number of vectors are extracted from the full dataset to form a validation dataset. The validation dataset is used as the query baseline range. Each vector in the test query dataset is queried using both the exact indexing method and the online indexing method to obtain a second query result. A second overall recall rate estimate is generated based on the second query result. The final recall estimate is calculated based on the first overall recall estimate and the second overall recall estimate.

[0007] Optionally, the step of querying each vector in the test query dataset using the first online vector index structure and the first and second recall query parameters respectively to obtain the first query result includes: Based on the first online vector index structure, the first recall query parameters are used to query each vector in the test query dataset to obtain a third preset number of first approximate nearest neighbor results for each vector, and the multiple first approximate nearest neighbor results for each vector are used as the approximate real dataset corresponding to the vector. Based on the first online vector index structure, the second recall query parameters are used to query each vector in the test query dataset to obtain a fourth preset number of second approximate nearest neighbor results for each vector, and the multiple second approximate nearest neighbor results corresponding to each vector are used as the first recall dataset corresponding to the vector.

[0008] Optionally, generating a first overall recall estimate based on the first query result includes: For each vector in the test query dataset, obtain the top four preset number of data points in the approximate real dataset corresponding to the vector, as an approximate data subset, and determine the first number of data points contained in the intersection of the approximate data subset and the first recall dataset; The ratio of the first quantity to the fourth preset quantity is used as the first recall rate estimate corresponding to the vector; Calculate the first average of the first recall estimates corresponding to all vectors in the test query dataset, and use the product of the first average and the first compensation coefficient as the first overall recall estimate.

[0009] Optionally, the step of using the verification dataset as the query baseline range and querying each vector in the test query dataset using both the exact indexing method and the online indexing method to obtain the second query result includes: A precise index structure is constructed based on the verification dataset. Each vector in the test query dataset is queried based on the precise index structure to obtain a fourth preset number of precise nearest neighbor results for each vector. The multiple precise nearest neighbor results corresponding to each vector are used as the precise real dataset corresponding to the vector. Using the same construction parameters as the first online vector index structure, a second online vector index structure is constructed based on the verification dataset. Based on the second online vector index structure, the second recall query parameters are used to query each vector in the test query dataset to obtain a fourth preset number of third approximate nearest neighbor results corresponding to each vector. The multiple third approximate nearest neighbor results corresponding to each vector are used as the second recall dataset corresponding to the vector.

[0010] Optionally, generating a second overall recall estimate based on the second query result includes: For each vector in the test query dataset, determine a second number of data contained in the intersection of the exact real dataset and the second recall dataset; The ratio of the second quantity to the fourth preset quantity is used as the second recall rate estimate corresponding to the vector; Calculate the second average of the second recall estimates corresponding to all vectors in the test query dataset, and use the product of the second average and the second compensation coefficient as the second overall recall estimate.

[0011] Optionally, the step of calculating the final recall estimate based on the first overall recall estimate and the second overall recall estimate includes: The average of the first overall recall estimate and the second overall recall estimate is used as the final recall estimate.

[0012] Optionally, the recall estimation method for the massive data vector index further includes: The setting size of the first recall query parameter satisfies the following condition: the error between the first overall recall rate estimate and the brute-force recall rate estimate is less than or equal to the error threshold, wherein the brute-force recall rate estimate is obtained by using the full dataset as the query benchmark range and performing a brute-force search method to obtain the recall rate estimate.

[0013] According to another aspect of this application, a recall estimation device for massive data vector indexes is provided, the device comprising: The query dataset acquisition module is used to obtain the full dataset corresponding to the first online vector index structure, and extract a first preset number of vectors from the full dataset to form a test query dataset. The first recall estimation module is used to query each vector in the test query dataset based on the first online vector index structure, using a first recall query parameter and a second recall query parameter respectively, to obtain a first query result, and generate a first overall recall estimate based on the first query result, wherein the first recall query parameter is greater than the second recall query parameter. The second recall estimation module is used to extract a second preset number of vectors from the full dataset to form a validation dataset, use the validation dataset as the query baseline range, and use the exact indexing method and the online indexing method to query each vector in the test query dataset to obtain a second query result, and generate a second overall recall estimate based on the second query result. The final recall estimation module is used to calculate the final recall estimate based on the first overall recall estimate and the second overall recall estimate.

[0014] Optionally, the first recall estimation module is further configured to: Based on the first online vector index structure, the first recall query parameters are used to query each vector in the test query dataset to obtain a third preset number of first approximate nearest neighbor results for each vector, and the multiple first approximate nearest neighbor results for each vector are used as the approximate real dataset corresponding to the vector. Based on the first online vector index structure, the second recall query parameters are used to query each vector in the test query dataset to obtain a fourth preset number of second approximate nearest neighbor results for each vector, and the multiple second approximate nearest neighbor results corresponding to each vector are used as the first recall dataset corresponding to the vector.

[0015] Optionally, the first recall estimation module is further configured to: For each vector in the test query dataset, obtain the top four preset number of data points in the approximate real dataset corresponding to the vector, as an approximate data subset, and determine the first number of data points contained in the intersection of the approximate data subset and the first recall dataset; The ratio of the first quantity to the fourth preset quantity is used as the first recall rate estimate corresponding to the vector; Calculate the first average of the first recall estimates corresponding to all vectors in the test query dataset, and use the product of the first average and the first compensation coefficient as the first overall recall estimate.

[0016] Optionally, the second recall estimation module is further configured to: A precise index structure is constructed based on the verification dataset. Each vector in the test query dataset is queried based on the precise index structure to obtain a fourth preset number of precise nearest neighbor results for each vector. The multiple precise nearest neighbor results corresponding to each vector are used as the precise real dataset corresponding to the vector. Using the same construction parameters as the first online vector index structure, a second online vector index structure is constructed based on the verification dataset. Based on the second online vector index structure, the second recall query parameters are used to query each vector in the test query dataset to obtain a fourth preset number of third approximate nearest neighbor results corresponding to each vector. The multiple third approximate nearest neighbor results corresponding to each vector are used as the second recall dataset corresponding to the vector.

[0017] Optionally, the second recall estimation module is further configured to: For each vector in the test query dataset, determine a second number of data contained in the intersection of the exact real dataset and the second recall dataset; The ratio of the second quantity to the fourth preset quantity is used as the second recall rate estimate corresponding to the vector; Calculate the second average of the second recall estimates corresponding to all vectors in the test query dataset, and use the product of the second average and the second compensation coefficient as the second overall recall estimate.

[0018] Optionally, the final recall estimation module is further configured to: The average of the first overall recall estimate and the second overall recall estimate is used as the final recall estimate.

[0019] Optionally, the recall estimation device for the massive data vector index further includes: The first recall query parameter setting module is used to set the size of the first recall query parameter to meet the following condition: the error between the first overall recall rate estimate and the brute-force recall rate estimate is less than or equal to the error threshold, wherein the brute-force recall rate estimate is obtained by using the full dataset as the query benchmark range and performing a brute-force search method to obtain the recall rate estimate.

[0020] According to another aspect of this application, a storage medium is provided that stores a computer program thereon, which, when executed by a processor, implements the above-described method for estimating the recall rate of a massive data vector index.

[0021] According to another aspect of this application, a computer device is provided, including a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, wherein the processor executes the program to implement the above-described method for estimating the recall rate of a massive data vector index.

[0022] Using the above technical solution, this application provides a method, apparatus, medium, and device for estimating the recall rate of a massive data vector index. The method involves obtaining the full dataset corresponding to a first online vector index structure, extracting a first preset number of vectors from the full dataset to form a test query dataset, querying each vector in the test query dataset using a first and a second recall query parameter based on the first online vector index structure, generating a first overall recall rate estimate based on the query results, extracting a second preset number of vectors from the full dataset to form a validation dataset, and using the validation dataset as the query baseline range. Instead of using precise indexing and online indexing methods to query each vector in the test query dataset, a second overall recall estimate is generated based on the second query result. The final recall estimate is calculated based on the first and second overall recall estimates. The entire evaluation process only requires a limited number of index query operations, without initiating additional, costly full scan calculations. Dual-check estimation can complete the recall evaluation of hundreds of millions of data vector indexes within seconds to tens of seconds, improving the efficiency of recall estimation, with extremely low resource consumption, and ensuring the accuracy of the estimation results through dual-check estimation.

[0023] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this application more obvious and understandable, the following are specific embodiments of this application. Attached Figure Description

[0024] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings: Figure 1 The illustration shows a flowchart of a method for estimating the recall rate of a massive data vector index according to an embodiment of this application; Figure 2 This paper illustrates another flowchart of a recall estimation method for a massive data vector index provided in an embodiment of this application. Figure 3 This illustration shows another flowchart of a recall estimation method for a massive data vector index provided in an embodiment of this application; Figure 4 This paper illustrates another flowchart of a method for estimating the recall rate of a massive data vector index provided in an embodiment of this application. Figure 5 This illustration shows a schematic diagram of the structure of a recall estimation device for massive data vector indexes provided in an embodiment of this application; Figure 6 A schematic diagram of the device structure of a computer device provided in an embodiment of this application is shown.

[0025] in, Figure 5 In the Chinese: 502 - Query dataset acquisition module; 504 - First recall estimation module; 506 - Second recall estimation module; 508 - Final recall estimation module. Detailed Implementation

[0026] The present application will be described in detail below with reference to the accompanying drawings and embodiments. It should be noted that, unless otherwise specified, the embodiments and features described in the embodiments of the present application can be combined with each other.

[0027] The recall estimation method for massive data vector indexes provided in this invention can be applied in server-side application environments with massive data retrieval capabilities.

[0028] This embodiment provides a method for estimating the recall rate of a massive data vector index, such as... Figure 1 As shown, the method includes: 102: Obtain the full dataset corresponding to the first online vector index structure, and extract the first preset number of vectors from the full dataset to form the test query dataset; 104: Based on the first online vector index structure, each vector in the test query dataset is queried using the first recall query parameter and the second recall query parameter respectively to obtain the first query result. The first overall recall rate estimate is generated based on the first query result, wherein the first recall query parameter is greater than the second recall query parameter. 106: Extract a second preset number of vectors from the full dataset to form a validation dataset. Use the validation dataset as the query baseline range. Use both the exact indexing method and the online indexing method to query each vector in the test query dataset to obtain the second query results. Generate the second overall recall estimate based on the second query results. 108: The final recall estimate is calculated based on the first overall recall estimate and the second overall recall estimate.

[0029] Specifically, a first preset number of vectors are extracted from the full dataset corresponding to the first online vector indexing system to form a test query dataset. Using the full dataset as the query scope, each vector in the test query dataset is queried using the first and second recall query parameters on the first online vector indexing structure. Based on the results of the two queries, a first overall recall estimate is calculated, achieving the first approximate evaluation. A second preset number of vectors are extracted from the full dataset to form a validation set. This second preset number is much smaller than the total number of vectors in the full dataset, and the validation set is distributed similarly to the full dataset. A precise targeting mechanism is then constructed on the validation set. The index structure is based on the exact index structure, and each vector in the test query dataset is queried. A second online vector index structure, identical to the first online vector index structure, is constructed on the validation set (e.g., maximum node connections M=64, efConstruction=200). The second recall query parameters are then used to query each vector in the test query dataset. Based on the query results from both methods in the validation set, a second overall recall estimate is calculated. Finally, based on the first and second overall recall estimates, the final recall estimate for the online vector index is calculated.

[0030] The average of the first overall recall estimate and the second overall recall estimate is used as the final recall estimate. By querying a small number of first recall query parameters (e.g., 1000 times), a near-real dataset can be constructed within seconds to tens of seconds, completely avoiding brute-force calculations on the full dataset (potentially hundreds of millions), reducing the evaluation time from hours to seconds.

[0031] The online vector indexing method is either HNSW online vector indexing or IVFPQ online vector indexing, and the precise indexing method is FLAT indexing or any indexing method that can return the exact nearest neighbor search results.

[0032] This application provides a payment order processing method. Compared with the prior art, it obtains the full dataset corresponding to a first online vector index structure, extracts a first preset number of vectors from the full dataset to form a test query dataset, and queries each vector in the test query dataset using a first recall query parameter and a second recall query parameter based on the first online vector index structure. A first overall recall estimate is generated based on the query results. A second preset number of vectors are extracted from the full dataset to form a validation dataset, which is used as the query baseline range. Each vector in the test query dataset is queried using both the exact indexing method and the online indexing method. A second overall recall estimate is generated based on the second query results. The final recall estimate is calculated based on the first and second overall recall estimates. The entire evaluation process requires only a limited number of index query operations, without initiating additional, costly full scan calculations. Dual validation estimation can complete the recall evaluation of a vector index with hundreds of millions of data points within seconds to tens of seconds, improving the efficiency of recall estimation, consuming very few resources, and ensuring the accuracy of the estimation results through dual validation estimation.

[0033] Furthermore, as a refinement and extension of the specific implementation of the above embodiments, and to fully illustrate the specific implementation process of this embodiment, another method for estimating the recall rate of massive data vector indexes is provided, such as... Figure 2 As shown, based on the first online vector index structure, each vector in the test query dataset is queried using the first recall query parameter and the second recall query parameter respectively to obtain the first query result. Based on the first query result, a first overall recall rate estimate is generated, including: 202: Based on the first online vector index structure, use the first recall query parameters to query each vector in the test query dataset to obtain the third preset number of first approximate nearest neighbor results corresponding to each vector, and use the multiple first approximate nearest neighbor results corresponding to each vector as the approximate real dataset corresponding to the vector; 204: Based on the first online vector index structure, the second recall query parameters are used to query each vector in the test query dataset to obtain the fourth preset number of second approximate nearest neighbor results corresponding to each vector, and the multiple second approximate nearest neighbor results corresponding to each vector are used as the first recall dataset corresponding to the vector. 206: For each vector in the test query dataset, obtain the top four preset number of data points in the approximate real dataset corresponding to the vector, as an approximate data subset, and determine the first number of data points contained in the intersection of the approximate data subset and the first recall dataset; 208: The ratio of the first quantity to the fourth preset quantity is used as the first recall estimate corresponding to the vector; 210: Calculate the first average of the first recall estimates corresponding to all vectors in the test query dataset, and use the product of the first average and the first compensation coefficient as the first overall recall estimate.

[0034] Specifically, from the full dataset corresponding to the first online vector index structure, a first preset number of M vectors are randomly selected to form the test query dataset Q. M is a small fixed value, preferably 1000, to ensure evaluation efficiency. On the first online vector index structure, a set of high recall query parameters (denoted as the first recall query parameter Params_high) is used to query each vector in the test query dataset. Each query returns the top three preset number of approximate nearest neighbor results (L is usually 100). The approximate nearest neighbor results corresponding to each vector are merged to form the approximate real dataset for that vector.

[0035] The setting of the first recall query parameter Params_high must ensure that, under the current index state, the error between the recall evaluation value of its query results and the recall evaluation value of the brute-force calculation results is within a known small preset threshold ε (for example, for HNSW online vector index, ε≤1%; for IVFPQ online vector index, ε≤10%).

[0036] On the first online vector index structure, the normal online service query parameters (denoted as the second recall query parameter Params_normal) are used to query each vector in the test query dataset. Each vector query returns a fourth preset number of K results (K is the value required by the service, such as 10, 20) as the first recall dataset for that vector.

[0037] For each query vector, calculate the number of data points contained in the intersection of the first recall dataset and the first K results of the corresponding vector in the approximate true dataset. Then, divide the number of data points in the intersection by the fourth preset number K to obtain the first recall evaluation value for that vector query. Calculate the average of the first recall evaluation values ​​for all vectors in the test query dataset.

[0038] Since the approximate real dataset is an approximation, to compensate for its deviation from the real data, the average of the first recall evaluation value is multiplied by a compensation coefficient α (0 < α ≤ 1) to obtain the first overall recall evaluation value Recall1 = Recall_raw. α. The value of α is related to the error threshold ε. For example, when ε is 10%, α can be set to 0.9, and when ε is 1%, α can be set to 0.99.

[0039] The setting of the first recall query parameter satisfies the following condition: the error between the first overall recall estimate and the brute-force recall estimate is less than or equal to the error threshold ε, wherein the brute-force recall estimate is obtained by using the entire dataset as the query baseline range and performing a brute-force search.

[0040] The first recall query parameter is configured such that the error between the query results from the online vector index and the actual results obtained from a brute-force search on the full dataset, under this parameter, is less than or equal to an error threshold. When the online vector index is an HNSW index, the error threshold is 1%; when the online vector index is an IVFPQ index, the error threshold is 10%. The compensation coefficient is determined based on the error threshold: when the error threshold is approximately 10%, the compensation coefficient is 0.9; when the error threshold is approximately 1%, the compensation coefficient is 0.99.

[0041] In one embodiment, the first preset quantity is 1000, the second preset quantity is 10000, the third preset quantity is 100, and the fourth preset quantity is 10, 20, or 100.

[0042] In one embodiment, such as Figure 3 As shown, a second preset number of vectors are extracted from the full dataset to form a validation dataset. Using this validation dataset as the query baseline, each vector in the test query dataset is queried using both precise indexing and online indexing methods to obtain the second query results. Based on these second query results, a second overall recall estimate is generated, including: 302: Construct an accurate index structure based on the verification dataset, query each vector in the test query dataset based on the accurate index structure, obtain the fourth preset number of accurate nearest neighbor results for each vector, and use the multiple accurate nearest neighbor results for each vector as the accurate real dataset corresponding to the vector. 304: Using the same construction parameters as the first online vector index structure, a second online vector index structure is constructed based on the validation dataset. Based on the second online vector index structure, the second recall query parameters are used to query each vector in the test query dataset to obtain the fourth preset number of third approximate nearest neighbor results corresponding to each vector. The multiple third approximate nearest neighbor results corresponding to each vector are used as the second recall dataset corresponding to the vector. 306: For each vector in the test query dataset, determine the second number of data contained in the intersection of the exact true dataset and the second recall dataset; 308: The ratio of the second quantity to the fourth preset quantity is used as the second recall estimate corresponding to the vector; 310: Calculate the second average of the second recall estimates corresponding to all vectors in the test query dataset, and use the product of the second average and the second compensation coefficient as the second overall recall estimate.

[0043] Specifically, a relatively large subset is randomly selected from the full data of the first online index structure as a verification dataset, and a second preset number N (e.g., N=10000) is extracted, and the data of the extracted verification set is distributed in the same way as the full data.

[0044] Build an exact index structure (such as a FLAT index, i.e., a brute-force search index) on the validation dataset without compression or approximation. Execute queries on the test query dataset on this exact index structure, returning the Top K (fourth preset number) exact approximate results. Use this result as the exact true dataset for each vector on the validation dataset.

[0045] On the same validation dataset, a small second online vector index structure with the same parameters as the first online vector index structure is constructed. Using the same second recall query parameters, each vector in the test query set is queried on the second online vector index structure to obtain a fourth preset number of near-neighbor results for each vector, which serve as the second recall dataset for each vector.

[0046] For each vector in the test query dataset, the second recall dataset is compared with the exact real dataset. Following the standard recall calculation method, the quotient of the number of data points in the intersection of the second recall dataset and the exact real dataset and a fourth preset number is used as the second recall evaluation value for that vector. The product of the average of the second recall evaluation values ​​of all vectors in the test query dataset and the second compensation coefficient is used as the second overall recall evaluation value. This value reflects the performance of the current index structure under a controlled and precise environment with consistent data distribution.

[0047] By introducing a second, small-scale, accurate validation based on identically distributed sampling, two problems are effectively solved: first, the estimation bias caused by the potential severe degradation of index performance in the first approximation method (leading to an increased compensation coefficient); and second, a performance benchmark is provided in a clean, controllable data environment. Averaging the dual results significantly improves the reliability and robustness of the final estimate.

[0048] In one embodiment, taking the HNSW (Hierarchical Navigable Small World) indexing method as an example, the HNSW online vector indexing system manages 100 million 1024-dimensional image feature vectors, with efSearch=200 (second recall query parameter=200), and requires the return of the Top 20 results (fourth preset quantity equals 20). Figure 4 As shown, the recall estimation method for massive data vector indexes is as follows: Step S1: Randomly extract 1,000 feature vectors from the 100 million full data points corresponding to the HNSW index structure on the first line, as the test query dataset Q.

[0049] Step S2-1: On the first line HNSW index structure, use the high recall parameter efSearch=800 (first recall query parameter=800, verified that the difference between the result under this parameter and the recall estimation rate under brute calculation is <1%) to query each vector in the test query dataset Q, obtain the approximate nearest neighbor results of the Top100 (third preset number=100) of each query, and summarize the approximate nearest neighbor results of each vector in the test query dataset to obtain the approximate GroundTruth set G1.

[0050] Steps S2-2 and S2-3: On the online HNSW index structure, use the normal parameter efSearch=200 to query each vector in the test query dataset Q, and obtain the Top 20 (fourth preset number = 20) approximate nearest neighbor results as the first recall dataset. For each vector in the test query dataset, calculate the overlap between the G1 corresponding to each vector and the corresponding first recall dataset. Assuming the average overlap of 1000 queries is 18.5, the estimated first recall rate Recall_raw = 18.5 / 20 = 92.5%.

[0051] Step S2-4: Considering that the test query dataset G1 has an error of approximately 1%, a compensation coefficient α = 0.99 is taken. Therefore, the first evaluation result Recall1 = 92.5%. 0.99 ≈ 91.58%.

[0052] Steps S3-1 and S3-2: From the 100 million data points, randomly select another 10,000 data points as the validation set Base_c. Build an exact index FLAT on Base_c and query each vector in the test query dataset Q to obtain the exact Top 20 results (exact nearest neighbor results), which serve as the exact GroundTruth set G2 (exact real dataset).

[0053] Steps S3-3 and S3-4: On Base_c, construct a small second-line HNSW index structure Index_c using the same construction parameters as the first online HNSW index structure (e.g., M=64, efConstruction=200). Query each vector in the test query dataset Q on Index_c with efSearch=200 to obtain the Top 20 nearest neighbor similarity results. Compare the Top 20 nearest neighbor similarity results for each vector with G2, assuming a second recall evaluation value of Recall2 = 93.00%.

[0054] Step S4: Calculate the final estimated recall rate: Recall_final = (91.58% + 93.00%) / 2 = 92.29%.

[0055] This result indicates that, under the current data and indexing conditions, the estimated recall rate for the online service is approximately 92.29%. The entire evaluation process, S2-1-S2-4, took only a few seconds. S3-1-S3-4, with only 10,000 records, could also be completed within minutes by brute-force building of the FLAT index and querying. The total time consumed was significantly lower than brute-force calculations on hundreds of millions of records, thus improving the efficiency of recall rate calculation.

[0056] For indexes with relatively low recall rates, such as the IVFPQ online index structure, the error between high-parameter query results and brute-force calculations may be larger (e.g., 10%). The compensation coefficient α can be lowered accordingly, such as setting it to 0.9, and then performing a second verification of the results, which can effectively prevent incorrect evaluations caused by excessive approximation errors.

[0057] Furthermore, as Figure 5 In terms of specific implementation, this application provides a recall estimation device for massive data vector indexes, such as... Figure 5 As shown, the device includes: 502: Query dataset acquisition module, used to obtain the full dataset corresponding to the first online vector index structure, and extract a first preset number of vectors from the full dataset to form the test query dataset; 504: The first recall estimation module is used to query each vector in the test query dataset based on the first online vector index structure, using the first recall query parameter and the second recall query parameter respectively, to obtain the first query result, and generate the first overall recall estimate based on the first query result, wherein the first recall query parameter is greater than the second recall query parameter; 506: The second recall estimation module is used to extract a second preset number of vectors from the full dataset to form a validation dataset. The validation dataset is used as the query baseline range. The exact indexing method and the online indexing method are used to query each vector in the test query dataset to obtain the second query result. The second overall recall estimate is generated based on the second query result. 508: Final recall estimation module, used to calculate the final recall estimate based on the first overall recall estimate and the second overall recall estimate.

[0058] This application provides a payment and receipt processing device. Compared with the prior art, it obtains the full dataset corresponding to a first online vector index structure, extracts a first preset number of vectors from the full dataset to form a test query dataset, and queries each vector in the test query dataset using a first recall query parameter and a second recall query parameter based on the first online vector index structure. A first overall recall estimate is generated based on the query results. A second preset number of vectors are extracted from the full dataset to form a verification dataset. The verification dataset is used as the query benchmark range, and each vector in the test query dataset is queried using both the exact index method and the online index method. A second overall recall estimate is generated based on the second query results. The final recall estimate is calculated based on the first and second overall recall estimates. The entire evaluation process only requires a limited number of index query operations, without initiating additional, costly full scan calculations. Dual verification estimation can complete the recall evaluation of a data vector index of hundreds of millions of data points within seconds to tens of seconds, improving the efficiency of recall estimation, with extremely low resource consumption, and ensuring the accuracy of the estimation results through dual verification estimation.

[0059] In one embodiment, the first recall estimation module is further configured to: Based on the first online vector index structure, the first recall query parameters are used to query each vector in the test query dataset to obtain the third preset number of first approximate nearest neighbor results corresponding to each vector, and the multiple first approximate nearest neighbor results corresponding to each vector are used as the approximate real dataset corresponding to the vector. Based on the first online vector index structure, the second recall query parameters are used to query each vector in the test query dataset to obtain the fourth preset number of second approximate nearest neighbor results for each vector. The multiple second approximate nearest neighbor results corresponding to each vector are used as the first recall dataset corresponding to the vector.

[0060] In one embodiment, the first recall estimation module is further configured to: For each vector in the test query dataset, obtain the first four preset number of data points in the approximate real dataset corresponding to the vector, as an approximate data subset, and determine the first number of data points contained in the intersection of the approximate data subset and the first recall dataset; The ratio of the first quantity to the fourth preset quantity is used as the first recall rate estimate corresponding to the vector; Calculate the first average of the first recall estimates corresponding to all vectors in the test query dataset, and use the product of the first average and the first compensation coefficient as the first overall recall estimate.

[0061] In one embodiment, the second recall estimation module is further configured to: A precise index structure is constructed based on the verification dataset. Each vector in the test query dataset is queried based on the precise index structure to obtain the fourth preset number of precise nearest neighbor results for each vector. The multiple precise nearest neighbor results corresponding to each vector are used as the precise real dataset corresponding to the vector. Using the same construction parameters as the first online vector index structure, a second online vector index structure is constructed based on the validation dataset. Based on the second online vector index structure, the second recall query parameters are used to query each vector in the test query dataset to obtain a fourth preset number of third approximate nearest neighbor results for each vector. The multiple third approximate nearest neighbor results corresponding to each vector are used as the second recall dataset corresponding to the vector.

[0062] In one embodiment, the second recall estimation module is further configured to: For each vector in the test query dataset, determine the second number of data contained in the intersection of the exact true dataset and the second recall dataset; The ratio of the second quantity to the fourth preset quantity is used as the second recall estimate corresponding to the vector; Calculate the second average of the second recall estimates corresponding to all vectors in the test query dataset, and use the product of the second average and the second compensation coefficient as the second overall recall estimate.

[0063] In one embodiment, the final recall estimation module is further used for: The average of the first overall recall estimate and the second overall recall estimate is used as the final recall estimate.

[0064] In one embodiment, the recall estimation device for massive data vector indexes further includes: The first recall query parameter setting module is used to set the size of the first recall query parameter to meet the following conditions: the error between the first overall recall rate estimate and the brute-force recall rate estimate is less than or equal to the error threshold, wherein the brute-force recall rate estimate is obtained by using the full dataset as the query benchmark range and performing a brute-force search.

[0065] It should be noted that other corresponding descriptions of the functional units involved in the payment and receipt processing device provided in this application embodiment can be found by referring to... Figures 1 to 3 The corresponding descriptions in the method will not be repeated here.

[0066] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 6 As shown. The computer device includes a processor, memory, and communication interface connected via a system bus, and may also include input / output interfaces and a display device. The processor provides computing and control capabilities. The memory includes non-volatile and / or volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The communication interface is used to communicate with external clients via a network connection. When the computer program is executed by the processor, it implements the functions or steps of a payment / receipt processing method on the server side.

[0067] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to perform the following steps: Obtain the full dataset corresponding to the first online vector index structure, and extract a first preset number of vectors from the full dataset to form the test query dataset; Based on the first online vector index structure, each vector in the test query dataset is queried using the first recall query parameter and the second recall query parameter respectively to obtain the first query result. The first overall recall rate estimate is generated based on the first query result, wherein the first recall query parameter is greater than the second recall query parameter. A second preset number of vectors are extracted from the full dataset to form a validation dataset. The validation dataset is used as the query baseline range. Each vector in the test query dataset is queried using both the exact indexing method and the online indexing method to obtain the second query result. The second overall recall rate estimate is generated based on the second query result. The final recall estimate is calculated based on the first overall recall estimate and the second overall recall estimate.

[0068] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, the computer program performing the following steps when executed by a processor: Obtain the full dataset corresponding to the first online vector index structure, and extract a first preset number of vectors from the full dataset to form the test query dataset; Based on the first online vector index structure, each vector in the test query dataset is queried using the first recall query parameter and the second recall query parameter respectively to obtain the first query result. The first overall recall rate estimate is generated based on the first query result, wherein the first recall query parameter is greater than the second recall query parameter. A second preset number of vectors are extracted from the full dataset to form a validation dataset. The validation dataset is used as the query baseline range. Each vector in the test query dataset is queried using both the exact indexing method and the online indexing method to obtain the second query result. The second overall recall rate estimate is generated based on the second query result. The final recall estimate is calculated based on the first overall recall estimate and the second overall recall estimate.

[0069] Those skilled in the art will understand that Figure 5 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0070] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0071] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0072] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0073] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0074] The above embodiments are merely illustrative of several implementation methods of this application, and their descriptions are relatively specific and detailed. However, they should not be construed as limiting the scope of this application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for estimating the recall rate of a massive data vector index, characterized in that, The method includes: Obtain the full dataset corresponding to the first online vector index structure, and extract a first preset number of vectors from the full dataset to form a test query dataset; Based on the first online vector index structure, each vector in the test query dataset is queried using the first recall query parameter and the second recall query parameter respectively to obtain the first query result. A first overall recall rate estimate is generated based on the first query result, wherein the first recall query parameter is greater than the second recall query parameter. A second preset number of vectors are extracted from the full dataset to form a validation dataset. The validation dataset is used as the query baseline range. Each vector in the test query dataset is queried using both the exact indexing method and the online indexing method to obtain a second query result. A second overall recall rate estimate is generated based on the second query result. The final recall estimate is calculated based on the first overall recall estimate and the second overall recall estimate.

2. The recall estimation method for massive data vector indexes according to claim 1, characterized in that, Based on the first online vector index structure, the first recall query parameter and the second recall query parameter are used to query each vector in the test query dataset to obtain the first query result, including: Based on the first online vector index structure, the first recall query parameters are used to query each vector in the test query dataset to obtain a third preset number of first approximate nearest neighbor results for each vector, and the multiple first approximate nearest neighbor results for each vector are used as the approximate real dataset corresponding to the vector. Based on the first online vector index structure, the second recall query parameters are used to query each vector in the test query dataset to obtain a fourth preset number of second approximate nearest neighbor results for each vector, and the multiple second approximate nearest neighbor results corresponding to each vector are used as the first recall dataset corresponding to the vector.

3. The recall estimation method for massive data vector indexes according to claim 2, characterized in that, The step of generating a first overall recall estimate based on the first query result includes: For each vector in the test query dataset, obtain the top four preset number of data points in the approximate real dataset corresponding to the vector, as an approximate data subset, and determine the first number of data points contained in the intersection of the approximate data subset and the first recall dataset; The ratio of the first quantity to the fourth preset quantity is used as the first recall rate estimate corresponding to the vector; Calculate the first average of the first recall estimates corresponding to all vectors in the test query dataset, and use the product of the first average and the first compensation coefficient as the first overall recall estimate.

4. The recall estimation method for massive data vector indexes according to claim 3, characterized in that, The step involves using the verification dataset as the query baseline range, and querying each vector in the test query dataset using both precise indexing and online indexing methods to obtain a second query result, including: A precise index structure is constructed based on the verification dataset. Each vector in the test query dataset is queried based on the precise index structure to obtain a fourth preset number of precise nearest neighbor results for each vector. The multiple precise nearest neighbor results corresponding to each vector are used as the precise real dataset corresponding to the vector. Using the same construction parameters as the first online vector index structure, a second online vector index structure is constructed based on the verification dataset. Based on the second online vector index structure, the second recall query parameters are used to query each vector in the test query dataset to obtain a fourth preset number of third approximate nearest neighbor results corresponding to each vector. The multiple third approximate nearest neighbor results corresponding to each vector are used as the second recall dataset corresponding to the vector.

5. The recall estimation method for massive data vector indexes according to claim 4, characterized in that, The step of generating a second overall recall estimate based on the second query result includes: For each vector in the test query dataset, determine a second number of data contained in the intersection of the exact real dataset and the second recall dataset; The ratio of the second quantity to the fourth preset quantity is used as the second recall rate estimate corresponding to the vector; Calculate the second average of the second recall estimates corresponding to all vectors in the test query dataset, and use the product of the second average and the second compensation coefficient as the second overall recall estimate.

6. The recall estimation method for massive data vector indexes according to any one of claims 1-5, characterized in that, The calculation of the final recall estimate based on the first overall recall estimate and the second overall recall estimate includes: The average of the first overall recall estimate and the second overall recall estimate is used as the final recall estimate.

7. The recall estimation method for massive data vector indexes according to claim 2, characterized in that, The recall estimation method for the massive data vector index also includes: The setting size of the first recall query parameter satisfies the following condition: the error between the first overall recall rate estimate and the brute-force recall rate estimate is less than or equal to the error threshold, wherein the brute-force recall rate estimate is obtained by using the full dataset as the query benchmark range and performing a brute-force search method to obtain the recall rate estimate.

8. A recall estimation device for massive data vector indexes, characterized in that, The device includes: The query dataset acquisition module is used to obtain the full dataset corresponding to the first online vector index structure, and extract a first preset number of vectors from the full dataset to form a test query dataset. The first recall estimation module is used to query each vector in the test query dataset based on the first online vector index structure, using a first recall query parameter and a second recall query parameter respectively, to obtain a first query result, and generate a first overall recall estimate based on the first query result, wherein the first recall query parameter is greater than the second recall query parameter. The second recall estimation module is used to extract a second preset number of vectors from the full dataset to form a validation dataset, use the validation dataset as the query baseline range, and use the exact indexing method and the online indexing method to query each vector in the test query dataset to obtain a second query result, and generate a second overall recall estimate based on the second query result. The final recall estimation module is used to calculate the final recall estimate based on the first overall recall estimate and the second overall recall estimate.

9. A storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 7.

10. A computer device, comprising a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method of any one of claims 1 to 7.

Images