Document vector similarity calculation method based on meta-learning adaptive selection of bm25 variants

Abstract

The application discloses a document vector similarity calculation method based on meta-learning adaptive selection of BM25 variants, comprising: extracting query meta features of a document; designing a query-based meta-learning selector, selecting BM25 variants, calculating evaluation index scores based on the selected variants, constructing a training sample set according to the scores, constructing a multi-layer perception machine (MLP) meta selector model, and training the model through the training sample set; recalling adaptive BM25; sorting documents according to relevance based on a Cross-Encoder model, and outputting high-relevance documents as retrieval results. The application introduces a lightweight and efficient meta-learning mechanism, realizes adaptive selection of BM25 variants in a 'one search and one strategy' manner, effectively solves the problem that the recall quality is limited due to fixed BM25 parameters in a traditional system, and significantly improves the recall rate in diversified scenarios such as professional terms, colloquial expressions, short tail and long tail queries.

Description

Technical Field

[0001] This invention belongs to the field of information retrieval and artificial intelligence interdisciplinary technology, and in particular relates to a document vector similarity calculation method based on meta-learning adaptive selection of BM25 variants. Background Technology

[0002] In large-scale document retrieval systems, the industry generally adopts a two-stage architecture of "recall + re-ranking": the first stage uses an efficient retrieval model (such as BM25) to quickly recall Top-K candidate documents from a library of hundreds of millions of documents; the second stage uses a high-precision but computationally expensive Cross-Encoder (CE) model to finely re-rank the candidate documents and output the final similarity score.

[0003] The quality of the recall phase directly determines the upper limit of the overall system performance. Currently, most systems use the standard BM25 model (or a specific variant) in the recall phase, and its hyperparameters (such as k1 and b) are uniformly applied globally after being tuned offline on the full dataset.

[0004] However, this "one-size-fits-all" strategy has obvious flaws:

[0005] (1) Different queries have significantly different requirements for retrieval strategies. For example, technical term queries (such as “CRISPR-Cas9 gene editing”) require stronger word frequency sensitivity (high k1), while colloquial long queries (such as “how to fix a computer that won’t turn on”) rely more on document length normalization (low b).

[0006] (2) Existing BM25 variants (such as Lucene BM25, BM25+, and BM25L) each have their own advantages in different scenarios, but there is a lack of a mechanism to select the optimal variant in real time based on query semantics;

[0007] (3) If relevant documents are missed during the recall phase, even if the Cross-Encoder is more powerful in the future, it will be impossible to recall them again, resulting in the overall search performance being limited to a suboptimal initial candidate set;

[0008] (4) There is currently no method to introduce meta-learning into the BM25 variant selection process to achieve “query-aware adaptive recall”.

[0009] Therefore, there is an urgent need for a large-scale document retrieval method that can dynamically select the optimal BM25 variant for high-quality recall based on query content through meta-learning, and effectively connect with Cross-Encoder reordering, in order to break through the performance bottleneck of the existing two-stage architecture. Summary of the Invention

[0010] Purpose of the Invention: The purpose of this invention is to provide a meta-learning-based adaptive recall and re-ranking method for BM25 variants, addressing the limitations in recall quality and the inability to dynamically optimize initial screening results based on query semantics in existing large-scale document retrieval systems due to fixed BM25 parameters. By introducing a query-aware meta-learning mechanism, the optimal BM25 variant is dynamically selected during the recall phase to generate a high-quality candidate set, and seamlessly integrated with a high-precision Cross-Encoder re-ranking module, thereby significantly improving the overall retrieval system's recall rate and ranking accuracy while maintaining industrial-grade deployment efficiency.

[0011] Technical solution: The present invention provides a document vector similarity calculation method based on meta-learning adaptive selection of BM25 variants, comprising the following steps:

[0012] Step 1: Extract the query meta-features of the document;

[0013] Step 2: Design a query-based meta-learning selector to select BM25 variants. Based on the selected variants, calculate the evaluation index score, construct a training sample set according to the score, construct a multilayer perceptron (MLP) meta-selector model, and train the model using the training sample set.

[0014] Step 3: Recall the adaptive BM25;

[0015] Step 4: Sort the documents according to relevance based on the Cross-Encoder model, and output the highly relevant documents as the search results.

[0016] Further, in step 11, query meta-feature extraction;

[0017] For any input query q, the system first extracts a 10-dimensional lightweight meta-feature vector m. q ∈R 10 This feature characterizes the semantic complexity and term distribution of queries. All features can be computed in <1ms, making it suitable for high-concurrency online services.

[0018] Features are divided into two categories:

[0019] The first type is querying its own characteristics (no external resources required):

[0020] q_len: Query the number of tokens after word segmentation;

[0021] unique_term_ratio: The percentage of unique terms (number of unique terms / total number of terms);

[0022] has_question_word: Whether the word contains a question word (such as what, how, why, who, when, where). If yes, it is 1; otherwise, it is 0.

[0023] contains_digit: Whether the data contains a digit; 1 if it does, 0 otherwise.

[0024] avg_word_length: The average length of each word in a single query.

[0025] The second category is query-corpus interaction features (requiring a pre-built document frequency (DF) table):

[0026] Assume the system has already built an inverted index and stored the document frequency (df) of terms. t (Typically from Pyserini, Lucene, or Elasticsearch), df t The number of documents containing word t (i.e., document frequency) can be calculated quickly:

[0027] avg_idf: The average IDF of each term in the query. , where t represents a specific term and N represents the total number of documents in the corpus;

[0028] max_idf: The maximum IDF value of each word in the query;

[0029] idf_std: The standard deviation of the IDF for each term in the query;

[0030] rare_term_count: The number of rare terms (df) t <10);

[0031] common_term_ratio: the proportion of high-frequency words (df) t >10,000).

[0032] All features rely solely on the query text and existing indexes, requiring no additional annotations or model inference.

[0033] Further, step 2, such as Figure 2 As shown, a query-based meta-learning selector is designed for BM25 variant selection.

[0034] Step 21, Define the set of candidate BM25 variants.

[0035] BM25 is a ranking algorithm based on the bag-of-words model, widely used in search engines and information retrieval systems. It ranks documents by calculating the degree of matching between query terms and terms in the documents. Its core idea comprehensively considers the following factors:

[0036] Term Frequency (TF): The frequency with which a word appears in a document. BM25 uses a saturated TF function, meaning that the contribution of increasing term frequency to the score gradually decreases, avoiding excessive dominance by high-frequency words.

[0037] Inverse Document Frequency (IDF): The prevalence of a word across a collection of documents. The rarer the word, the higher its IDF value, indicating greater discriminative power.

[0038] Document length normalization: Longer documents may naturally contain more query terms. BM25 adjusts the score by introducing a comparison between document length and average document length to reduce bias towards longer documents.

[0039] To improve adaptability in different query scenarios, this invention predefines the following four BM25 variants as candidate strategies, each of which makes targeted modifications to the term frequency or length normalization terms based on the standard BM25.

[0040] General symbol explanation: d: document to be rated; q = { t | t is the term appearing in the query}: set of terms in the query; df t : Number of documents containing the word t (document frequency); tf td : The number of times word t appears in document d; L d : The length of document d (in words, which can be the compressed length after removing stop words); L avg : Average length of all documents in the corpus (in words); N: Total number of documents in the corpus; k1>0: Hyperparameter controlling TF saturation; b∈[0,1]: Hyperparameter controlling the strength of length normalization. rsv q(d) : Query the similarity score of documents under q. The higher the value, the more relevant they are.

[0041] The variants are defined as follows:

[0042] Variant 0 (Lucene BM25)

[0043] Derived from the Okapi BM25 model proposed by Robertson et al. in 1994, and adopted as the default similarity function by Apache Lucene since version 6.0 (2016). Its specific formula retains the original BM25's TF saturation and length normalization structure, while using a smoothed logarithmic form of IDF (+0.5) to ensure numerical stability. The scoring function is:

[0044] ;

[0045] Variant 1 (BM25L)

[0046] To address the issue of low-frequency words being underestimated in short documents, a smoothing bias δ ​​(usually set to 1) is added before the TF item to enhance the weight of low-frequency words. The scoring function is as follows:

[0047] ;

[0048] in, ;

[0049] This variant increases the weight of low-frequency words (such as technical terms) in short documents.

[0050] Variant 2 (BM25+)

[0051] To address the issue of excessive penalty for high-frequency words in the original BM25 method in long documents, a constant term is added to the denominator. (Usually set to 1) to mitigate upper limit bias in TF. Its scoring function is:

[0052] ;

[0053] This variant performs better in complex query scenarios such as multi-hop question answering.

[0054] Variant 3 (Log-log BM25)

[0055] Based on the idea of ​​logarithmic word frequency modeling, Rousseau and Vazirgiannis argue that extra occurrences of terms should contribute to the score logarithmically, while preserving the BM25 pivot length normalization structure. Their scoring function is:

[0056] ;

[0057] Among them, the smoothing constant term (Usually set to 1).

[0058] The four variants mentioned above each have their advantages in different query types (such as short queries vs. long queries, factual queries vs. multi-hop inference queries), but traditional systems only use one of them (usually variant 0), which fails to realize their full potential.

[0059] Step 22, construct the training sample set

[0060] Perform the following operations on multiple publicly available search datasets (including but not limited to MS MARCO, HotpotQA, and Quora):

[0061] 1) For each query q, perform document retrieval using the four BM25 variants described above;

[0062] 2) For each variant, calculate its score under the preset evaluation metrics (such as Recall@100 or NDCG@10);

[0063] 3) Select the variant index with the highest score as the true label y for this query. q ∈{0,1,2,3};

[0064] 4) Extract the 10-dimensional feature vector m corresponding to the query. q (Including statistical features such as query length, average IDF, maximum TF, number of unique terms, and IDF variance);

[0065] 5) Construct the training sample set , where N ≥ 50000.

[0066] Step 23, Data Preprocessing and Partitioning

[0067] Data preprocessing: To address the differences in numerical magnitude across different feature dimensions, the input features are standardized using the standard deviation method. Specifically, the mean and standard deviation of the training set features are calculated, and all feature values ​​are converted to a standard normal distribution with a mean of 0 and a variance of 1. This step effectively prevents the vanishing or exploding gradient problem and accelerates the convergence process of the optimizer.

[0068] Data partitioning: A stratified sampling strategy is used to divide the dataset into training and test sets in an 8:2 ratio to ensure that the distribution ratio of each category label in the training and test sets is consistent with that of the original dataset, thus avoiding model evaluation errors caused by data distribution bias.

[0069] Step 24: Construct the Multilayer Perceptron (MLP) meta-selector model

[0070] Design a multilayer perceptron (MLP) classifier with a funnel-shaped hierarchical structure as the meta-selector, whose input is the aforementioned 10-dimensional feature vector m. q The output is the optimal variant index v. q ∗∈{0,1,2,3}.

[0071] This MLP deep neural network aims to extract nonlinear semantic features from a high-dimensional feature space and perform classification predictions. Its feature dimension decreases layer by layer to achieve feature compression and refinement. The network mainly consists of an input interface, three stacked hidden processing blocks, and an output classification layer.

[0072] Input layer: 10-dimensional standardized feature vectors;

[0073] Three hidden blocks (512 → 128 → 64 dimensions respectively): each block contains a fully connected layer, batch normalization, ReLU activation, and Dropout (dropout rate 0.2);

[0074] Output layer: Contains only a single linear transformation layer, mapping the 64-dimensional high-order features to the final 4-dimensional class space, outputting unnormalized log probabilities, and finally using the maximum index selection method to obtain the optimal variant index v. q ∗∈{0,1,2,3}.

[0075] The specific design of the hidden processing block: To alleviate gradient instability and suppress overfitting during deep network training, this invention employs a modular cascaded structure design in each hidden layer. Specifically, each hidden processing block sequentially contains the following four units:

[0076] Fully connected unit: As the core of feature transformation, it is responsible for using linear transformations to map the input vector to a specific hidden space dimension.

[0077] Batch normalization unit: placed after the fully connected layer and before the activation function, it is used to normalize the output of the fully connected unit and stabilize the feature distribution.

[0078] Nonlinear activation unit: The ReLU (Rectified Linear Unit) function is used as the activation function.

[0079] ,

[0080] in This is the received feature scalar. This function enhances the model's ability to fit complex patterns by introducing non-linear expressive power by setting negative values ​​to zero.

[0081] Dropout Layer: Located after the activation function, it randomly disconnects neuronal connections during training (dropout rate set to 0.2) to enhance the robustness of the model.

[0082] Step 25, Model Training and Deployment

[0083] This invention employs an adaptive moment estimation (Adam) optimizer to perform end-to-end training of the multilayer perceptron element selector and achieves efficient deployment. The specific implementation includes the following sub-steps:

[0084] Step 251, Optimizer Configuration and Parameter Update Mechanism

[0085] The Adam optimizer is used to perform parameter updates. This optimizer combines the advantages of the momentum method and the RMSProp algorithm, and can dynamically adjust the learning rate of each parameter based on the first and second moment estimates of the gradient. It is particularly suitable for handling non-stationary objective functions containing noise or sparse gradients. Specifically, in the... In the next iteration step, for the objective function Regarding parameters gradient The execution logic of the Adam optimizer includes the following mathematical operations:

[0086] (1) Gradient calculation: Calculate the gradient based on the current batch of data in the parameters. gradient at:

[0087]

[0088] (2) Moment estimation update: Update the first moment vector of the gradient (i.e., the exponential moving average of the gradient) and the second moment vector (i.e., the exponential moving average of the squared gradient).

[0089]

[0090] in: This is a first-order moment estimate; It is a second-order moment estimate; The optimal setting is for the exponential decay rate hyperparameter. Close to 0.9 Close to 0.999; This represents the element-wise square of the gradient. This gives the optimizer "inertia," enabling it to smooth gradient oscillations.

[0091] (3) Deviation correction:

[0092]

[0093] (4) Parameter Update: The system calculates the parameter update step size based on the corrected moment estimates and updates the model parameters.

[0094] ,

[0095] in, The global learning rate can be set to 1e-4; This is the numerical stability constant, usually set to 1. To prevent division by zero errors.

[0096] Step 252, Batch Processing and Hyperparameter Settings

[0097] Batch Size: To balance memory consumption and computational efficiency, the batch size can be configured as an integer between 16 and 8192 (typically a power of 2). Smaller batches help escape local minima, while larger batches help leverage the parallel advantages of matrix operations. Considering the scale of the training data and the hardware's parallel computing capabilities, this invention specifically sets the batch size to 2048. This setting achieves an optimal balance between ensuring the statistical stability of gradients and training throughput.

[0098] Learning rate: Fixed initial learning rate No decay strategy is required, and it has been proven to effectively avoid training oscillations and accelerate convergence.

[0099] Training epochs: set to 40 epochs. Training logs indicate that the model reached performance saturation around epoch 40, and further training did not bring significant performance improvements; therefore, training was terminated to avoid overfitting.

[0100] Step 253, Model Deployment and Inference

[0101] After training, the MLP meta-selector is integrated into the front end of the retrieval system to perform online inference: for any new query q, its 10-dimensional feature vector is extracted in real time, and after the same standardization process as in the training phase, it is input into the model; the model outputs 4-dimensional logits, and the optimal BM25 variant index v is determined by taking the maximum value index. q ∗ ∈{0,1,2,3}.

[0102] Inference latency is less than 0.1 milliseconds, meeting the requirements of high-concurrency real-time retrieval; it supports cross-dataset generalization capability (zero-shot to new domains); and it can achieve interpretable decisions through SHAP or feature importance analysis, for example: "When the query length is long and the average IDF value is high, the system tends to select BM25+ (variant 2)".

[0103] Further, step 3, the adaptive BM25 recall phase;

[0104] Based on the variant index v output by the meta-selector q * The system loads the corresponding BM25 variant. For public datasets, after using a meta-learning selector to output the variant indexes for each query in the dataset, the optimal variant index for the dataset is determined based on the majority rule. Efficient retrieval is performed on the inverted index of hundreds of millions of documents, returning a Top-K (e.g., K=100 or 1000) candidate document set, Dcand.

[0105] This stage ensures a high-quality initial candidate set and avoids missing relevant documents due to fixed BM25.

[0106] Step 4, Cross-Encoder fine-tuning and reordering stage;

[0107] Prepare the input format: For each candidate document d(i) (e.g., the 1st, 2nd, ..., 100th), concatenate it with the query q to form a single sequence, in the following format:

[0108] "[CLS] q [SEP] d(i) [SEP]",

[0109] [CLS] and [SEP] are special markers that tell the model where the beginning, the end of the question, and the end of the document are.

[0110] Feeding into the Cross-Encoder model: The concatenated text is fed into a pre-trained deep neural network model, the Cross-Encoder model (such as a BERT-based binary classification or regression model). This model has a key capability: to allow each word in the query to "communicate" with each word in the document (through self-attention and cross-attention mechanisms).

[0111] Calculate the relevance score: After processing the input, the model outputs a fine-grained semantic matching score, denoted as .

[0112] s ce (q, d(i))=ModelCE(q,d(i)),

[0113] This score is usually a real number (e.g., 0.92, 0.35), with higher values ​​indicating greater relevance. It is calculated using the vector at the [CLS] position within the model, followed by a small classification layer (e.g., a linear layer + sigmoid). This score comprehensively considers word order, contextual interaction (e.g., does "apple" refer to fruit or a mobile phone?), and deep semantic matching (it can determine relevance even without identical words), significantly outperforming vector inner product models. Reordering: Repeat the above process for all k candidate documents d(1), d(2), ..., d(k) to obtain k refined scores:

[0114] s ce (q, d(1)), s ce (q, d(2)), ..., s ce (q, d(k)).

[0115] Then, sort the documents in descending order of score, placing the most relevant documents at the top. Output the Top-N (e.g., top-10 or all 100) highly relevant documents as the search results.

[0116] The present invention also discloses a computer device, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the method of the present invention.

[0117] The present invention also discloses a computer-readable storage medium having a computer program / instructions stored thereon, which, when executed by a processor, implements the steps of the method of the present invention.

[0118] Beneficial effects: Compared with the prior art, the present invention has the following significant advantages:

[0119] This invention introduces a lightweight and efficient meta-learning mechanism to achieve adaptive selection of BM25 variants on a "one-query-one-policy" basis. This effectively solves the problem of limited recall quality caused by fixed BM25 parameters in traditional systems, significantly improving recall rates in diverse scenarios such as technical terms, colloquial expressions, and short-tail and long-tail queries. The generated high-quality initial candidate set effectively breaks through the recall bottleneck in the two-stage retrieval architecture, providing better input for subsequent Cross-Encoder re-ranking and avoiding performance ceilings caused by initial screening omissions. Overall, key indicators such as MRR@10 and Recall@100 are significantly improved. At the same time, this method only applies to the recall stage, without modifying the existing re-ranking architecture, and can be seamlessly integrated into industrial-grade "BM25 + Cross-Encoder" pipelines, with low deployment costs and strong compatibility. The meta-learning classifier has a simple structure and an inference latency of less than 1ms, without affecting system throughput. In addition, the framework design is open, supporting flexible expansion of any new BM25 variants. Future new and improved versions can be directly included in the selection pool without system reconstruction. Attached Figure Description

[0120] Figure 1 This is a flowchart of the present invention.

[0121] Figure 2 This is a schematic diagram of a meta-learning selector. Detailed Implementation

[0122] The technical solution of the present invention will be further described below with reference to the accompanying drawings.

[0123] like Figure 1 As shown, the document vector similarity calculation method based on meta-learning adaptive selection of BM25 variants according to the present invention includes the following steps:

[0124] Step 1: Extract the query meta-features of the document;

[0125] Step 2: Design a query-based meta-learning selector to select BM25 variants. Based on the selected variants, calculate the evaluation index score, construct a training sample set according to the score, construct a multilayer perceptron (MLP) meta-selector model, and train the model using the training sample set.

[0126] Step 3: Recall the adaptive BM25;

[0127] Step 4: Sort the documents according to relevance based on the Cross-Encoder model, and output the highly relevant documents as the search results.

[0128] Example

[0129] To verify the effectiveness and generalization ability of the proposed method, this embodiment uses multiple publicly available datasets from the BEIR (Benchmarking IR) benchmark as experimental data sources. These datasets cover a wide range of retrieval task scenarios, including question answering, fact-checking, biomedical information retrieval, and news retrieval, specifically including: ANTIQUE (non-factual question answering), ARGUANA (debate retrieval), CLIMATE-FEVER (climate fact-checking), CQADUPSTACK (technical question answering), DBPEDIA-ENTITY (entity retrieval), FEVER (fact-checking), FIQA (financial question answering), HOTPOTQA (multi-hop question answering), MSMARCO (large-scale reading comprehension), NFCORPUS (medical question answering), NQ (nature question answering), QUORA (duplicate question detection), SCIDOCS (citation recommendation), SCIFACT (scientific fact-checking), and TREC-COVID (COVID-19 literature retrieval).

[0130] These datasets possess diverse query lengths, document lengths, and semantic features, enabling a comprehensive evaluation of the adaptability of the meta-learning selector in this invention under different data distributions. In the experiments, a stratified sampling strategy was employed to partition the datasets, with the ratio of training to test sets set to 8:2 according to the standards of each dataset, ensuring the fairness of the evaluation results.

[0131] Results Analysis

[0132] To comprehensively evaluate the effectiveness of the method proposed in this invention, a variety of ranking-based retrieval evaluation metrics (such as Recall@K) were used to systematically compare and evaluate the vector similarity ranking results obtained in the embodiments with other comparison methods.

[0133] (1) Explanation of evaluation indicators

[0134] Recall (Recall@K): Recall measures the ability of a retrieval system to "search all" of items, i.e., the number of items retrieved. The percentage of results containing relevant documents. For a single query. The calculation formula is as follows:

[0135]

[0136] in, Indicates the preceding The number of relevant documents in each result (False Negatives) represent the number of documents that are actually relevant but were incorrectly judged as irrelevant by the system (i.e., not recalled).

[0137] Normalized Discounted Cumulative Gain (NDCG@K): NDCG comprehensively considers the relevance of documents and their ranking position; the earlier the document appears, the greater its weight, making it suitable for evaluating ranking quality. For a single query... The calculation formula is as follows:

[0138]

[0139] Among them, the cumulative gain of loss , For the first The relevance level of each document; This represents the maximum DCG value under ideal conditions.

[0140] Precision (P@K): Precision measures the accuracy of search results, i.e., the accuracy of the first few searches. The percentage of relevant documents in each result. For a single query. The calculation formula is as follows:

[0141]

[0142] Average Precision (AP@K): AP reflects the stability of the system's ranking accuracy on a single query. For a single query... The calculation formula is as follows:

[0143]

[0144] in It is up to the number The accuracy of the bit, It is a binary correlation (1 or 0).

[0145] Dataset-level metric calculation instructions (macro average): The above evaluation metric calculation formulas are all for a single query. Evaluation calculations. In the statistical analysis of the experimental results in this embodiment, in order to measure the overall performance of the method on the entire dataset, all reported metric values ​​(Recall, NDCG, P, MAP) are the macro-average of all test queries.

[0146] Specifically, the system first calculates the aforementioned metrics individually for each query in the test set, then performs an arithmetic average of the metric scores for all queries to obtain the final evaluation result for that dataset. (Based on metrics) For example, the formula for calculating the macro average of its dataset is uniformly expressed as:

[0147]

[0148] in For querying sets, To query the total number.

[0149] (2) Experimental results and comparative analysis

[0150] This embodiment compares the proposed meta-learning adaptive method with four standard BM25 variants. The four benchmark variants are: variant 0 (Lucene BM25), variant 1 (BM25L), variant 2 (BM25+), and variant 3 (Log-log BM25).

[0151] Table 1. Experimental results of Top-100 recall rate

[0152]

[0153] As shown in Table 1, a single variant fails to outperform on all datasets. For example, variant 2 performs best on MSMARCO (0.46225), but is significantly weaker than variant 3 (0.7884) on HOTPOTQA (0.76313). The meta-learning method of this invention, by analyzing query features, adaptively achieves the highest recall rates on datasets such as ANTIQUE, ARGUANA, HOTPOTQA, and MSMARCO, demonstrating the effectiveness of its strategy.

[0154] Table 2 Experimental results of Top-100 NDCG

[0155]

[0156] The NDCG results further confirm the advantages of this invention in optimizing ranking positions. The data shows that different datasets have drastically different requirements for the ranking function: the FEVER dataset prefers variant 3 (0.8099), while the MSMARCO dataset prefers variant 2 (0.5697). The meta-learning method proposed in this invention is not limited to a single strategy, achieving 0.8099 on FEVER and 0.5697 on MSMARCO, both on par with the best individual performance on these datasets. This demonstrates that the meta-learning selector can effectively improve the ranking of highly relevant documents in the returned list.

[0157] Table 3 Experimental results of Top-100 P

[0158]

[0159] Precision data demonstrate that this invention effectively improves the relevance density of the results list. In the FIQA dataset, variant 1 (0.01306) slightly outperforms other variants, and the method of this invention captures this subtle advantage, also achieving a top value of 0.01306. In TREC-COVID, for the poorly performing variant 3 (0.4878), the method of this invention intelligently avoids it, maintaining a high precision of 0.5084, consistent with the best variant 0. This further proves the necessity and effectiveness of achieving "one-stop service" through meta-learning.

[0160] Table 4 Experimental results of Top-100 MAP

[0161]

[0162] The MAP metric reflects the robustness of the system. The meta-learning method demonstrates excellent adaptability and robustness on multiple information retrieval benchmark datasets. As shown in Table 4, in tasks sensitive to semantic understanding, multi-hop inference, or query sparsity, such as MS MARCO and HOTPOTQA, this method can automatically select the optimal variant, achieving Top-100 MAP performances of 0.37374 and 0.6298, respectively, significantly outperforming the fixed-configuration baseline model. Simultaneously, it can stably match the optimal or near-optimal variant on most other datasets without performance degradation. Although the performance of each variant is similar in some tasks such as CQADUPSTACK, DBPEDIA-ENTITY, CLIMATE-FEVER, and ARGUANA, meta-learning, while not bringing significant gains, still maintains stable performance on par with the optimal variant, without performance degradation. This fully verifies that the invention possesses good generalization ability and stability under different domains and task characteristics, effectively solving the performance fluctuation problem caused by fixed strategies in traditional retrieval models.

Claims

1. A document vector similarity calculation method based on meta-learning adaptive selection of BM25 variants, characterized in that, Includes the following steps: Step 1: Extract the query meta-features of the document; Step 2: Design a query-based meta-learning selector to select BM25 variants. Based on the selected variants, calculate the evaluation index score, construct a training sample set according to the score, construct a multilayer perceptron (MLP) meta-selector model, and train the model using the training sample set. Step 3: Recall the adaptive BM25; Step 4: Sort the documents according to relevance based on the Cross-Encoder model, and output the highly relevant documents as the search results.

2. The document vector similarity calculation method based on meta-learning adaptive selection of BM25 variants according to claim 1, characterized in that, Step 1 is as follows: For any input query q, the system first extracts a 10-dimensional lightweight meta-feature vector m. q ∈R 10 This is used to characterize the semantic complexity and term distribution characteristics of a query; The features are divided into two categories: The first category, querying self-features, includes: q_len: Query the number of tokens after word segmentation; unique_term_ratio: The percentage of unique terms; has_question_word: Whether the question word is present; 1 if yes, 0 otherwise. contains_digit: Whether the data contains a digit; 1 if it does, 0 otherwise. avg_word_length: The average length of each word in a single query; The second category is query-corpus interaction features: Assume the system has already built an inverted index and stored the document frequency (df) of terms. t df t The number of documents containing the word t, including features: avg_idf: The average IDF of each term in the query. , where t represents a specific term and N represents the total number of documents in the corpus; max_idf: The maximum IDF value of each word in the query; idf_std: The standard deviation of the IDF for each term in the query; rare_term_count: the number of rare terms (df) t <10; common_term_ratio: the proportion of high-frequency terms (df) t >10,000; All features rely solely on the query text and existing indexes, requiring no additional annotations or model inference.

3. The document vector similarity calculation method based on meta-learning adaptive selection of BM25 variants according to claim 1, characterized in that, In step 2, the design of the query-based meta-learning selector for BM25 variant selection specifically involves: predefining the following four BM25 variants as candidate strategies, each of which makes targeted modifications to the term frequency or length normalization term based on the standard BM25; General symbol explanation: d: document to be scored; q = { t | t is the term appearing in the query}: set of terms in the query; df t : The number of documents containing the word t; tf td : The number of times word t appears in document d; L d : The length of document d; L avg : Average length of all documents in the corpus; N: Total number of documents in the corpus; k1>0: Hyperparameter controlling TF saturation; b∈[0,1]: Hyperparameter controlling length normalization intensity; rsv q (d): Query the similarity score of documents under q. The larger the value, the more relevant they are. The variants are defined as follows: Variant 0-Lucene BM25: The scoring function is: ； Variant 1-BM25L: To address the issue of low-frequency words being underestimated in short documents, a smoothing bias δ ​​is added before the TF item to enhance the weight of low-frequency words. The scoring function is as follows: ； in, ; Variant 2-BM25+: To address the issue of excessive penalty for high-frequency words in the original BM25 method in long documents, a constant term is added to the denominator. To mitigate the bias at the upper limit of TF, its scoring function is: ； Variant 3-Log-log BM25: Its scoring function is: ； The four variants mentioned above each have their advantages in different query types, including short queries vs. long queries, factual queries vs. multi-hop inference queries.

4. The document vector similarity calculation method based on meta-learning adaptive selection of BM25 variants according to claim 3, characterized in that, In step 2, the calculation of evaluation metric scores based on the selected variants and the construction of a training sample set based on the scores specifically involves performing the following operations on multiple publicly available retrieval datasets, including MS MARCO, HotpotQA, or Quora: 1) For each query q, document retrieval was performed using four different BM25 variants; 2) For each variant, calculate its score under preset evaluation metrics including Recall@100 or NDCG@10; 3) Select the variant index with the highest score as the true label y for this query. q ∈{0,1,2,3}; 4) Extract the 10-dimensional feature vector m corresponding to the query. q This includes query length, average IDF, maximum TF, number of unique terms, and IDF variance; 5) Construct the training sample set , where N ≥ 50000.

5. The document vector similarity calculation method based on meta-learning adaptive selection of BM25 variants according to claim 1, characterized in that, In step 2, the construction of the multilayer perceptron (MLP) meta-selector model specifically involves designing a multilayer perceptron (MLP) classifier with a funnel-shaped hierarchical structure as the meta-selector, whose input is the aforementioned 10-dimensional feature vector m. q The output is the optimal variant index v. q ∗∈{0,1,2,3}; The MLP deep neural network consists of an input interface, three stacked hidden processing blocks, and an output classification layer; Input layer: 10-dimensional standardized feature vectors; The three hidden blocks are 512 → 128 → 64 dimensions respectively: each block contains a fully connected layer, batch normalization, ReLU activation, and Dropout; Output layer: Contains only a single linear transformation layer, mapping the 64-dimensional high-order features to the final 4-dimensional class space, outputting unnormalized log probabilities, and finally using the maximum index selection method to obtain the optimal variant index v. q *∈{0,1,2,3}; Each hidden processing block contains the following four units in sequence: Fully connected unit: As the core of feature transformation, it is responsible for using linear transformations to map the input vector to a specific dimension of the hidden space; Batch normalization unit: placed after the fully connected layer and before the activation function, it is used to normalize the output of the fully connected unit and stabilize the feature distribution. Nonlinear activation unit: The ReLU (Rectified Linear Unit) function is used as the activation function. ； in It is the received feature scalar; Dropout Layer: Located after the activation function, it randomly disconnects neuronal connections during training to enhance the robustness of the model.

6. The document vector similarity calculation method based on meta-learning adaptive selection of BM25 variants according to claim 1, characterized in that, In step 2, training the model using the training sample set specifically involves: using the Adam optimizer to perform parameter updates, and in the... In the next iteration step, for the objective function Regarding parameters gradient The execution logic of the Adam optimizer includes the following mathematical operations: Gradient calculation: Calculates the gradient based on the current batch of data in the parameters. gradient at: ； Moment estimation update: Update the first and second moment vectors of the gradient; ； in: This is a first-order moment estimate; It is a second-order moment estimate; The optimal setting is for the exponential decay rate hyperparameter. Close to 0.9 Close to 0.999; This represents the element-wise square of the gradient; Deviation correction: ； Parameter Update: The system calculates the parameter update step size based on the corrected moment estimates and updates the model parameters accordingly. ； in, The global learning rate; is the numerical stability constant.

7. The document vector similarity calculation method based on meta-learning adaptive selection of BM25 variants according to claim 1, characterized in that, Step 3 specifically involves: based on the variant index v output by the meta-selector. q * The system loads the corresponding BM25 variant. For public datasets, it uses a meta-learning selector to output the variant indexes of each query in the dataset. Then, it determines the best variant index for the dataset based on the majority rule and performs efficient retrieval on the inverted index of hundreds of millions of documents, returning the Top-K candidate document set Dcand.

8. The document vector similarity calculation method based on meta-learning adaptive selection of BM25 variants according to claim 1, characterized in that, Step 4 specifically involves preparing the input format: For each candidate document d(i), concatenate it with the query q to form a single sequence, as follows: [CLS] q [SEP] d(i) [SEP]; [CLS] and [SEP] are special markers used to indicate the start of the model, the end of the question, and the end of the document; Feeding into the Cross-Encoder model: The concatenated text is input into the Cross-Encoder model; Calculate the relevance score: After processing the input, the model outputs a fine-grained semantic matching score, denoted as . s ce (q, d(i))=ModelCE(q,d(i)); The score is a real number, and the larger the value, the more relevant it is; it is calculated by passing the vector at the [CLS] position inside the model through a small classification layer. Reordering: Repeat the above process for all k candidate documents d(1), d(2), ..., d(k) to obtain k refined scores: s ce (q, d(1)),s ce (q, d(2)),……,s ce (q, d(k)); Sort the documents in descending order of their scores, placing the most relevant documents at the top, and output the Top-N highly relevant documents as the search results.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory, characterized in that, The processor executes the computer program to implement the steps of the method of claim 1.

10. A computer-readable storage medium having a computer program / instructions stored thereon, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the method of claim 1.

Images