Document retrieval method and device, electronic equipment, storage medium and product

Abstract

The invention discloses a document retrieval method and device, electronic equipment, a storage medium and a product. The method comprises the steps of obtaining a retrieval question input by a user; determining an original similarity score of each document in the to-be-retrieved document set and the retrieval problem based on a preset BM25 algorithm; determining a first normalization factor according to the document length of each document and a preset normalization parameter; the first normalization factor is used for eliminating the influence of the document length on the original similarity score; performing normalization processing on the original similarity score according to a first normalization factor to obtain a first similarity score; and generating a target retrieval result based on the first similarity score corresponding to each document. By adopting the document retrieval method provided by the invention, the influence of the length difference of the retrieval documents on the similarity score output by the traditional BM25 algorithm can be eliminated, and the comparability of the similarity score of the retrieval result in a given retrieval problem is realized, so that the accuracy and the reliability of the retrieval result are improved.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a document retrieval method, device, electronic device, storage medium and product. Background Art

[0002] The BM25 algorithm, currently the most commonly used sparse search method, has been widely adopted by various full-text search systems due to its excellent performance. However, the traditional BM25 algorithm does not consider the impact of the length of the search result documents, resulting in the calculated similarity score being inconsistent with the actual situation, leading to less than ideal document retrieval results. Summary of the Invention

[0003] The present invention provides a document retrieval method, apparatus, electronic device, storage medium, and product, which achieve comparability of similarity scores of retrieval results within a given retrieval question by eliminating the impact of differences in retrieved document lengths on the similarity scores output by the traditional BM25 algorithm, thereby improving the accuracy and reliability of retrieval results.

[0004] According to one aspect of the present invention, a document retrieval method is provided, the method comprising:

[0005] Get the search question entered by the user;

[0006] Determine the original similarity score between each document in the document set to be retrieved and the retrieval question based on the preset BM25 algorithm;

[0007] Determining a first normalization factor based on the document length of each document and a preset normalization parameter; the first normalization factor is used to eliminate the influence of the document length on the original similarity score;

[0008] Normalizing the original similarity score according to a first normalization factor to obtain a first similarity score;

[0009] Generate target retrieval results based on the first similarity scores corresponding to the documents.

[0010] According to another aspect of the present invention, there is provided a document retrieval device, the device comprising:

[0011] Question acquisition module, used to obtain the search question input by the user;

[0012] An original similarity score determination module is used to determine the original similarity score between each document in the document set to be retrieved and the retrieval question based on a preset BM25 algorithm;

[0013] a first normalization factor determination module, configured to determine a first normalization factor based on the document length of each document and a preset normalization parameter; the first normalization factor is configured to eliminate the influence of the document length on the original similarity score;

[0014] a first similarity score determination module, configured to normalize the original similarity score according to a first normalization factor to obtain a first similarity score;

[0015] The retrieval result generating module is used to generate a target retrieval result based on the first similarity score corresponding to each document.

[0016] According to another aspect of the present invention, an electronic device is provided, comprising:

[0017] at least one processor; and

[0018] a memory communicatively connected to the at least one processor; wherein,

[0019] The memory stores a computer program executable by the at least one processor. The computer program is executed by the at least one processor so that the at least one processor can execute the document retrieval method according to any embodiment of the present invention.

[0020] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the document retrieval method according to any embodiment of the present invention when executed.

[0021] According to another aspect of the present invention, a computer program product is provided. The computer program product includes a computer program. When the computer program is executed by a processor, the document retrieval method according to any embodiment of the present invention is implemented.

[0022] The technical solution of the embodiment of the present invention obtains the search question input by the user; determines the original similarity score of each document in the document set to be retrieved and the search question based on the preset BM25 algorithm; determines a first normalization factor according to the document length of each document and the preset normalization parameter; the first normalization factor is used to eliminate the influence of the document length on the original similarity score; normalizes the original similarity score according to the first normalization factor to obtain a first similarity score; and generates a target search result based on the first similarity score corresponding to each document. This technical solution determines the first normalization factor according to the document length of each document and the preset normalization parameter, and normalizes the original similarity score obtained by the BM25 algorithm according to the first normalization factor, and then generates a target search result based on the normalized first similarity score, thereby eliminating the influence of the difference in the length of the retrieved documents on the similarity score output by the traditional BM25 algorithm, achieving comparability of the similarity scores of the retrieval results within a given search question, thereby improving the accuracy and reliability of the retrieval results.

[0023] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present invention, nor is it intended to limit the scope of the present invention. Other features of the present invention will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0025] Figure 1 This is a flowchart of a document retrieval method provided in accordance with the first embodiment of the present invention;

[0026] Figure 2 This is a flowchart of a document retrieval method provided according to the second embodiment of the present invention;

[0027] Figure 3 This is a schematic diagram of the structure of a document retrieval device provided according to a third embodiment of the present invention;

[0028] Figure 4 The figure is a schematic diagram of the structure of an electronic device for implementing the document retrieval method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0029] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0030] It should be noted that the terms "first", "second", etc. in the description and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0031] Example 1

[0032] Figure 1 This is a flowchart of a document retrieval method provided in the first embodiment of the present invention. This embodiment can be applied to eliminate the influence of the difference in the length of the retrieved documents on the similarity score output by the traditional BM25 algorithm, achieve the comparability of the similarity score of the retrieval results within a given retrieval question, and thus improve the accuracy and reliability of the retrieval results. This method can be executed by a document retrieval device, which can be implemented in the form of hardware and / or software, and can be configured in an electronic device. Figure 1 As shown, the document retrieval method provided in this embodiment includes the following steps:

[0033] S110: Obtain the search question input by the user.

[0034] The search question may refer to keywords, phrases, or natural language sentences input by the user to find relevant documents.

[0035] In an embodiment of the present invention, the document retrieval device may receive one or more search questions input by a user, and then retrieve corresponding search documents from a set of documents to be retrieved. In a specific embodiment, the document retrieval device may obtain search data submitted by the user through a front-end search interface (e.g., an input box, voice input, etc.), and after cleaning and verifying the search data, obtain the search question desired by the user.

[0036] S120 , determining the original similarity score between each document in the document set to be retrieved and the retrieval question based on a preset BM25 algorithm.

[0037] Among them, the preset BM25 algorithm may refer to the traditional BM25 algorithm, which is a text similarity calculation method based on a probability model, and evaluates the relevance between a document and a query (retrieval question) by combining term frequency (TF) and inverse document frequency (IDF).

[0038] The document set to be retrieved may refer to a predefined document library containing all documents to be retrieved. The raw similarity score may refer to a similarity score between the document and the retrieval question directly calculated by a preset BM25 algorithm without normalization.

[0039] In an embodiment of the present invention, after receiving the retrieval question input by the user, the document retrieval device can call a pre-configured preset BM25 algorithm to determine the similarity score corresponding to the retrieval question and each document in the document set to be retrieved. The similarity score is the original similarity score that has not been normalized.

[0040] S130 , determining a first normalization factor according to the document length of each document and a preset normalization parameter; the first normalization factor is used to eliminate the influence of the document length on the original similarity score.

[0041] The document length may refer to the number of words or characters contained in the document. The preset normalization parameter may refer to a pre-configured parameter for adjusting the sensitivity of the first normalization factor to document length. The first normalization factor may refer to a correction factor for eliminating the effect of document length on the original similarity score. This can be used to balance the score bias between long and short documents, making the score more dependent on content relevance rather than document length, thereby enhancing cross-document comparability.

[0042] In an embodiment of the present invention, the document retrieval device can obtain a pre-configured preset normalization parameter, and at the same time count the document length of each document, and then determine the corresponding first normalization factor based on the preset normalization parameter and the document length of each document. Subsequently, the original similarity score will be normalized based on the first normalization factor to eliminate the influence of the document length on the original similarity score.

[0043] S140: Normalize the original similarity score according to a first normalization factor to obtain a first similarity score.

[0044] The first similarity score may refer to the similarity score corrected by the first normalization factor.

[0045] In the embodiment of the present invention, for each document, the original similarity score corresponding to the document may be multiplied by the first normalization factor to obtain a modified first similarity score.

[0046] S150: Generate target retrieval results based on the first similarity score corresponding to each document.

[0047] The target search result may refer to a document list returned after being sorted according to the modified first similarity score.

[0048] In an embodiment of the present invention, after the original similarity score is corrected, the final target retrieval result can be determined according to the corrected first similarity score. For example, all documents can be sorted in descending order according to the first similarity score, and the top K documents (Top-K) can be returned as the final target retrieval result. Alternatively, all documents whose first similarity score is greater than a preset score threshold can be determined as the final target retrieval result, and so on. The specific output rules of the target retrieval result are not specifically limited in the embodiment of the present invention.

[0049] The technical solution of the embodiment of the present invention obtains the search question input by the user; determines the original similarity score of each document in the document set to be retrieved and the search question based on the preset BM25 algorithm; determines a first normalization factor according to the document length of each document and the preset normalization parameter; the first normalization factor is used to eliminate the influence of the document length on the original similarity score; normalizes the original similarity score according to the first normalization factor to obtain a first similarity score; and generates a target search result based on the first similarity score corresponding to each document. This technical solution determines the first normalization factor according to the document length of each document and the preset normalization parameter, and normalizes the original similarity score obtained by the BM25 algorithm according to the first normalization factor, and then generates a target search result based on the normalized first similarity score, thereby eliminating the influence of the difference in the length of the retrieved documents on the similarity score output by the traditional BM25 algorithm, achieving comparability of the similarity scores of the retrieval results within a given search question, thereby improving the accuracy and reliability of the retrieval results.

[0050] In one embodiment, the preset normalization parameters specifically include: a first preset normalization parameter t1 and a second preset normalization parameter t2. The first normalization factor S1 can be determined by the document length |D| and the first preset normalization parameter t1 and the second preset normalization parameter t2 as follows:

[0051]

[0052] Furthermore, based on the above-mentioned embodiment of the invention, the document retrieval method provided in this embodiment further includes:

[0053] determining a second normalization factor according to the question length of the retrieval question and the first normalization factor; the second normalization factor is used to eliminate the influence of the document length and the question length on the original similarity score;

[0054] Normalizing the original similarity score according to the second normalization factor to obtain a second similarity score;

[0055] A target retrieval result is generated based on the second similarity score corresponding to each document.

[0056] The question length may refer to the number of words or characters contained in the search question. The second normalization factor may refer to a correction factor for simultaneously eliminating the effects of both document length and question length on the original similarity score. This can be used to further balance the score scale when the query (i.e., search question) length varies significantly (e.g., a mixture of short and long queries). The second similarity score may refer to the similarity score corrected by the second normalization factor.

[0057] In an embodiment of the present invention, the question length |Q| of the search question may be calculated, and then the second normalization factor S2 may be determined based on the question length |Q| and the first normalization factor S1, as specifically expressed as follows:

[0058]

[0059] Next, for each document, its corresponding original similarity score may be multiplied by the second normalization factor to obtain a modified second similarity score, and a final target retrieval result is generated based on the second similarity score.

[0060] The embodiments of the present invention can improve the consistency of scores across retrieval questions by simultaneously eliminating the influence of document length and question length on the similarity scoring results of the traditional BM25 algorithm.

[0061] Furthermore, based on the above-mentioned embodiment of the invention, S150 specifically includes the following steps:

[0062] S1501: Determine a document having a first similarity score greater than a preset score threshold as a target document;

[0063] S1502: Arrange the target documents in descending order according to the first similarity score to generate a target retrieval result.

[0064] In an embodiment of the present invention, documents whose first similarity scores are greater than a preset score threshold (e.g., 0.7) can be screened out from all documents as target documents, and then all target documents are sorted in descending order according to the first similarity scores to generate corresponding target retrieval results.

[0065] Furthermore, based on the above-mentioned embodiment of the invention, the document retrieval method provided in this embodiment further includes:

[0066] Obtaining a third similarity score corresponding to each document; wherein the third similarity score is calculated by a preset retrieval algorithm based on the retrieval question and the set of documents to be retrieved, and the preset retrieval algorithm is different from the preset BM25 algorithm;

[0067] The first similarity score and the third similarity score of each document are combined according to the preset weights to generate a mixed similarity score;

[0068] Generate target retrieval results based on the hybrid similarity scores corresponding to each document.

[0069] Among them, the preset retrieval algorithm may refer to a retrieval algorithm that is different from the traditional BM25 (sparse retrieval algorithm), for example, it may include: sparse retrieval algorithms such as TF-IDF, IB (Information-Based), or dense retrieval algorithms such as DPR (Dense Passage Retrieval), SBERT (Sentence-BERT), ColBERT, etc. Preferably, the preset retrieval algorithm can select a dense retrieval algorithm. The third similarity score may refer to the similarity score between the retrieval question and the document calculated using the non-traditional BM25 algorithm. The hybrid similarity score may refer to the weighted fusion result of the first similarity score and the third similarity score, which is used to combine the advantages of different retrieval algorithms to generate more robust retrieval results.

[0070] In an embodiment of the present invention, the document retrieval device can obtain a third similarity score between the retrieval question determined based on a preset retrieval algorithm (not the preset BM25 algorithm) and each document; then, for each document D, a hybrid similarity score score can be generated according to the following score fusion formula: hybrid (D):

[0071] score hybrid (D)=α·score1(D)+(1-α)·score3(D)

[0072] Wherein, score1(D) and score3(D) represent the first similarity score and the third similarity score corresponding to document D respectively; α represents the preset weight and satisfies 0<α<1.

[0073] Next, documents with a mixed similarity score greater than a preset score threshold (eg, 0.7) may be selected from all documents as target documents, and all target documents are sorted in descending order according to the mixed similarity score to generate corresponding target retrieval results.

[0074] Furthermore, based on the above embodiments of the invention, the first preset normalization parameter and the second preset normalization parameter are determined by any one of the following methods:

[0075] Determined based on statistical characteristics of the document set to be retrieved;

[0076] The preset validation data set was used for linear regression fitting.

[0077] In the embodiment of the present invention, the first preset normalization parameter t1 and the second preset normalization parameter t2 may be determined by any of the following methods:

[0078] 1. Determination based on the statistical characteristics of the document set to be retrieved

[0079] ①. Read the document set S to be retrieved, including the text content and document length information of all documents D.

[0080] ② Extract global statistical features of the document set S to be retrieved, including: 1) average document length That is, the average value of the document length |D| of all documents D in the document set S to be retrieved; 2) the number of documents N; 3) the average number of documents containing keywords n S .

[0081] ③ Based on the Poisson process, the document length and keyword distribution are modeled and the expressions of the first preset normalization parameter t1 and the second preset normalization parameter t2 are derived as follows:

[0082]

[0083] Where k1 and b represent the hyperparameters of the traditional BM25 algorithm, and k1∈{1.2,2.0}, b=0.75; λ represents the proportional coefficient, and v represents the Poisson process parameter, both of which can be estimated through historical data or document collection statistics.

[0084] ④. Substituting the global statistical features determined in step ② into the above expression, the first preset normalization parameter t1 and the second preset normalization parameter t2 can be determined.

[0085] 2. Use the preset validation data set to perform linear regression fitting

[0086] ① Read the preset verification dataset, which contains several retrieval questions Q j and its corresponding document set D j , and also records the BM25 raw similarity score s for each document BM25 (D i ,Q j ), document length | D i | and search question length |Q j|.

[0087] ② Assume that the relationship between the normalized score and the document length and the search question length satisfies the approximate formula:

[0088]

[0089] After logarithmic linearization, the regression target is:

[0090]

[0091] ③Minimize the mean square error (MSE):

[0092]

[0093] ④ Use gradient descent or Newton method to iteratively optimize parameters t1 and t2.

[0094] It should be understood that if the document collection is stable and the keyword distribution conforms to the Poisson assumption, the statistical feature method can be selected to determine the parameters t1 and t2; if the document is dynamically updated or the data distribution is complex, the regression fitting method can be used to determine the parameters t1 and t2, and the parameters can be updated regularly through incremental learning.

[0095] Example 2

[0096] Figure 2 This is a flowchart of a document retrieval method provided in Example 2 of the present invention. Based on the above examples, this example provides an implementation of a document retrieval method that can normalize the similarity score output by the traditional BM25 algorithm and integrate it with the similarity score of the dense algorithm, thereby achieving hybrid retrieval. Figure 2 As shown, a document retrieval method provided in the second embodiment of the present invention specifically includes the following steps:

[0097] S210: Obtain the search question input by the user.

[0098] S220: Call the preset BM25 algorithm to determine the original similarity score between each document in the document set to be retrieved and the retrieval question.

[0099] S230: Determine a first preset normalization parameter and a second preset normalization parameter.

[0100] The first preset normalization parameter t1 and the second preset normalization parameter t2 can be determined by using a statistical feature method or a regression fitting method.

[0101] S240. Determine a first normalization factor based on the document length of each document, a first preset normalization parameter, and a second preset normalization parameter, or determine a second normalization factor based on the document length of each document, the question length of the retrieval question, the first preset normalization parameter, and the second preset normalization parameter.

[0102] The first normalization factor S1 and the second normalization factor S2 are respectively expressed as follows:

[0103]

[0104] S250 . For each document, normalize the original similarity score using a first normalization factor to obtain a first similarity score, or normalize the original similarity score using a second normalization factor to obtain a second similarity score.

[0105] The first similarity score score1 and the second similarity score score2 are respectively expressed as follows:

[0106] score1=s BM25 S1, score2 = s BM25 S2

[0107] S260: Obtain a third similarity score between the preset dense search algorithm and the search question and each document.

[0108] S270 : Fusing the first similarity score or the second similarity score of each document with the corresponding third similarity score according to a preset weight to generate a mixed similarity score.

[0109] S280: Generate target retrieval results based on the mixed similarity scores corresponding to each document.

[0110] To facilitate understanding by those skilled in the art, the derivation process of this technical solution is given below.

[0111] First, the similarity scoring principle and characteristics of the traditional BM25 algorithm are analyzed.

[0112] The main idea of ​​the traditional BM25 algorithm can be briefly described as follows: First, the query question (retrieval question) Q is decomposed to generate several feature items (keywords) q i ; Then for each document D in the document set to be retrieved, calculate each keyword q i The similarity score between the query Q and the document D is obtained by weighted summing of these similarity scores.

[0113] Specifically, given a retrieval question Q, which contains keywords q1, q2, ..., q n, given a document D in the document set S to be retrieved, the similarity score calculation formula of the BM25 algorithm is as follows:

[0114]

[0115] Where, f(q i ,D) is the keyword q in document D i The number of occurrences, |D| represents the length of document D, represents the average length of all documents in the document set S to be retrieved. k1 and b are both hyperparameters. IDF(q i ) is the keyword q i The IDF (Inverse Document Frequency) weight is expressed as follows:

[0116]

[0117] Where N is the total number of documents contained in the document set S to be retrieved, n(q i ) is the document set S to be retrieved containing the keyword q i The number of documents.

[0118] Although the BM25 algorithm has long been proven to be an excellent classic sparse search method, its calculated similarity score (D, Q) has at least the following two shortcomings, which can be expressed as two propositions:

[0119] Proposition 1: The BM25 similarity scores calculated for any two different search questions on documents in the same search question set are not comparable.

[0120] Proposition 2: The BM25 similarity score has no upper bound, making it difficult to compare with similarity scores obtained by other commonly used similarity score calculation models.

[0121] The existing mainstream similarity score normalization schemes based on the BM25 algorithm mainly include the following three:

[0122] (1) Maximum-minimum method, the calculation formula is as follows:

[0123]

[0124] In the formula, Min-Max(·) represents the normalization function, and C represents the score set of all retrieval results of the retrieval question Q on the document set S to be retrieved, that is, C = {score(q i ,D)|D∈S}, min(·) and max(·) represent the minimum value calculation function and the maximum value calculation function respectively.

[0125] (2) Standardization method, the calculation formula is as follows:

[0126]

[0127] Where μ(·) and σ(·) represent the mathematical expectation and standard deviation of all retrieval results of the retrieval problem on the document set S, respectively. In practical applications, they can be calculated using the corresponding unbiased estimation statistics.

[0128] (3) Summation method, the calculation formula is as follows

[0129]

[0130] Where sum(·) represents the summation function.

[0131] The common shortcomings of the three BM25 algorithm similarity score normalization techniques mentioned above include at least the following:

[0132] Disadvantage A: Existing normalization methods lack theoretical analysis and targeted treatment of the reasons for the non-comparability of BM25 similarity scores. The calculation formulas of the schemes are directly or indirectly "transplanted" from commonly used normalization methods in statistics. Although these methods can achieve numerical "normalization", the validity of the normalized similarity scores (whether they can still truly reflect the similarity between the search results and the search questions) and comparability (there are many methods that simply roughly compress the scores under different search questions into a unified value range. Whether the purpose behind normalization can be achieved by combining and comparing scores through arithmetic operations is the key) are both questionable. At the same time, they also lack interpretability (statistical theory can only explain the reliability of the numerical "normalization" of the results falling within a certain interval or obeying a certain distribution, but it cannot explain the rationality of calculating and comparing the normalized results between different search questions or between similarity scores of different search methods).

[0133] Disadvantage B: Existing normalization methods only utilize the statistical characteristics of different search result documents under a given search question (maximum, minimum, mean and standard deviation of the similarity scores of the search result documents, etc.) to normalize the results of the search question, but none of them utilize the statistical characteristics of the search question. This shows that the existing normalization methods implicitly believe that "the reason why the BM25 similarity score scale is not uniform and comparable between different search questions is irrelevant to the search question." The rationality of this view is obviously questionable, because: considering that the increase in the length of the search question generally leads to an increase in the number of search keywords, which in turn affects the number of search keywords n in the BM25 similarity score calculation formula (increases), and ultimately leads to the overall scale change of the similarity score score (D, Q) (most likely increases with the increase in the length of the search question)

[0134] To address the two shortcomings of similarity score normalization mentioned above, we first attempt to analyze the BM25 similarity score calculation formula to find the theoretical root cause of the non-comparability of BM25 similarity scores. Next, we use the correlation between similarity scores, search question length, and search result document length found through theoretical analysis to propose a new and targeted similarity score normalization scheme.

[0135] Prerequisites

[0136] Lemma 1: For a given keyword q i , the number of times the keyword appears in document D is f(q i ,D) can be approximately considered to be proportional to the document length |D|.

[0137] Proof: f(q i ,D) is regarded as a random variable, denoted as η i According to the definition of the Poisson process, it is reasonable to assume that there is a parameter λ i (0≤λ i ≤1, the subscript i is retained because it is not guaranteed that the parameter is the same as the search keyword q at this time i irrelevant), so that η i The parameter λ is about the document length |D| i Poisson process, that is, when the length of document D (here in units of words) grows from s to |D|+s, keyword q i The probability of increasing the number of n follows the following distribution:

[0138]

[0139] In particular, when s=0 in the formula, that is, the document length is exactly equal to |D|, since η i (0) = 0 (obviously, a document with a length of 0 cannot contain any search keywords), η i (|D|) is a random variable with the following Poisson distribution:

[0140]

[0141] According to the properties of Poisson distribution, η i The mathematical expectation of (|D|) is λ i |D|, according to the definition of mathematical expectation, this conclusion means that the random variable f(q i ,D) takes values ​​around λ i |D| fluctuates. Therefore, when the variance is within the acceptable range of actual application scenarios, it can be approximately considered that:

[0142] f(q,D)∝|D|

[0143] However, it should be noted that the proportional coefficient λ in the direct proportional relationship described by this formula i It is not guaranteed to be irrelevant to the retrieval question q. Therefore, the following Lemma 2 is required.

[0144] Lemma 2: If we assume that for different search keywords q i , the number of times the keyword appears in document D f(q i ,D) is considered as a random variable, and for any given q i All obey the same distribution, then f(q i ,D) can be approximately considered to be proportional to the document length |D|, and the proportional coefficient is proportional to the search keyword q i Not relevant.

[0145] Proof: f(q i ,D) is denoted as η i , similar to the proof of Lemma 1, it is easy to prove that for any q i , both have η i The mathematical expectation of (|D|) is λ|D|. Therefore, it can be approximately considered that f(q i ,D)∝|D|, and the proportional coefficient is the same as the search keyword q i It doesn’t matter, that is λ.

[0146] Lemma 3: If we assume that for different retrieval questions Q j , the total number of search keywords n contained in j (0≤n j ≤N) is considered as a random variable, and for any given Q j All obey the same distribution, then n j It can be approximately considered to be proportional to the length of the retrieval question |Q j |, and the proportionality coefficient is related to the retrieval problem Q j Not relevant.

[0147] Proof: The proof is similar to Lemma 1 and Lemma 2. Here we briefly describe the idea as follows:

[0148] (1) Reasonable assumption n j is a Poisson process with parameter v (0≤v≤1);

[0149] (2) For a given retrieval question length |Q j |,n j Degenerates into a parameter v|Q j | is a Poisson distribution, so its mean is v|Q j |, so the approximate direct proportional relationship is proved;

[0150] (3) Due to the distribution and Qj So the Poisson process parameter v is independent of Q j It is irrelevant, so we can prove that the proportional coefficient in (2) is also related to Q j Nope, end of proof.

[0151] The search keyword q i The number of times f(q i ,D),Search keyword q i The number of documents n(q i ) and the retrieval question Q j The total number of search keywords N contained in j All are considered as random variables, and the following reasonable assumptions are given as premises (the rationality of the assumptions will be analyzed later):

[0152] Assumption 1: For different i, the random variable f(q i ,D) follow the same distribution.

[0153] Assumption 2: For different i, the random variable n(q i ) obeys the same distribution (you can assume a normal distribution with a mean greater than 0 or a Gamma distribution with appropriate parameters. Next, we will see about n(q i The assumption of the distribution does not affect the following derivation, as long as its expectation exists), and its mean is n S .

[0154] Assumption 3: For different j, the random variable N j obey the same distribution.

[0155] According to Assumption 1, f(q i ,D) is a random variable that is independent of i, and f(q i ,D) is simplified as η. ,Therefore, the second term in the summation sign on the right side of the similarity score calculation formula of the BM25 algorithm can be written as:

[0156]

[0157] The expression on the right side of the equal sign in the above formula is also independent of i, so the similarity score calculation formula of the BM25 algorithm can be transformed into:

[0158]

[0159]

[0160] According to Lemma 2, the formula can be approximated as:

[0161]

[0162] Due to Assumption 2, the IDF(q i ) is a quantity that is independent of i, and the random variable n(q i ) is recorded as ξ, then the above formula is further deduced to obtain:

[0163]

[0164] According to Lemma 3, if n is regarded as a random variable and its mathematical expectation is used to approximate it, the above formula can be approximated as follows:

[0165]

[0166] In this formula, and n S They are all statistical values ​​related to the document set S to be retrieved (hence the S in the subscript). Therefore, in practice, they can be pre-statistically obtained based on the training set data, so they can be regarded as constants and further simplified to:

[0167]

[0168] Where, parameters t1 and t2 are both positive numbers, and their calculation formulas are as follows:

[0169]

[0170] For the above simplified formula, the expression on the right side of the approximate equal sign can be regarded as two parts. The actual meaning and rationality of |Q| are analyzed as follows:

[0171] First, the first part is analyzed as follows:

[0172] It is not difficult to find from the expression that this part reflects the relationship between the BM25 retrieval similarity score score(D,Q) and the length of the retrieval result document |D|:

[0173] a) When the document length |D| decreases, the search similarity score (score(D,Q)) increases, and vice versa. This is consistent with objective facts (the longer the document, the more likely it is to contain more types and numbers of search keywords);

[0174] b) When the document length |D| approaches 0, the retrieval similarity score score(D,Q) will also approach a lower bound value: 0, which is consistent with objective facts (documents without content should not be similar to any retrieval questions)

[0175] c) When the document length |D| approaches infinity, the retrieval similarity score score(D,Q) does not grow infinitely, but approaches an upper bound: This is also consistent with objective facts.

[0176] In summary, the first part of the formula reflects the extent to which the BM25 similarity score is affected by the length of the retrieved document within a given query. That is, given a certain query question Q, for each retrieved document D, the BM25 similarity score is affected by the length of the retrieved document under different query keywords q. i The approximation of the upper value (note: in the derivation of the formula, the mathematical expectation is used to approximate the value of the random variable. In common distributions such as the normal distribution, the mathematical expectation represents the "overall trend" of the random variable value) is inversely proportional to the linear combination of the reciprocal of the document length |D|. The significance of this conclusion is:

[0177] ① This theoretically supports the idea that the length of the retrieved document itself may affect the similarity score of the classic BM25 method, and this effect may not be what the user expects. For example, imagine copying a document and pasting it at the end of the original. In theory, the new document will contain twice as many keywords as the original, but intuitively, we would not expect such a document (full of redundant information) to have a higher similarity to the retrieval problem than the original document.

[0178] ②Through theoretical derivation, a quantitative approximate calculation method for the impact of the retrieved document length on the BM25 similarity score is given, which provides an important basis for designing a normalization method to offset the impact based on this.

[0179] Secondly, the analysis of the second part of the expression on the right side of the approximate equal sign in the formula is relatively simple, namely: the search similarity score score(D,Q) is approximately proportional to the search question length |Q|. Its significance is:

[0180] ① Through mathematical derivation, it is revealed that when designing the normalization method for BM25 similarity score, in addition to considering the factors within the given search question, the impact of different search question lengths should also be considered. This is exactly what the three existing mainstream normalization methods mentioned above do not address;

[0181] ②Through theoretical derivation, a quantitative approximate calculation method for the impact of retrieval question length on the BM25 similarity score is given, which provides an important basis for designing a normalization method to offset the impact.

[0182] Based on the above theoretical analysis, two similarity score normalization methods are proposed below:

[0183] Solution 1: We only try to eliminate the influence of the search result document length on the BM25 similarity score scale and normalize the search result similarity score within each given search question. The calculation formula is as follows:

[0184]

[0185] Where Scale1(·) is the name of the normalization calculation function of this scheme 1.

[0186] It can be seen that the implementation of this solution 1 actually only utilizes the first part on the right side of the approximate equal sign in the above formula: an approximate estimate of the degree of influence of the length of the retrieval result document on the BM25 similarity score scale. Solution 1 normalizes the score of each retrieval result document by dividing it by a value that is only related to the document length. Compared with the three existing technical solutions introduced above, this solution 1 is similar in that none of them considers the influence of the retrieval question length on the similarity score in the normalization calculation; the difference is that this solution fully utilizes the theoretical approximate estimate of the BM25 similarity score of the retrieval result document length, rather than relying solely on a "transplanted" general statistical formula, which makes this solution more interpretable and has better expected effects.

[0187] Solution 2: We attempt to eliminate the relationship between the length of the search result document and the search question length and the BM25 similarity score, and normalize the score of each search result within each search question. The calculation formula is as follows:

[0188]

[0189] As can be seen, this solution 2 utilizes all of the above formulas: an approximate estimate of the impact of the length of the search result document and the search question length on the BM25 similarity score scale. This solution normalizes the similarity scores by first dividing the similarity score of each search result document by a value that is only related to the document length, eliminating the effect of the search document length on the results. Furthermore, it divides the score by the search question length to eliminate the effect of varying search question lengths. Compared to the three existing solutions described above, the most significant feature of this solution 2 is that it not only considers the impact of varying search result document length on the similarity score, but also the impact of varying search question lengths on the similarity score. This makes this solution more interpretable and effective.

[0190] In terms of application, there are several points to note:

[0191] 1) Solution 1 is mainly suitable for scenarios where the length of different retrieval questions does not vary much, while Solution 2 is more suitable for scenarios where the length of retrieval questions varies significantly or the length of retrieval questions is not restricted;

[0192] 2) Since the theoretical basis of Schemes 1 and 2 comes from derived formulas, in practice, it should be confirmed that the data in the application scenario meets or approximately meets the three assumptions mentioned above, otherwise the effectiveness of the scheme may deteriorate.

[0193] The above similarity score normalization scheme has at least the following beneficial effects:

[0194] ① This approach implements BM25 score similarity normalization for similarity score fusion for hybrid retrieval, supported by a solid theoretical foundation. The solution not only provides a specific similarity score normalization method, but also rigorously demonstrates the underlying principle of the method design using mathematical tools such as the Poisson process: the correlation between the BM25 similarity score and the length of both the search result document and the search question document. Compared to existing normalization schemes that solely utilize statistical features for normalization, this technical solution is clearly more targeted and interpretable in design, and is more likely to achieve better results in hybrid retrieval practice.

[0195] ② The proposed similarity score normalization method not only considers the problem of eliminating the similarity score scale scaling caused by the differences in the retrieval result documents belonging to a given retrieval question (different retrieval result document lengths), but also considers the problem of eliminating the similarity score scale scaling caused by the differences between different retrieval questions (different retrieval question lengths).

[0196] ③ It is simple and convenient to implement, has high computational efficiency, and does not require major adjustments to the basic framework of existing hybrid retrieval. The variables and hyperparameters of the method proposed in this patent are either easy to obtain (document length and retrieval question length), or can be obtained by fitting or approximate estimation on existing data sets, so there is no need to worry about computational efficiency (at least not less than the closest solution); in addition, the method of this patent only requires adjustments to the "BM25 retrieval similarity score normalization" part of the hybrid retrieval framework, and does not involve adjustments to the functions and associations of any other modules.

[0197] In summary, compared with the existing similarity score normalization method, this technical solution has a more rigorous and scientific theoretical basis. While achieving the comparability of scores after normalization, it better retains the effectiveness of the search result ranking, and at the same time achieves the comparability of the similarity scores of search results within the same search question and between different search questions. Therefore, it can better guarantee the retrieval effect after the hybrid search upgrade and transformation of the sparse full-text retrieval system based on the BM25 algorithm.

[0198] Example 3

[0199] Figure 3 This is a schematic diagram of the structure of a document retrieval device provided in the third embodiment of the present invention. Figure 3 As shown, the device includes:

[0200] The question acquisition module 31 is used to obtain the search question input by the user;

[0201] An original similarity score determination module 32 is used to determine the original similarity score between each document in the document set to be retrieved and the retrieval question based on a preset BM25 algorithm;

[0202] A first normalization factor determination module 33 is configured to determine a first normalization factor based on the document length of each document and a preset normalization parameter; the first normalization factor is configured to eliminate the influence of the document length on the original similarity score;

[0203] a first similarity score determination module 34 for normalizing the original similarity score according to a first normalization factor to obtain a first similarity score;

[0204] The first search result generating module 35 is configured to generate a target search result based on the first similarity score corresponding to each document.

[0205] Furthermore, based on the above-mentioned embodiment of the invention, the preset normalization parameters include: a first preset normalization parameter and a second preset normalization parameter, and the first normalization factor is calculated by the following formula:

[0206]

[0207] Wherein, S1 represents the first normalization factor; |D| represents the document length; t1 represents the first preset normalization parameter; and t2 represents the second preset normalization parameter.

[0208] Furthermore, based on the above-mentioned embodiment of the invention, the document retrieval device further includes:

[0209] a second normalization factor determination module, configured to determine a second normalization factor based on the question length of the retrieval question and the first normalization factor; the second normalization factor is configured to eliminate the influence of the document length and the question length on the original similarity score;

[0210] a second similarity score determination module, configured to normalize the original similarity score according to a second normalization factor to obtain a second similarity score;

[0211] A second retrieval result generating module, configured to generate a target retrieval result based on a second similarity score corresponding to each document;

[0212] The second normalization factor is calculated by the following formula:

[0213]

[0214] Where S1 represents the first normalization factor; S2 represents the second normalization factor; |Q| represents the question length.

[0215] Furthermore, based on the above embodiment of the invention, the first search result generating module includes:

[0216] a target document determining unit, configured to determine a document having a first similarity score greater than a preset score threshold as a target document;

[0217] The retrieval result generating unit is configured to arrange the target documents in descending order according to the first similarity score to generate a target retrieval result.

[0218] Furthermore, based on the above-mentioned embodiment of the invention, the document retrieval device further includes:

[0219] a third similarity score acquisition module, configured to acquire a third similarity score corresponding to each document; wherein the third similarity score is calculated by a preset retrieval algorithm based on the retrieval question and the set of documents to be retrieved, and the preset retrieval algorithm is different from the preset BM25 algorithm;

[0220] A score fusion module is used to fuse the first similarity score and the third similarity score of each document according to a preset weight to generate a mixed similarity score;

[0221] The third retrieval result generating module is used to generate target retrieval results based on the mixed similarity scores corresponding to each document.

[0222] Furthermore, based on the above embodiments of the invention, the first preset normalization parameter and the second preset normalization parameter are determined by any one of the following methods:

[0223] Determined based on statistical characteristics of the document set to be retrieved;

[0224] The preset validation data set was used for linear regression fitting.

[0225] The document retrieval device provided in the embodiment of the present invention can execute the document retrieval method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0226] Example 4

[0227] Figure 4A schematic diagram of the structure of an electronic device 40 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or claimed herein.

[0228] like Figure 4 As shown, the electronic device 40 includes at least one processor 41 and a memory, such as a read-only memory (ROM) 42, a random access memory (RAM) 43, etc., which is communicatively connected to the at least one processor 41. The memory stores a computer program that can be executed by the at least one processor, and the processor 41 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 42 or the computer program loaded from the storage unit 48 into the random access memory (RAM) 43. Various programs and data required for the operation of the electronic device 40 can also be stored in the RAM 43. The processor 41, ROM 42, and RAM 43 are connected to each other via a bus 44. An input / output (I / O) interface 45 is also connected to the bus 44.

[0229] Multiple components in the electronic device 40 are connected to the I / O interface 45, including an input unit 46, such as a keyboard, a mouse, etc.; an output unit 47, such as various types of displays, speakers, etc.; a storage unit 48, such as a magnetic disk, an optical disk, etc.; and a communication unit 49, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 49 allows the electronic device 40 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0230] Processor 41 can be any general-purpose and / or specialized processing component with processing and computing capabilities. Some examples of processor 41 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various specialized artificial intelligence (AI) computing chips, various processors that run machine learning model algorithms, a digital signal processor (DSP), and any other suitable processor, controller, microcontroller, etc. Processor 41 executes the various methods and processes described above, such as the document retrieval method.

[0231] In some embodiments, the document retrieval method can be implemented as a computer program tangibly embodied in a computer-readable storage medium, such as storage unit 48. In some embodiments, part or all of the computer program can be loaded and / or installed on electronic device 40 via ROM 42 and / or communication unit 49. When the computer program is loaded into RAM 43 and executed by processor 41, one or more steps of the document retrieval method described above can be performed. Alternatively, in other embodiments, processor 41 can be configured to perform the document retrieval method in any other suitable manner (e.g., via firmware).

[0232] Various embodiments of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system-on-chip systems (SOCs), programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0233] In some embodiments, the document retrieval method can be implemented as a computer program, which is invisibly contained in a computer program product. When executed by a processor, the computer program implements the document retrieval method of the present invention. The computer program product can be understood as a software product that primarily implements its solution through the computer program. The computer program used to implement the method of the present invention can be written in any combination of one or more programming languages. These computer programs can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when executed by the processor, the computer program implements the functions / operations specified in the flowcharts and / or block diagrams. The computer program can be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0234] In the context of the present invention, computer-readable storage media can be tangible media that can contain or store a computer program for use with an instruction execution system, device or equipment or used in combination with an instruction execution system, device or equipment. Computer-readable storage media can include but are not limited to electronic, magnetic, optical, electromagnetic, infrared or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, computer-readable storage media can be machine-readable signal media. More specific examples of machine-readable storage media can include electrical connections based on one or more lines, portable computer disks, hard disks, random access memories (RAM), read-only memories (ROM), erasable programmable read-only memories (EPROM or flash memory), optical fibers, portable compact disk read-only memories (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0235] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0236] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0237] A computing system may include clients and servers. The clients and servers are typically remote from each other and typically interact via a communication network. This client-server relationship arises through computer programs running on the respective computers, creating a client-server relationship. The server may be a cloud server, also known as a cloud computing server or cloud host. This server is a hosting product within the cloud computing service ecosystem that addresses the management difficulties and limited scalability of traditional physical hosting and VPS services.

[0238] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in the present invention can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solution of the present invention can be achieved. This is not limited herein.

[0239] The above specific embodiments do not limit the scope of protection of the present invention. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention are intended to be included within the scope of protection of the present invention.

Claims

1. A document retrieval method, characterized in that: The method comprises: Get the search question entered by the user; Determine the original similarity score between each document in the document set to be retrieved and the retrieval question based on the preset BM25 algorithm; Determining a first normalization factor according to the document length of each document and a preset normalization parameter; the first normalization factor is used to eliminate the influence of the document length on the original similarity score; Normalizing the original similarity score according to the first normalization factor to obtain a first similarity score; Generate a target retrieval result based on the first similarity score corresponding to each of the documents.

2. The method according to claim 1, characterized in that The preset normalization parameters include: a first preset normalization parameter and a second preset normalization parameter. The first normalization factor is calculated by the following formula: Wherein, S1 represents the first normalization factor; |D| represents the document length; t1 represents the first preset normalization parameter; and t2 represents the second preset normalization parameter.

3. The method according to claim 1, characterized in that Also includes: determining a second normalization factor according to the question length of the search question and the first normalization factor; The second normalization factor is used to eliminate the influence of the document length and the question length on the original similarity score; Normalizing the original similarity score according to the second normalization factor to obtain a second similarity score; generating the target retrieval result based on the second similarity score corresponding to each of the documents; The second normalization factor is calculated by the following formula: Wherein, S1 represents the first normalization factor; S2 represents the second normalization factor; |Q| represents the question length.

4. The method according to claim 1, wherein Generating a target search result based on the first similarity score corresponding to each of the documents includes: Determine the document whose first similarity score is greater than a preset score threshold as a target document; The target documents are arranged in descending order according to the first similarity score to generate the target retrieval result.

5. The method according to claim 1, wherein Also includes: Obtaining a third similarity score corresponding to each of the documents; wherein the third similarity score is calculated by a preset retrieval algorithm based on the retrieval question and the set of documents to be retrieved, and the preset retrieval algorithm is different from the preset BM25 algorithm; fusing the first similarity score and the third similarity score of each of the documents according to a preset weight to generate a hybrid similarity score; The target retrieval result is generated based on the mixed similarity score corresponding to each of the documents.

6. The method according to claim 2, characterized in that The first preset normalization parameter and the second preset normalization parameter may be determined by any one of the following methods: Determining based on statistical characteristics of the set of documents to be retrieved; The preset validation data set was used for linear regression fitting.

7. A document retrieval device, characterized in that: The device comprises: Question acquisition module, used to obtain the search question input by the user; An original similarity score determination module is used to determine the original similarity score of each document in the document set to be retrieved and the search question based on a preset BM25 algorithm; a first normalization factor determination module, configured to determine a first normalization factor according to the document length of each document and a preset normalization parameter; the first normalization factor is configured to eliminate the influence of the document length on the original similarity score; a first similarity score determination module, configured to normalize the original similarity score according to the first normalization factor to obtain a first similarity score; A retrieval result generating module is used to generate a target retrieval result based on the first similarity score corresponding to each of the documents.

8. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor. The computer program is executed by the at least one processor so that the at least one processor can execute the document retrieval method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the document retrieval method according to any one of claims 1 to 6 when executed.

10. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the computer program implements the document retrieval method according to any one of claims 1 to 6.

Images