A time-sensitive dual-link hybrid search enhancement method

Abstract

The application provides a time-sensitive dual-link hybrid retrieval enhancement method, which is characterized by containing four modules, namely a sparse retrieval module, a dense retrieval module, a dual-link hybrid sorting module and a time-sensitive list recall module. The list recall rules based on fixed time conditions are set to control the return results of the sparse retrieval module, the return results of the dense retrieval module and the triggering of the dual-link hybrid sorting. The sparse retrieval module is based on an improved inverted index structure, which increases the keyword idf value and the tf value of the keyword in the corresponding document, and reduces the operation complexity in the retrieval stage. The dense retrieval module uses the 15% quantile of the HNSW graph index inner distance as the accuracy threshold, and returns the process list with a certain accuracy. Compared with the general hybrid retrieval process, this method shortens the time of returning the retrieval list, can return effective retrieval results in special scenes with high time limit requirements such as high-frequency transactions and rescue and disaster relief, prompts the generation of large models, and supports the intelligent decision-making process of large models.

Description

TECHNICAL FIELD

[0001] The present application relates to retrieval augmented generation of large language models under time constraints, and belongs to the technical field of retrieval augmented generation. BACKGROUND

[0002] Retrieval augmented generation (RAG) technology can integrate external knowledge generation prompts to improve the accuracy and relevance of large language model answers, and is a key technology to alleviate the "hallucination" problem of large language models. The core step of RAG technology is to retrieve the most relevant data from the external knowledge base to generate effective prompts for large models. Specifically, RAG retrieval methods can be divided into two categories: sparse retrieval and dense retrieval. Among them: sparse retrieval uses discrete representation for data indexing and retrieval, with TF-IDF and BM25 algorithms as typical representatives; dense retrieval uses deep learning technology to map data to vector space to establish index, and uses cosine distance, Euclidean distance, inner product and other algorithms to recall data. These retrieval methods focus more on improving the recall rate of retrieval, and rarely consider the impact of retrieval efficiency on the number of recalls under limited time, which is not conducive to recalling effective prompt information under limited retrieval time, and even cannot prompt the large language model in time. In high-frequency trading, rescue and disaster relief and other time-critical scenarios, large models often need to quickly access external knowledge to process sensory information and respond in a timely manner. Therefore, the present application is proposed, and the main idea is to set a fixed retrieval time parameter for the retrieval process, and return as many effective retrieval results as possible within this period to prompt the large language model. SUMMARY

[0003] The purpose of the present application is to retrieve as much effective information as possible under limited retrieval time. As shown in the figure, the method mainly includes four modules: sparse retrieval module, dense retrieval module, double-link hybrid sorting module and time-sensitive list recall module. Among them: the sparse retrieval module and the dense retrieval module use a single algorithm to retrieve data related to the problem in the database and return a sorted list; the double-link hybrid sorting module is based on the output of the sparse retrieval module and the dense retrieval module to de-duplicate and re-sort the data; the time-sensitive list recall module controls the source of the list used to generate prompts according to certain rules. The following steps are used to carry out retrieval: Figure 1

[0004] First, run the sparse retrieval module and the dense retrieval module in parallel, wherein:

[0005] (1) The sparse retrieval module realizes full-text retrieval based on an improved inverted index and a BM25 algorithm. The inverted index table generally includes keywords and their corresponding data numbers, term frequencies and positions. The improved inverted index (1) additionally includes keyword idf values and keyword tf values in the document, and the calculation formula is: Figure 2 ​​

[0006] idf i = ln(1 + (m - n i + 0.5) / (n i + 0.5))(1)

[0007] tf i = freq i × (1 + k1) / (freq i + k1 × (1 - b + b × dl / avgdl))(2)

[0008] In formula (1), n i is the number of documents containing keyword i, and m is the total number of documents in the database; in formula (2), freq i is the number of keyword i in the document; k1 and b are two constants, which are 1.2 and 0.75 respectively; dl is the total number of tokens of the document, and avgdl is the average number of tokens of all documents in the database.

[0009] In sparse retrieval, first, the problem is tokenized to obtain the keyword, the corresponding inverted file information of the keyword is queried, the BM25 score of each piece of data is calculated, that is, BM25 j = Σtf i × idf i , and a list L1 is generated according to the sorting, and it is judged whether the retrieval time is overtime, if not, the first k pieces of data in L1 are added to the list L s to be sorted.

[0010] (2) The dense retrieval module constructs an index based on the HNSW algorithm and performs retrieval. When constructing the index, the database documents are converted into 1024-dimensional vectors using the embedding model, the maximum number of “neighbor points” of each node is set to 1.5k (if it is not an integer, it is rounded down), the Euclidean distance is selected to construct the HNSW graph index, and the vector distance between the inserted point and the “neighbor point” is recorded (only once when two vectors are “neighbor points”).

[0011] Before retrieval, first, the problem is converted into a 1024-dimensional vector, and a dynamic list is created to record the document information and distance information; a “starting point” is randomly selected from the top layer of the HNSW graph index, the point closest to the problem vector is found among all “neighbor points” of the point and the point, the point is taken as the next “starting point” and the finding is repeated, and when the point closest to the problem vector no longer changes, the single-layer retrieval is completed; in the lower layer retrieval, the last “starting point” of the upper layer retrieval is taken as the starting point, and the single-layer retrieval steps are repeated, until the bottom layer; in the bottom layer retrieval, the documents in the dynamic list are sequentially calculated, the distance between the documents and the problem vector and all “neighbor points” of the documents is calculated, compared with the information in the list, sorted and updated, and when the dynamic list no longer changes, the bottom layer search is completed.

[0012] The vector distance in the retrieval process selects the Euclidean distance metric, that is Wherein Q, C are the question vector and the document vector in the database respectively. During retrieval, set the 15% quantile of all distances in the graph index as the distance threshold, return the documents in the dynamic list below the threshold to generate the list L2, and judge whether the time is up. If not, add the list L2 to the list L s to be sorted.

[0013] If the sparse retrieval module and the dense retrieval module are both completed and the time window is not exceeded, run the double-link hybrid sorting module, otherwise directly generate the prompt with L s . The double-link hybrid sorting is based on the sequence values (rs j , rdj) of the documents under the two retrieval methods, and the comprehensive score RRF j =1 / (60+rs j )+1 / (60+rd j ) is calculated to reorder the data to generate the list L s ', and judge whether the time is up. If not, generate the prompt with L s ', otherwise generate the prompt with L s .

[0014] The advantages of the present application are as follows:

[0015] (1) Set the recall rule based on time sensitivity. Compared with the general hybrid retrieval process, the time for returning the retrieval list is shorter, and the response is more timely under the fixed time.

[0016] (2) When constructing the inverted index, calculate the keyword idf value and its tf value in each document. Through the pre-position part calculation, the calculation amount of the BM25 algorithm in the retrieval stage is reduced, the sparse retrieval process is faster, and the possibility of returning the retrieval result in time within the time window is higher.

[0017] (3) When using the HNSW algorithm to construct the graph index, record the distance information. During retrieval, use the 15% quantile of all distances in the graph as the accuracy threshold, and dynamically return the vector process list to shorten the initial response time of vector retrieval. BRIEF DESCRIPTION OF DRAWINGS

[0018] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed in the embodiments of the present application will be briefly introduced as follows. Obviously, the drawings described below are only some examples of the present application, and other drawings can also be obtained according to these drawings without creative labor for those skilled in the art.

[0019] Figure 1 A structure diagram of a time-sensitive double-link hybrid retrieval enhancement method.

[0020] Figure 2 : Improved inverted index structure;

[0021] Figure 3 : A time-sensitive dual-link hybrid retrieval enhancement method flow chart. DETAILED DESCRIPTION

[0022] In order to make the purposes, technical solutions and advantages of the present application clearer, the technical solutions of the present application will be described clearly and completely below in combination with specific embodiments of the present application and corresponding drawings. Obviously, the described embodiments are only some of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0023] The time-sensitive dual-link hybrid retrieval enhancement method flow of the present application is as shown in Figure 3 , and the structure is as shown in Figure 1 .

[0024] First, run the sparse retrieval module and the dense retrieval module in parallel, and record the time when the user question is input as T0, and the retrieval time window as t, wherein:

[0025] (1) The sparse retrieval step is as follows:

[0026] ① Use the word segmenter to extract keywords from the user question to form a set W;

[0027] ② Calculate the BM25 score of each piece of data in the database. The keyword set in the improved inverted index (the improved inverted index structure is as shown in Figure 2 ) is recorded as I, find the problem keywords and all the inverted lists corresponding thereto from the inverted index, and calculate the BM25 scores of all the documents in the inverted list, and the calculation formula is:

[0028]

[0029] ③ Generate a list L1 according to the BM25 j scores from high to low, and record the time T1. If T1-T0>t, do not output the list, otherwise take the first k pieces of data in L1 and add them to the to-be-sorted list L s .

[0030] (2) The dense retrieval step is as follows:

[0031] ① Use the m3e-large model to convert the user question into a 1024-dimensional vector, recorded as Q. Calculate the 15% quantile of all distances in the HNSW graph index, recorded as S0.

[0032] ② Create a dynamic list with length of 1.5k, containing document number and distance S between document and Q j , using Euclidean distance measurement, formula is Wherein: C is the vector corresponding to the document.

[0033] ③ Randomly select a "starting point" at the top layer of HNSW graph index, add the id and S j score of the "starting point" and all its "neighbor points" to the dynamic list, and sort them in ascending order of S j score.

[0034] ④ When searching above the bottom layer, select the point with the smallest S j score as the next "starting point", calculate the S j score of the "starting point" and all its "neighbor points" (points that have been calculated are not repeated), compare, sort and update the existing data in the dynamic list, repeat the search steps, when the "starting point" no longer changes, the single-layer search is completed. Take the last "starting point" as the initial "starting point" of the lower layer, and repeat the above search steps.

[0035] ⑤ When searching the bottom layer, find the "neighbor points" of each point in the dynamic list and calculate their S j score, compare, sort and update the existing data in the list, when the dynamic list no longer changes, the bottom layer search stops.

[0036] ⑥ When the previous steps are running, return the top k data with S j score < S0 from the dynamic list, add them to list L2, and record time T2. If T2-T0>t, do not output the list, otherwise add list L2 to the sorted list L s .

[0037] When both the sparse retrieval module and the dense retrieval module are completed and max(T1, T2)-T0<t, run the dual-link hybrid sorting module, otherwise directly generate a prompt with L s . The purpose of the dual-link hybrid sorting module is to reorder L s , for j∈L s , calculate the comprehensive score RRF j , formula is:

[0038] RRF j =1 / (60+rs j )+1 / (60+rd j )(4)

[0039] In formula (4), rs j is the sequence value of j in L1, and rd j is the sequence value of j in L2. Then according to RRF jreordering from high to low, generating the ordered list L s T3, and if T3-T0>t, then generating a prompt with L s generating a prompt, otherwise generating L s ′ generating a prompt.

[0040] The above describes the specific embodiments of the present application in conjunction with the drawings, but is not a limitation on the scope of protection. Those skilled in the art should understand that various modifications or variations made by those skilled in the art on the basis of the technical solutions of the present application without creative labor are still within the scope of protection of the present application.

Claims

1. A time-sensitive dual-link hybrid search enhancement method, characterized by The retrieval module comprises a sparse retrieval module, a dense retrieval module, a double-link hybrid sorting module, and a time-sensitive list recall module, and the retrieval is carried out by using the following steps: S1: running the sparse retrieval module and the dense retrieval module in parallel, wherein the sparse retrieval step is as follows: ① extracting keywords from the user question using a word segmenter; ② calculating the BM25 score of the document containing the question keywords, finding the question keywords and their corresponding inverted list from the improved inverted index, and calculating the BM25 score of the document in the inverted list, the calculation formula is: BM25 = ∑tf × idf; ③ According to the BM25 score from high to low, generate list L1, and determine whether the retrieval time is overdue. If not, take the first k data of L1 and add them to the list L to be sorted s ; S2: running the sparse retrieval module and the dense retrieval module in parallel, wherein the dense retrieval step is as follows: ① converting the user question into a high-dimensional vector using an embedding model to create a dynamic list of fixed length; ② randomly selecting a starting point at the top layer of the HNSW graph index, calculating the Euclidean distance between the point and all its neighbor points and the question vector, and adding it to the dynamic list, taking the point with the smallest S as the next starting point and repeating the search, when the search point no longer changes, the single-layer retrieval is complete; ③ in the lower layer retrieval, taking the last starting point of the upper layer as the beginning, repeating step ② until the bottom layer; ④ in the bottom layer retrieval, finding all the neighbor points of the document corresponding point in the dynamic list, respectively calculating the distance between these neighbor points and the question vector, comparing, sorting and updating the information in the list, when the dynamic list no longer changes, the bottom layer search is complete; ⑤ When the pre-step runs, return the data meeting the distance threshold to the dynamic list, add it to the list L2, and judge whether it is overtime. If not, take the first k data of L2 and add it to the list L to be sorted s ; S3: judging whether to perform mixed sorting, when both step S1 and S2 are completed and still not timed out, performing double-link mixed sorting, otherwise directly performing L s generating a prompt; S4: RRF = 1 / (60 + rs j ) + 1 / (60 + rd j ) j reordering the data in list L s , generating ordered list L' s , where rs j , rd j are the sequence values of document j ordered under the sparse and dense retrieval methods, respectively. S5: Determine whether steps S1-S4 are timed out, if not yet timed out, generate a prompt with L' s generate a prompt, otherwise generate a prompt with L s generate a prompt.

2. The time-sensitive dual-link hybrid search enhancement method of claim 1, wherein In step S1, the improved inverted index contains not only the keywords and their corresponding data number, word frequency and position, but also the idf value of the keyword and the tf value of the keyword in the document.

Images