Knowledge-enhanced question-answering method based on multi-dimensional mixed retrieval strategy

Abstract

The invention discloses a knowledge-enhanced question-answering method based on a multi-dimensional mixed retrieval strategy, and relates to the technical field of computer processing, S01, setting a plurality of question-answering scenes based on preset literature materials, and inputting the plurality of question-answering scenes as training data into a BERT model for training to obtain a fine tuning model; s02, segmenting the preset document material through a granularity document block segmentation algorithm to obtain a plurality of paragraphs Sn and a plurality of sentences Cn corresponding to each paragraph Sn; s03, the multiple paragraphs Sn are imported into the BERT model to execute similarity judgment, the multiple similar paragraphs Sn are combined according to a judgment result, and a combined paragraph Sn set is obtained; and S04, key information of each paragraph Sn in the combined paragraph Sn set is extracted. According to the method, a multi-dimensional mixed retrieval strategy is adopted to create a quick lookup model about periodical literatures, so that data response is quicker.

Description

Technical Field

[0001] The present invention relates to the field of computer processing technology, and in particular to a knowledge-enhanced question-answering method based on a multi-dimensional hybrid retrieval strategy. Background Art

[0002] With the widespread application of large language models (LLMs) in various scenarios, they face challenges in providing accuracy and reliability of information. Retrieval-augmented generation (RAG) technology has emerged as the times require, providing an effective framework for improving model performance by combining the generation capabilities of pre-trained models with retrieval-based models.

[0003] The RAG process currently used in the banking field includes the following steps: data processing, document block segmentation, text vectorization, text retrieval, text rearrangement, prompt engineering optimization, and large model iteration. However, after the project was launched, the Q&A effect did not meet expectations, and the retrieval took a long time, resulting in low response efficiency for the entire RAG.

[0004] After combining the bad-case analysis, we found that the following factors led to the poor performance of the RAG knowledge base question answering system: 1) The current RAG question-answering system does not include query classification. The purpose of query classification is to filter and divert queries that require RAG to RAG, and directly send queries that do not require RAG to the large model. Query classification can not only improve the RAG effect, but also reduce the average latency, which is an almost immediate effect.

[0005] 2) Currently common chunking methods, such as fixed delimiter cutting and fixed length cutting, are convenient, but they may destroy the continuity of the context and truncate the semantics, and there are concerns about too much noise and loss of semantic information.

[0006] 3) Currently, the commonly used retrieval methods are mainly keyword retrieval and vector retrieval. Keyword retrieval mainly calculates the character similarity between the query and the knowledge fragments in the knowledge base. Vector retrieval is to vectorize the query and then calculate the cosine value with the vectorized knowledge fragments in the knowledge base to obtain the semantic similarity between the query and the knowledge fragments. However, these two methods have some disadvantages, such as the inability to obtain long-range associated knowledge and low information density (especially when the LLMcontextwindow is small). The recall rate of retrieval in some scenarios is only 70%, which seriously affects the effect of RAG. The traditional method to improve the recall effect is to rewrite the query, but after many changes, the similarity between the query and the document is not as good as the similarity between the documents. Summary of the invention

[0007] In view of the above technical problems, the technical solution adopted by the present invention is a knowledge-enhanced question-answering method based on a multi-dimensional hybrid retrieval strategy, comprising the following steps: S01. Setting multiple question-answering scenarios based on preset literature materials, and inputting the multiple question-answering scenarios as training data into the BERT model for training to obtain a fine-tuning model; S02, segmenting the preset document material by a granular document block segmentation algorithm to obtain a plurality of paragraphs Sn and a plurality of sentences Cn corresponding to each of the paragraphs Sn; S03, importing the plurality of paragraphs Sn into the BERT model to perform similarity judgment, and merging the plurality of similar paragraphs Sn according to the judgment result to obtain a collection of merged paragraphs Sn; S04, extracting key information of each paragraph Sn in the merged collection of paragraphs Sn; S05, importing the key information corresponding to the paragraph Sn and the plurality of sentences Cn as knowledge fragments into the ES index library and the milvus vector library; S06. Extract entity information of the preset literature material to construct a knowledge graph.

[0008] Preferably, the question-and-answer scenario is a term or a short sentence proposed by a preset user regarding part or / and all of the content of the preset literature material.

[0009] Preferably, the training data in step S01 is input into a BERT model for training, including: S11, preprocessing the plurality of question-answering scenarios to obtain texts sufficient for the BERT model to recognize word segments and symbols; S12, after inputting the text into the BERT model, inputting one of the question-answering scenarios, and obtaining a validation set through the BERT model; S13, comparing the verification set with the data set of the question-answering scenario input in step S12 to obtain difference data; S14, calculating the loss function value according to the difference data, obtaining the gradient of the loss function, and then transmitting the error signal from the output layer of the BERT model back to the input layer of the BERT model through the back propagation algorithm; S15. Repeat steps S11 to S14 until there is no difference between the validation set output by the BERT model and the data set, and then end.

[0010] Preferably, the formula for calculating the loss function value in step S14 is: , where: C is the total number of results of the dataset corresponding to the question-answering scenario, y i is the i-th component given by the validation set, pi is the probability of the i-th category in the probability distribution output by the model.

[0011] Preferably, the ES index library and milvus vector library in step S05 and the knowledge graph in step S06 belong to the same level of data feedback.

[0012] Preferably, the step S04 of performing similarity judgment and merging multiple similar keywords according to the judgment result includes: S41, use the BERT model to calculate the semantic similarity of multiple paragraphs Sn; S42, based on the comparison of the obtained multiple semantic similarity data with the preset threshold t, if it is greater than t, it is identified as a similar sentence, and then the obtained multiple similar paragraphs Sn are merged to obtain a merged paragraph, if it is less than or equal to t, the paragraph is kept unchanged.

[0013] Preferably, the key information includes paragraph abstract, keywords related to the paragraph abstract, author, publication date and publisher.

[0014] Preferably, the entity information in step S06 includes slicing the preset literature material to extract structured information, wherein the structured information includes a semantic query expression corresponding one-to-one to the key information stored in the ES index library and the milvus vector library.

[0015] The present invention has at least the following beneficial effects: 1. Improve the accuracy of responses while improving response efficiency; 2. Optimize the slicing method. The present invention uses a multi-granularity document block segmentation method so that the knowledge fragments cut out can maintain semantic integrity and can adapt to different document contents; 3. On the basis of commonly used keyword retrieval and vector retrieval, a knowledge graph context enhancement strategy parallel to the vector library is added. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0017] Figure 1 A flowchart provided for Embodiment 1 of the present invention; Figure 2 A flowchart of BERT model fine-tuning provided in Embodiment 1 of the present invention; Figure 3 Main architecture provided for embodiment 1 of the present invention Figure 1 ; Figure 4 Main architecture provided for embodiment 1 of the present invention Figure 2 . DETAILED DESCRIPTION

[0018] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0019] It should be noted that the terms "primary", "secondary", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances. In addition, the terms "including" and "having" and any variations are intended to cover non-exclusive inclusions. For example, a process, method, system, product or server that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0020] Embodiment 1:

[0021] This embodiment provides a knowledge-enhanced question-answering method based on a multi-dimensional hybrid retrieval strategy, the method comprising the following steps: Figure 1 As shown: S01. Set up multiple question-answering scenarios based on preset literature materials, and input the multiple question-answering scenarios as training data into the BERT model for training to obtain a fine-tuned model; S02, segmenting the preset document material by a granular document block segmentation algorithm to obtain a plurality of paragraphs Sn and a plurality of sentences Cn corresponding to each paragraph Sn; S03, importing multiple paragraphs Sn into the BERT model to perform similarity judgment, and merging multiple similar paragraphs Sn according to the judgment result to obtain a collection of merged paragraphs Sn; S04, extracting key information of each paragraph Sn in the merged collection of paragraphs Sn; S05, import the key information of the corresponding paragraph Sn and multiple sentences Cn as knowledge fragments into the ES index library and milvus vector library; S06. Extract entity information of preset literature materials to construct a knowledge graph.

[0022] The above, such as Figure 2 As shown, in step S01, the literature materials prepared for building the database, that is, the 'preset literature materials' are imported into the BERT model for training, and the fine-tuning model is adjusted to be sufficient to match these data. When the user inputs the predetermined search words into the fine-tuning model, the fine-tuning model extracts the search words, thereby organizing the keywords of the information extracted from the database.

[0023] The above database is actually the ES index library and milvus vector library obtained in step S05. When constructing the ES index library and milvus vector library, the literature material is segmented into paragraphs by the granular document block segmentation algorithm in steps S02-S04, and then the paragraphs are segmented into sentences, thereby obtaining paragraphs and sentences; two different data, but in essence the two are the same, and their purpose is to obtain more question-answering scenarios, so that when the fine-tuning model gives keywords, the ES index library and milvus vector library will give corresponding data.

[0024] Secondly, the knowledge graph created in step S06 is used in conjunction with the ES index library and the milvus vector library. In short, after the user enters the query information, the fine-tuning model is used to perform the judgment and extract it to the question-answering scenario, and then the ES index library, the milvus vector library and the knowledge graph are searched at the same time, and all matching information is fed back. That is, the ES index library, the milvus vector library and the knowledge graph belong to the same level of data feedback.

[0025] The question-and-answer scenario is a preset entry or short sentence proposed by a user regarding part or all of the content of the preset literature material.

[0026] Furthermore, in step S01, the training data is input into the BERT model for training, including: S11. Preprocess multiple question-answering scenarios to obtain texts that are sufficient for the BERT model to recognize word segments and symbols; S12. After inputting the text into the BERT model, a question-answering scenario is input and a validation set is obtained through the BERT model. S13, comparing the verification set with the data set of the question-answering scenario input in step S12 to obtain difference data; S14. Calculate the loss function value according to the difference data, obtain the gradient of the loss function, and then transmit the error signal from the output layer of the BERT model back to the input layer of the BERT model through the back propagation algorithm; S15. Repeat steps S11 to S14 until the validation set output by the BERT model is the same as the data set.

[0027] Furthermore, the formula for calculating the loss function value in step S14 is: , where: C is the total number of results of the dataset corresponding to the question-answering scenario, y i is the i-th component given by the validation set, p i is the probability of the i-th category in the probability distribution output by the model.

[0028] As mentioned above, by setting up question-answering scenarios for literature materials and introducing these labels and corresponding data sets during the training process, the BERT model can be guided to learn these classification information during the training process. In this way, when the model processes new literature materials, it can more accurately determine the category to which it belongs, thereby improving the accuracy and efficiency of the question-answering system.

[0029] Secondly, the preprocessing step converts the literature material into a format that the BERT model can recognize; after the validation set is input, the loss function value can be calculated by comparing the difference between the dataset and the validation set; then the model parameters are adjusted through the back-propagation algorithm so that the validation set output by the model gradually approaches the dataset. This process is an iterative process until the validation set output by the model is indistinguishable from the dataset, and the training ends. This ensures that the model has high accuracy and generalization ability when processing literature materials.

[0030] Moreover, the loss function value is used to measure the accuracy of the model output. The loss function value can be obtained by calculating the difference between the components given by the validation set and the probability distribution of the model output. The smaller this value is, the higher the accuracy of the model output is. The given formula can be used to calculate the cross entropy loss function value, which is a commonly used loss function in machine learning.

[0031] Specifically, Figure 3 and Figure 4 As shown, in step S03, similarity judgment is performed, and multiple similar keywords are merged according to the judgment result, including: S31. Use the BERT model to calculate the semantic similarity of multiple paragraphs Sn; S32, based on the comparison of the obtained multiple semantic similarity data with the preset threshold t, if it is greater than t, it is identified as a similar sentence, and then the obtained multiple similar paragraphs Sn are merged to obtain a merged paragraph, if it is less than or equal to t, the paragraph is kept unchanged.

[0032] Specifically, the BERT model (that is, the BERT-base-Chinese model) can be used to calculate the semantic similarity between multiple paragraphs Sn. By comparing the semantic similarity with the preset threshold t, it is possible to determine which sentences are similar. After merging similar paragraphs Sn, redundant information can be reduced and the efficiency and accuracy of the question-answering system can be improved. At the same time, the keyword set extracted from the merged paragraphs includes paragraph summaries, keywords, authors, publication dates, and publishers, which are helpful in building knowledge graphs and performing multi-dimensional hybrid retrieval.

[0033] In addition, the ES index library and the milvus vector library are both efficient retrieval tools. After storing the merged keyword set as knowledge fragments in these libraries, you can use the fast retrieval function they provide to speed up the retrieval. At the same time, because these libraries support distributed storage and parallel computing, they can process large-scale data sets and improve the response speed and processing power of the question-answering system.

[0034] Specifically, the entity information in step S06 includes slicing the preset document material and extracting structured information, and the structured information includes a semantic query expression corresponding to the key information stored in the ES index library and the milvus vector library.

[0035] In the above technology, the purpose of the added knowledge graph is to artificially extract the preset literature materials to construct structured information. This structured information is not directly extracted from the preset literature materials. For example, if you want to query the third quarter earnings of a company, the information stored in the ES index library and the milvus vector library may be the earnings from a certain month to a certain month, or from a certain month to a certain month. The earnings from a certain month to a certain month, or from a certain month to a certain month, are directly extracted from the preset literature materials. However, it is impossible to extract information similar to "query the third quarter earnings of a company". The added knowledge graph is to make up for the lack of information feedback in the ES index library and the milvus vector library.

[0036] This embodiment improves the reply accuracy and reply efficiency at the same time; uses a multi-granularity document block segmentation method so that the knowledge fragments cut out can maintain semantic integrity. It can adapt to different document contents; and adds a knowledge graph context enhancement strategy parallel to the vector library on the basis of commonly used keyword retrieval and vector retrieval.

[0037] Embodiment 2:

[0038] An embodiment of the present invention provides a non-transitory computer-readable storage medium, wherein at least one instruction or at least one program is stored in the non-transitory computer-readable storage medium, and the at least one instruction or at least one program is loaded and executed by a processor to implement the steps: Set up multiple question-answering scenarios based on preset literature materials, and input the multiple question-answering scenarios as training data into the BERT model for training to obtain a fine-tuned model; The preset document material is segmented by a granular document block segmentation algorithm to obtain a plurality of paragraphs Sn and a plurality of sentences Cn corresponding to each paragraph Sn; Import multiple paragraphs Sn into the BERT model to perform similarity judgment, and merge multiple similar paragraphs Sn according to the judgment results to obtain a collection of merged paragraphs Sn; Extract key information of each paragraph Sn in the merged paragraph collection Sn; Import the key information of the corresponding paragraph Sn and multiple sentences Cn as knowledge fragments into the ES index library and milvus vector library; Extract entity information of preset literature materials to construct a knowledge graph.

[0039] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0040] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as examples. In actual applications, the above-mentioned functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0041] Embodiment three:

[0042] An embodiment of the present invention provides an electronic device, including a processor and a memory, wherein the memory stores at least one instruction or at least one program, and the at least one instruction or the at least one program is loaded and executed by the processor to implement the steps: Set up multiple question-answering scenarios based on preset literature materials, and input the multiple question-answering scenarios as training data into the BERT model for training to obtain a fine-tuned model; The preset document material is segmented by a granular document block segmentation algorithm to obtain a plurality of paragraphs Sn and a plurality of sentences Cn corresponding to each paragraph Sn; Import multiple paragraphs Sn into the BERT model to perform similarity judgment, and merge multiple similar paragraphs Sn according to the judgment results to obtain a collection of merged paragraphs Sn; Extract key information of each paragraph Sn in the merged paragraph collection Sn; Import the key information of the corresponding paragraph Sn and multiple sentences Cn as knowledge fragments into the ES index library and milvus vector library; Extract entity information of preset literature materials to construct a knowledge graph.

[0043] The above description is only a preferred embodiment of the present invention and does not limit the present invention in any form. Although the present invention has been disclosed as a preferred embodiment as above, it is not used to limit the present invention. Any technician familiar with this profession can make some changes or modifications to equivalent embodiments of equivalent changes using the technical contents disclosed above without departing from the scope of the technical solution of the present invention. However, any simple modifications, equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solution of the present invention are still within the scope of the technical solution of the present invention.

Claims

1. A knowledge-enhanced question-answering method based on a multi-dimensional hybrid retrieval strategy, characterized in that: The following steps are involved: S01. Setting multiple question-answering scenarios based on preset literature materials, and inputting the multiple question-answering scenarios as training data into the BERT model for training to obtain a fine-tuning model; S02, segmenting the preset document material by a granular document block segmentation algorithm to obtain a plurality of paragraphs Sn and a plurality of sentences Cn corresponding to each of the paragraphs Sn; S03, importing the plurality of paragraphs Sn into the BERT model to perform similarity judgment, and merging the plurality of similar paragraphs Sn according to the judgment result to obtain a collection of merged paragraphs Sn; S04, extracting key information of each paragraph Sn in the merged collection of paragraphs Sn; S05, importing the key information corresponding to the paragraph Sn and the plurality of sentences Cn as knowledge fragments into the ES index library and the milvus vector library; S06. Extract entity information of the preset literature material to construct a knowledge graph.

2. According to claim 1, a knowledge-enhanced question-answering method based on a multi-dimensional hybrid retrieval strategy is characterized in that: The question-and-answer scenario is a term or short sentence proposed by a preset user regarding part or / and all of the content of the preset literature material.

3. A knowledge-enhanced question-answering method based on a multi-dimensional hybrid retrieval strategy according to claim 2, characterized in that: The training data in step S01 is input into the BERT model for training, including: S11, preprocessing the plurality of question-answering scenarios to obtain texts sufficient for the BERT model to recognize word segments and symbols; S12, after inputting the text into the BERT model, inputting one of the question-answering scenarios, and obtaining a validation set through the BERT model; S13, comparing the verification set with the data set of the question-answering scenario input in step S12 to obtain difference data; S14, calculating the loss function value according to the difference data, obtaining the gradient of the loss function, and then transmitting the error signal from the output layer of the BERT model back to the input layer of the BERT model through the back propagation algorithm; S15. Repeat steps S11 to S14 until there is no difference between the validation set output by the BERT model and the data set, and then end.

4. The knowledge-enhanced question-answering method based on a multi-dimensional hybrid retrieval strategy according to claim 3 is characterized in that: The formula for calculating the loss function value in step S14 is: , where: C is the total number of results of the dataset corresponding to the question-answering scenario, y i is the i-th component given by the validation set, p i is the probability of the i-th category in the probability distribution output by the model.

5. The knowledge-enhanced question-answering method based on a multi-dimensional hybrid retrieval strategy according to claim 1, characterized in that: The ES index library and milvus vector library in step S05 and the knowledge graph in step S06 belong to the same level of data feedback.

6. The knowledge-enhanced question-answering method based on a multi-dimensional hybrid retrieval strategy according to claim 1, characterized in that: The step S03 of performing similarity judgment and merging multiple similar keywords according to the judgment result includes: S31. Use the BERT model to calculate the semantic similarity of multiple paragraphs Sn; S32, based on the comparison of the obtained multiple semantic similarity data with the preset threshold t, if it is greater than t, it is identified as a similar sentence, and then the obtained multiple similar paragraphs Sn are merged to obtain a merged paragraph, if it is less than or equal to t, the paragraph is kept unchanged.

7. The knowledge-enhanced question-answering method based on a multi-dimensional hybrid retrieval strategy according to claim 1, characterized in that: The key information includes paragraph abstract, keywords about the paragraph abstract, author, publication date and publisher.

8. The knowledge-enhanced question-answering method based on a multi-dimensional hybrid retrieval strategy according to claim 1, characterized in that: The entity information in step S06 includes slicing the preset document material and extracting structured information, wherein the structured information includes a semantic query expression corresponding one-to-one to the key information stored in the ES index library and the milvus vector library.

9. A non-transitory computer-readable storage medium, wherein at least one instruction or at least one program is stored in the non-transitory computer-readable storage medium, characterized in that: The at least one instruction or the at least one program is loaded and executed by the processor to implement the steps of the knowledge enhanced question-answering method based on a multi-dimensional hybrid retrieval strategy as described in any one of claims 1-8.

10. An electronic device, characterized in that: It includes a processor and a memory, wherein the memory stores at least one instruction or at least one program, and the at least one instruction or the at least one program is loaded and executed by the processor to implement the steps of the knowledge enhanced question-answering method based on a multidimensional hybrid retrieval strategy as described in any one of claims 1-8.

Images