Retrieval question and answer method applied to large language model

Abstract

The invention relates to the technical field of data processing, in particular to a retrieval question and answer method applied to a large language model. In the embodiment of the invention, the electronic equipment retrieves a target information block similar to a question text based on a TF-IDF retrieval method, retrieves a target document fragment similar to the question text in the target information block based on a keyword and a vector, and takes the target document fragment as a knowledge text used for replying the question text, so that the question text is replied to the question text. According to the method, the retrieval effectiveness is improved, the influence of redundant knowledge is reduced, the cue words are dynamically constructed based on the retrieved knowledge text, and the question and answer retrieval accuracy is improved.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a retrieval question-answering method applied to a large language model. Background Art

[0002] With the advent of large language models and the improvement of information technology capabilities, the road hazard inspection question-answering method has gradually become intelligent. Because large language models are affected by data update deadlines, usage restrictions, etc., when facing the field of road hazard inspection, it is easy to have problems such as erroneous factual information and outdated information.

[0003] Based on this, the related art proposes to search the knowledge base based on semantic retrieval after receiving the question text input by the user, determine the knowledge text that the large language model may use when replying to the question text, and call the large language model to answer questions based on the knowledge text. However, relying solely on semantic retrieval of knowledge texts can easily retrieve irrelevant knowledge, which can induce hallucinations in the large language model and affect the accuracy of the reply text output by the large language model. Summary of the invention

[0004] The present application provides a retrieval question and answer method applied to a large language model, which is used to solve the problem that the existing retrieval question and answer method only relies on semantic retrieval of knowledge text, and is prone to retrieve irrelevant knowledge, which can induce hallucinations in the large language model and affect the accuracy of the reply text output by the large language model.

[0005] In a first aspect, an embodiment of the present application provides a retrieval question answering method applied to a large language model, the method comprising:

[0006] Using the TF-IDF retrieval method, the target information blocks similar to the question text are retrieved from the information blocks stored in the knowledge base, where each information block contains multiple document fragments;

[0007] Retrieve a target document segment similar to the question text from each document segment included in the target information block according to the first feature vector and the first keyword corresponding to each document segment included in the target information block, and the second feature vector and the second keyword of the question text;

[0008] The target document fragment is used as the knowledge text used to reply to the question text, and prompt words are constructed. The large language model is called according to the prompt words to generate the reply text.

[0009] The above technical solution has the following advantages or beneficial effects: the electronic device retrieves the target information block similar to the question text based on the TF-IDF retrieval method, and retrieves the target document fragment similar to the question text in the target information block based on keywords and vectors, and uses the target document fragment as the knowledge text used when replying to the question text, thereby improving the effectiveness of the retrieval, reducing the impact of redundant knowledge, and dynamically constructing prompt words based on the retrieved knowledge text to improve the accuracy of the retrieval question and answer.

[0010] In some embodiments of the present application, the TF-IDF retrieval method is used to retrieve a target information block similar to the question text from the information blocks stored in the knowledge base, including:

[0011] determining each second keyword contained in the question text;

[0012] Determine a score value of each second keyword in each information block according to the number of times each second keyword appears in the abstract corresponding to each information block of the knowledge base;

[0013] For each information block, determine the sum of the score values ​​of each second keyword corresponding to the information block, and determine the sum as the keyword score value corresponding to the information block;

[0014] A first preset number of information blocks having the largest corresponding keyword scores are determined as the target information blocks.

[0015] The above technical solution has the following advantages or beneficial effects: there may be multiple second keywords in the question text, and the electronic device can determine the score value of each second keyword in each information block, thereby determining the keyword score value corresponding to each information block according to the score value of each second keyword in each information block, and determining the target information block based on each keyword score value, so that the target information block finally determined refers to the importance of each second keyword, and the target information block has a higher correlation with the question text.

[0016] In some embodiments of the present application, after determining the score value of each second keyword in each information block according to the number of times each second keyword appears in the abstract corresponding to each information block of the knowledge base, the method further includes:

[0017] For each information block, obtain the target document type corresponding to the information block stored in the knowledge base; determine the target preset weight corresponding to the target document type based on the correspondence between the pre-saved document type and the preset weight; update the score value of each second keyword in the information block based on the target preset weight; wherein the document types include national standard documents, industry standard documents, guideline documents and landmark documents.

[0018] The above technical solution has the following advantages or beneficial effects: different preset weights are designed for different document types to enhance the guidance effect of national standards, industry standards, guidelines, and landmark documents.

[0019] In some embodiments of the present application, searching for a target document segment similar to the question text in each document segment included in the target information block according to the first feature vector and the first keyword corresponding to each document segment included in the target information block, and the second feature vector and the second keyword of the question text includes:

[0020] For each document fragment, determine the vector similarity between the second feature vector and the target first feature vector corresponding to the document fragment according to the cosine similarity algorithm; determine the first number of keywords included in the intersection of the first keyword and the second keyword; determine the ratio of the first number to the second number of the first keyword as the keyword similarity between the document fragment and the question text; determine the similarity between the document fragment and the question text according to the vector similarity and the keyword similarity;

[0021] A second preset number of document segments with the largest corresponding similarities are determined as the target document segments.

[0022] The above technical solution has the following advantages or beneficial effects: based on keywords and vectors, the target document fragment similar to the question text in the target information block is retrieved, and the target document fragment is used as the knowledge text when replying to the question text, thereby improving the effectiveness of the retrieval and reducing the impact of redundant knowledge.

[0023] In some embodiments of the present application, after using the target document segment as the knowledge text used to reply to the question text and before constructing the prompt word, the method further includes:

[0024] Inputting the target document segment and the question text into a trained back-check model, and obtaining a relevance score between the target document segment and the question text output by the back-check model;

[0025] The target relevance category corresponding to the target document segment is determined according to the relevance score and a preset score interval corresponding to each relevance category.

[0026] The above technical solution has the following advantages or beneficial effects: applying a lightweight back-checking model to evaluate the overall quality of the retrieved knowledge text, and improving the guiding effect of national standards, industry standards, guidelines, and landmark documents.

[0027] In some embodiments of the present application, the construction prompt words include:

[0028] Acquire a pre-saved prompt word template, wherein the prompt word template includes each first field for writing the knowledge text of each relevance category, and a second field for writing the question text;

[0029] Determine a target first field corresponding to the target relevance category, write the target document fragment into the target first field, and write the question text into the second field;

[0030] The completed prompt word template is determined as the prompt word.

[0031] The above technical solution has the following advantages or beneficial effects: dynamically adjusting prompt words according to the adapted search results, further stimulating the overall performance of the road hazard inspection question-and-answer method.

[0032] In some embodiments of the present application, the process of constructing the knowledge base includes:

[0033] Obtain each original document from a preset data source;

[0034] According to a preset information block segmentation rule, each original document is segmented into each information block;

[0035] For each information block, a summary contained in the information block is determined, and the information block and the summary are stored in correspondence with each other.

[0036] The above technical solution has the following advantages or beneficial effects: a knowledge element storage method suitable for the field of road hazard inspection is designed to store knowledge from multiple dimensions and multiple levels.

[0037] In some embodiments of the present application, the method further includes:

[0038] According to a preset document segmentation rule, each information block is segmented into each document segment;

[0039] For each document fragment, identify the first keyword contained in the document fragment and determine the first feature vector corresponding to the document fragment; save the document fragment, the first keyword and the first feature vector in correspondence; and save the correspondence between the document fragment and the information block corresponding to the document fragment.

[0040] The above technical solution has the following advantages or beneficial effects: a knowledge element storage method suitable for the field of road hazard inspection is designed to store knowledge from multiple dimensions and multiple levels.

[0041] In some embodiments of the present application, determining the summary contained in the information block includes:

[0042] According to a preset summary format rule, determining a subtext in the information block whose format complies with the summary format rule, and determining the subtext as the summary;

[0043] If the subtext does not exist, the information block is input into a large language model so that the large language model generates a summary of the information block, and an output of the large language model is determined as the summary.

[0044] The above technical solution has the following advantages or beneficial effects: a knowledge element storage method suitable for the field of road hazard inspection is designed to store knowledge from multiple dimensions and multiple levels.

[0045] In some embodiments of the present application, the method further includes:

[0046] The reply text is displayed, and the knowledge text is displayed.

[0047] The above technical solution has the following advantages or beneficial effects: the reply text and the knowledge text are displayed, thereby realizing a user interaction experience of a rich client.

[0048] In an embodiment of the present application, the electronic device uses a TF-IDF retrieval method to retrieve a target information block similar to the question text in the information block stored in the knowledge base, wherein each information block contains multiple document fragments; based on the first feature vector and the first keyword corresponding to each document fragment contained in the target information block, and the second feature vector and the second keyword of the question text, the target document fragment similar to the question text is retrieved in each document fragment contained in the target information block; the target document fragment is used as the knowledge text used when replying to the question text, and a prompt word is constructed, and a large language model is called according to the prompt word to generate a reply text. In an embodiment of the present application, the electronic device retrieves a target information block similar to the question text based on the TF-IDF retrieval method, and retrieves a target document fragment similar to the question text in the target information block based on keywords and vectors, and uses the target document fragment as the knowledge text used when replying to the question text, thereby improving the effectiveness of the retrieval, reducing the impact of redundant knowledge, and dynamically constructing prompt words based on the retrieved knowledge text to improve the accuracy of the retrieval question and answer. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the technical solution of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0050] Figure 1 A schematic diagram of a retrieval question answering process applied to a large language model provided in an embodiment of the present application;

[0051] Figure 2 A schematic diagram of enhanced retrieval based on a backcheck model provided in an embodiment of the present application;

[0052] Figure 3 A schematic diagram of constructing dynamic prompt words provided in an embodiment of the present application;

[0053] Figure 4 A schematic diagram of document segmentation provided in an embodiment of the present application;

[0054] Figure 5 A schematic diagram of the storage of the knowledge base provided in the embodiment of the present application;

[0055] Figure 6 An interactive schematic diagram of a response system provided in an embodiment of the present application;

[0056] Figure 7 A schematic diagram of displaying the knowledge text provided in the embodiment of the present application;

[0057] Figure 8 A schematic diagram showing details of the knowledge text provided in the embodiment of the present application;

[0058] Fig. 9 A schematic diagram of an enhanced search and answer process for road hazard inspection provided in an embodiment of the present application;

[0059] Fig.10 A schematic diagram of the structure of a retrieval question-answering device applied to a large language model provided in an embodiment of the present application;

[0060] Fig.11 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0061] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present application.

[0062] In order to improve the effectiveness of retrieval, reduce the impact of redundant knowledge, and thereby improve the accuracy of retrieval question and answering of large language models, an embodiment of the present application provides a retrieval question and answering method applied to a large language model.

[0063] In an embodiment of the present application, the electronic device adopts a TF-IDF retrieval method to retrieve a target information block similar to a question text from information blocks stored in a knowledge base, wherein each information block contains multiple document fragments; based on a first feature vector and a first keyword corresponding to each document fragment contained in the target information block, and a second feature vector and a second keyword of the question text, a target document fragment similar to the question text is retrieved from each document fragment contained in the target information block; the target document fragment is used as a knowledge text used when replying to the question text, and a prompt word is constructed, and a large language model is called according to the prompt word to generate a reply text.

[0064] Figure 1 A schematic diagram of a retrieval question answering process applied to a large language model provided in an embodiment of the present application, the process includes:

[0065] S101: Using the TF-IDF retrieval method, a target information block similar to the question text is retrieved from the information blocks stored in the knowledge base, wherein each information block contains multiple document fragments.

[0066] A large language model retrieval question and answer method provided in an embodiment of the present application is applied to an electronic device, which may be a PC or a server.

[0067] The large language model retrieval question-answering method provided in the embodiment of the present application can be used for road hazard inspection and applied to intelligent transportation systems or terminals. In order to improve the retrieval effectiveness of the road hazard inspection knowledge base and reduce the impact of redundant knowledge, the knowledge base of the embodiment of the present application adopts a knowledge element storage method suitable for the field of road hazard inspection, storing knowledge from multiple dimensions and multiple levels.

[0068] Specifically, in the embodiment of the present application, a plurality of information blocks are stored in the knowledge base, and each information block is divided into a plurality of document fragments. The information block may be a complete document or a part of a document, and no limitation is made here. Regardless of whether the information block is a complete document, the information block is divided into a plurality of document fragments.

[0069] Based on this, in an embodiment of the present application, when performing knowledge retrieval, the electronic device only needs to first retrieve the target information block similar to the question text, and then retrieve the related knowledge text based on the target information block. On the basis of reducing redundant text, it also avoids directly retrieving knowledge text from a large number of document fragments, thereby reducing the workload of knowledge retrieval and improving the efficiency and accuracy of knowledge retrieval.

[0070] Specifically, in an embodiment of the present application, the electronic device uses a TF-IDF retrieval method to retrieve target information blocks similar to the question text from the information blocks stored in the knowledge base. Among them, the TF-IDF (term frequency-inverse document frequency) retrieval method is a commonly used weighted technology for information retrieval and data mining, which consists of two parts: term frequency (Term Frequency, TF) and inverse document frequency (Inverse Document Frequency, IDF). TF represents the frequency of a word appearing in a document, and IDF represents a measure of the general importance of the word. Among them, if the fewer documents contain word t, the larger the IDF, indicating that word t has a good ability to distinguish categories; if the more documents contain word t, the smaller the IDF, indicating that the word t has a weak ability to distinguish categories.

[0071] Based on this, in an embodiment of the present application, the electronic device can determine the keywords contained in the question text. For each information block stored in the knowledge base, the electronic device determines the TF and IDF of the keyword in the information block, and based on the TF and IDF, determines the score value of the keyword in the information block. The electronic device screens one or more information blocks with high scores based on the score value of the keyword in each information block, and determines the screened information blocks as target information blocks similar to the question text.

[0072] In the embodiment of the present application, the number of target information blocks screened out by the electronic device is one or more, which is not limited here.

[0073] S102: Retrieve a target document segment similar to the question text from each document segment included in the target information block according to the first feature vector and the first keyword corresponding to each document segment included in the target information block, and the second feature vector and the second keyword of the question text.

[0074] In the embodiment of the present application, the knowledge base stores multiple information blocks, and stores each document fragment contained in each information block, the first keyword contained in each document fragment, and the first feature vector corresponding to each document fragment.

[0075] Based on this, in an embodiment of the present application, after the electronic device has screened out the target information block based on the TF-IDF retrieval method, the electronic device obtains each document fragment and the corresponding first keyword and first feature vector contained in the target information block stored in the knowledge base. The electronic device determines the second keyword contained in the question text and the second feature vector corresponding to the question text. And based on the first keyword, the first feature vector, the second keyword and the second feature vector, the target text fragment similar to the question text is retrieved in each document fragment.

[0076] Among them, in an embodiment of the present application, the electronic device can perform keyword recognition based on a keyword recognition model: the electronic device inputs the question text into the keyword recognition model, and obtains the second keyword contained in the question text output by the keyword recognition model. The electronic device can also perform keyword recognition based on a pre-configured keyword vocabulary: the electronic device searches the keyword dictionary for the second keyword contained in the question text. The electronic device can also receive user operations to perform keyword recognition: the electronic device receives a selection operation input by the user, and determines the character selected by the user as the second keyword. The electronic device can also determine the second keyword based on other methods, which are not limited here.

[0077] In an embodiment of the present application, the electronic device can obtain a second feature vector based on a text embedding model: the electronic device inputs the question text into the text embedding model, and obtains a second feature vector output by the text embedding model. The electronic device can also obtain the second feature vector based on a word vector dictionary: the electronic device searches the word vector dictionary for each word corresponding to each word in the question text, and splices each word vector according to the position of each word in the question text, and determines the spliced ​​vector as the second feature vector. The electronic device can also determine the second feature vector based on other methods, which are not limited here.

[0078] Based on this, in an embodiment of the present application, for each document segment, the electronic device can determine the similarity between the document segment and the question text based on the first keyword and the first feature vector corresponding to the document segment, and the second keyword and the second feature vector corresponding to the question text. Based on the similarity between each document segment and the question text, the electronic device selects one or more document segments with high similarity, and determines the selected document segments as target document segments similar to the question text.

[0079] In this embodiment of the present application, the number of target document fragments screened out by the electronic device is one or more, which is not limited here.

[0080] In a possible real-time manner, when the electronic device determines the similarity between the document segment and the question text based on the first keyword and the first feature vector corresponding to the document segment, and the second keyword and the second feature vector corresponding to the question text, the electronic device can determine the keyword similarity based on the first keyword and the second keyword, and determine the vector similarity based on the first feature vector and the second feature vector. The electronic device determines the similarity between the document segment and the question text based on the keyword similarity and the vector similarity.

[0081] The electronic device may determine a first sub-feature vector corresponding to the first keyword, determine a second sub-feature vector corresponding to the second keyword, and determine the similarity between the first sub-feature vector and the second sub-feature vector, and determine the similarity as the keyword similarity. In addition, the electronic device may also determine the keyword similarity in other ways, which are not limited here.

[0082] S103: Using the target document fragment as the knowledge text used to reply to the question text, constructing prompt words, and calling a large language model to generate a reply text according to the prompt words.

[0083] In the embodiment of the present application, after the electronic device determines the target document fragment, the electronic device uses the target document fragment as the knowledge text used to reply to the question text, and constructs a prompt word based on the knowledge text and the question text. The constructed prompt word can prompt the large language model to generate and output the reply text corresponding to the question text based on the knowledge carried in the knowledge text.

[0084] Specifically, in the embodiment of the present application, the electronic device obtains a pre-configured prompt text for prompting the large language model to answer questions based on the knowledge text. For example, the prompt text may be "You are now a question-answering expert, and your current task is to generate a reply text based on the user input content and the knowledge text."

[0085] The electronic device splices the prompt text, the knowledge text and the question text, and determines the spliced ​​text as the prompt word. The electronic device inputs the prompt word into the large language model, and the large language model generates and outputs the reply text corresponding to the question text according to the prompt word.

[0086] In an embodiment of the present application, the electronic device retrieves a target information block similar to the question text based on the TF-IDF retrieval method, and retrieves a target document fragment similar to the question text in the target information block based on keywords and vectors, and uses the target document fragment as the knowledge text used when replying to the question text, thereby improving the effectiveness of the retrieval, reducing the impact of redundant knowledge, and dynamically constructing prompt words based on the retrieved knowledge text to improve the accuracy of the retrieval question and answer.

[0087] In order to improve the effectiveness of retrieval, reduce the impact of redundant knowledge, and thus improve the accuracy of large language model retrieval question and answer, based on the above embodiment, in the embodiment of the present application, the TF-IDF retrieval method is used to retrieve the target information block similar to the question text in the information block stored in the knowledge base, including:

[0088] determining each second keyword contained in the question text;

[0089] Determine a score value of each second keyword in each information block according to the number of times each second keyword appears in the abstract corresponding to each information block of the knowledge base;

[0090] For each information block, determine the sum of the score values ​​of each second keyword corresponding to the information block, and determine the sum as the keyword score value corresponding to the information block;

[0091] A first preset number of information blocks having the largest corresponding keyword scores are determined as the target information blocks.

[0092] In actual application, the information block is divided into multiple text segments. If the electronic device searches for the target information block based on the specific content of the information block, and searches for the target text segment based on the specific content of the text segment, the target information block and the target text segment are actually retrieved based on the same content. Based on this, in order to improve the effectiveness and accuracy of the search, in an embodiment of the present application, a summary corresponding to each information block is stored in the knowledge base, and the electronic device can search for the target information block based on the summary corresponding to each information block stored in the knowledge base.

[0093] In addition, in the embodiment of the present application, the question text may contain multiple keywords. Based on this, the embodiment of the present application provides a TF-IDF retrieval method for multiple keywords.

[0094] Specifically, the electronic device determines the second keywords included in the question text. If the number of the second keywords is two or more, the electronic device determines the score value of each second keyword in each information block according to the number of times each second keyword appears in the abstract corresponding to each information block of the knowledge base. For each information block, the electronic device determines the sum of the score values ​​of each second keyword corresponding to the information block, and determines the sum as the keyword score value corresponding to the information block. The electronic device determines the first preset number of information blocks with the largest corresponding keyword scores as target information blocks.

[0095] In addition, if the electronic device determines that the number of second keywords included in the question text is one, the electronic device determines the score value of the second keyword in each information block according to the number of times the second keyword appears in the abstract corresponding to each information block of the knowledge base, and determines the score value corresponding to each information block as the keyword score value. The electronic device determines the first preset number of information blocks with the largest corresponding keyword scores as target information blocks.

[0096] The first preset number can be configured according to actual needs. Generally, the first preset number is 10.

[0097] In order to improve the effectiveness of retrieval, reduce the impact of redundant knowledge, and thus improve the accuracy of large language model retrieval question and answer, on the basis of the above embodiments, in the embodiment of the present application, after determining the score value of each second keyword in each information block according to the number of times each second keyword appears in the abstract corresponding to each information block of the knowledge base, the method further includes:

[0098] For each information block, obtain the target document type corresponding to the information block stored in the knowledge base; determine the target preset weight corresponding to the target document type based on the correspondence between the pre-saved document type and the preset weight; update the score value of each second keyword in the information block based on the target preset weight; wherein the document types include national standard documents, industry standard documents, guideline documents and landmark documents.

[0099] In an embodiment of the present application, in the process of road hazard inspection, different types of documents have different importances. Therefore, in order to make retrieval questions and answers more suitable for road hazard inspection, in an embodiment of the present application, the knowledge base also stores the document type corresponding to each information block, and the electronic device pre-stores the preset weight corresponding to each document type. The electronic device can update the score value corresponding to each information block according to the preset weight corresponding to the cultural type of each information block.

[0100] Specifically, for each information block, the electronic device obtains the target document type corresponding to the information block stored in the knowledge base, and determines the target preset weight corresponding to the target document type according to the correspondence between the pre-stored document type and the preset weight. The electronic device multiplies the score value of each second keyword in the information block by the target preset weight, and determines each product as the updated score value of each second keyword.

[0101] Among them, document types include but are not limited to national standard documents, industry standard documents, guideline documents and landmark documents.

[0102] In the embodiment of the present application, the electronic device may determine the score value of each second keyword in each information block based on the following formula:

[0103] TF IDF(t,d,D) =θ*TF(t,d)×IDF(t,D)

[0104] Among them, TF IDF(t,d,D) represents the score of the second keyword t in the information block d, θ represents the preset weight corresponding to the target document type of the information block d, TF(t,d) represents the term frequency, IDF(t,D) represents the inverse document frequency, and D represents all the information blocks in the knowledge base.

[0105] In order to improve the effectiveness of retrieval, reduce the impact of redundant knowledge, and thus improve the accuracy of large language model retrieval question and answer, based on the above embodiments, in the embodiment of the present application, according to the first feature vector and the first keyword corresponding to each document fragment contained in the target information block, and the second feature vector and the second keyword of the question text, searching for a target document fragment similar to the question text in each document fragment contained in the target information block includes:

[0106] For each document fragment, determine the vector similarity between the second feature vector and the target first feature vector corresponding to the document fragment according to the cosine similarity algorithm; determine the first number of keywords included in the intersection of the first keyword and the second keyword; determine the ratio of the first number to the second number of the first keyword as the keyword similarity between the document fragment and the question text; determine the similarity between the document fragment and the question text according to the vector similarity and the keyword similarity;

[0107] A second preset number of document segments with the largest corresponding similarities are determined as the target document segments.

[0108] In order to make knowledge retrieval more accurate, the embodiment of the present application designs a knowledge meta-strategy (RetrievalStrategy), which includes at least a front-end strategy (Front Strategy) and a back-end strategy (BackendStrategy). Among them, the front-end strategy is information retrieval, that is, retrieving the target information block related to the question text from the knowledge base; the back-end strategy is knowledge retrieval, that is, retrieving the target document fragment from the document fragment corresponding to the target information block; the information retrieval and knowledge retrieval are connected in series to form the knowledge meta-retrieval strategy of this application.

[0109] Among them, the information retrieval adopts the TF-IDF retrieval method to calculate the keyword score value of each information block in the knowledge base, and the keyword score value represents the importance score of each information block in answering the question text. In addition, the keyword score value establishes a mapping relationship between the question text and the information block, and the electronic device selects the first ten information blocks with the highest keyword score value as the first-level retrieval result R1 obtained by filtering.

[0110] Knowledge retrieval adopts the retrieval method of the enhanced module, and uses the first-level retrieval result R1 to filter each document fragment corresponding to each target information block. According to the first feature vector and the first keyword corresponding to each document fragment contained in the saved target information block, as well as the second feature vector and the second keyword of the question text, the evaluation index corresponding to each document fragment is calculated, and then the five document fragments with higher evaluation index values ​​are selected as the target document fragments.

[0111] In this embodiment of the present application, the evaluation index corresponding to each document fragment is the similarity between the document fragment and the question text.

[0112] In an embodiment of the present application, for each document fragment, the electronic device determines the vector similarity between the second feature vector of the question text and the target first feature vector of the document fragment based on the cosine similarity algorithm; and determines the keyword similarity based on the second keyword of the question text and the first keyword of the document fragment.

[0113] Specifically, the electronic device determines a first number of keywords included in the intersection of the first keyword and the second keyword; determines a ratio of the first number to the second number of the first keyword, and determines the ratio as the keyword similarity between the document segment and the question text. The electronic device performs a weighted summation of the vector similarity and the keyword similarity, and determines the result of the weighted summation as the similarity between the document segment and the question text.

[0114] In the embodiment of the present application, the electronic device may determine the similarity between the document segment and the question text based on the following formula:

[0115] WeightFormula=α*(Simese(query_vector,vectors))+β*(|A∩B| / A)

[0116] Among them, WeightFormula represents similarity, Simese (query_vector, vectors) represents vector similarity, (|A∩B| / A) represents keyword similarity, query_vector represents the second feature vector of the question text, vectors represents the first feature vector of the document fragment, Simese is the cosine similarity algorithm, |A∩B| is the first number of keywords contained in the intersection of the first keyword and the second keyword, A represents the second number, α and β are preset correction indexes. Generally, α (α=0.6), β (β=0.4).

[0117] After determining the similarity between each document segment and the question text, the electronic device determines a second preset number of document segments with the greatest similarity, and determines the second preset number of document segments as target document segments.

[0118] The specific value of the second preset number can be set according to actual conditions. Generally, the second preset number is 5.

[0119] In order to improve the effectiveness of retrieval, reduce the impact of redundant knowledge, and thus improve the accuracy of large language model retrieval question and answer, based on the above embodiments, in the embodiment of the present application, after using the target document fragment as the knowledge text used when replying to the question text and before constructing the prompt word, the method further includes:

[0120] Inputting the target document segment and the question text into a trained back-check model, and obtaining a relevance score between the target document segment and the question text output by the back-check model;

[0121] The target relevance category corresponding to the target document segment is determined according to the relevance score and a preset score interval corresponding to each relevance category.

[0122] In order to further improve the effect of retrieval question and answer, in an embodiment of the present application, after the electronic device determines the knowledge text used to reply to the question text, the electronic device also classifies the target document fragments to determine the relevance category of the target document fragments.

[0123] Specifically, the electronic device is pre-configured with a back-check model, and the electronic device inputs the target document segment and the question text into the trained back-check model to obtain the relevance score of the target document segment and the question text output by the back-check model. The electronic device also stores the score interval corresponding to each relevance type, and the electronic device determines the target relevance category corresponding to the target document segment based on the relevance score of the target document segment and the question text and the score interval corresponding to each relevance type.

[0124] Generally, in the embodiments of the present application, the relevance categories include at least high relevance, medium relevance and low relevance. Among them, the value of the relevance score output by the back-checking model does not exceed 1, the score interval corresponding to high relevance is 0.7 and above, the score interval corresponding to medium relevance is 0.3-0.7, and the score interval corresponding to low relevance is 0.3 and below.

[0125] The back-check model can be an xlm-roberta-base structure, with the information association dataset DataA as input and the loss function L as the evaluation index, and the back-check model is trained until convergence. The information association dataset consists of multiple groups of information data with semantic associations and multiple groups of information data without semantic associations, and each information data includes two texts. For example, "the weather is sunny" and "the sun is shining" are a group of information data with semantic associations.

[0126] Specifically, the loss function L can be expressed by the following formula:

[0127] L=-(y·log(p)+(1-y)·log(1-p))

[0128] Where L represents the loss value, y represents the real sample label, and p represents the probability that the back-checking model predicts that the sample has information association with the category. Where y=1 corresponds to information data with semantic association, and y=0 corresponds to information data without semantic association.

[0129] Figure 2 A schematic diagram of enhanced retrieval based on a backcheck model provided in an embodiment of the present application, such as the Figure 2 As shown, the electronic device determines that the original retrieval results are fragment 1, fragment 2, fragment 8 and fragment 13. Based on the review model (ReviewStrategy) and the pre-configured scoring interval corresponding to each relevance category, the electronic device determines that fragment 1 is a highly relevant document, fragments 2 and 8 are medium relevant documents, and fragment 13 is a low relevant document.

[0130] In order to improve the effectiveness of retrieval, reduce the impact of redundant knowledge, and thus improve the accuracy of large language model retrieval question and answer, based on the above embodiments, in the embodiment of the present application, the construction prompt words include:

[0131] Acquire a pre-saved prompt word template, wherein the prompt word template includes each first field for writing the knowledge text of each relevance category, and a second field for writing the question text;

[0132] Determine a target first field corresponding to the target relevance category, write the target document fragment into the target first field, and write the question text into the second field;

[0133] The completed prompt word template is determined as the prompt word.

[0134] In the embodiment of the present application, the electronic device can dynamically construct prompt words according to the retrieved knowledge text and the corresponding relevance category.

[0135] Specifically, a prompt word template is pre-stored in the electronic device, and the prompt word template includes each first field for writing the knowledge text of each relevance category, and a second field for writing the question text. In the process of constructing the prompt word, the electronic device determines the target first field corresponding to the target relevance category of the knowledge text, writes the target document fragment into the target first field, and writes the question text to be answered into the second field. The electronic device determines the completed prompt word template as the constructed prompt word.

[0136] Figure 3 A schematic diagram of constructing dynamic prompt words provided in an embodiment of the present application, such as the Figure 3 As shown, the enhanced result RF of the road hidden danger inspection knowledge base is that segment 1 is a high-relevance document, segments 2 and 8 are medium-relevance documents, and segment 13 is a low-relevance document. The electronic device uses a dynamic prompt word method to obtain a dynamic retrieval prompt word Dynamic Prompt based on the enhanced result RF of the road hidden danger inspection knowledge base. The dynamic retrieval prompt word Dynamic Prompt is:

[0137] "Highly relevant documents (add positive statement)

[0138] The answer requires reference to the following relevant knowledge:

[0139] 1. Clip 1

[0140] Related documents in (add possible expressions, reduce the knowledge proportion)

[0141] The following knowledge may be related to the question. Please evaluate the knowledge base content and extract relevant knowledge answers:

[0142] 1. Clip 2

[0143] 2. Clip 8

[0144] Low relevance documents (do not add)".

[0145] It should be noted that the dynamic search prompt word Dynamic Prompt should also include the question text and the prompt text used to prompt the large language model to answer questions based on the knowledge text. Figure 3 The question text and prompt text are omitted.

[0146] In an embodiment of the present application, the electronic device constructs a generated question-and-answer structure based on ChatGLM as a road hazard inspection question-and-answer method ModelLLM, and uses the dynamic search prompt word Dynamic Prompt as input to obtain a road hazard inspection response result.

[0147] In order to improve the effectiveness of retrieval, reduce the impact of redundant knowledge, and thus improve the accuracy of large language model retrieval questions and answers, based on the above embodiments, in the embodiment of the present application, the process of constructing the knowledge base includes:

[0148] Obtain each original document from a preset data source;

[0149] According to a preset information block segmentation rule, each original document is segmented into each information block;

[0150] For each information block, a summary contained in the information block is determined, and the information block and the summary are stored in correspondence with each other.

[0151] In order to improve the effectiveness of retrieval and enhance the applicability and accuracy of the road hazard inspection question-and-answer method, in the embodiment of the present application, a knowledge element storage method suitable for the field of road hazard inspection is designed to store knowledge in a multi-dimensional and multi-level manner. In the embodiment of the present application, a knowledge base is pre-constructed to store knowledge in the field of road hazard inspection, and the knowledge base is a road hazard inspection knowledge base.

[0152] In one implementation, the electronic device obtains each original document from a preset data source; and divides each original document into each information block according to a preset information block segmentation rule.

[0153] Specifically, the electronic device obtains road hazard inspection data D from a data source stored in the traffic field. The road hazard inspection data D contains multiple original documents, each of which is divided into national standard documents, industry standard documents, guide documents and landmark documents according to the document type. The electronic device is pre-configured with the information block segmentation rule SplitStrategy, which specifically limits the length of each information block by the maximum length and the minimum length, and uses the overlap parameter to allow the text length of each information block to overlap.

[0154] The information block segmentation rule can be expressed by the following formula:

[0155] INFX=SplitStrategy(D,MaxLength=512,MinLength=256,OverLap=64)

[0156] Among them, INFX represents an information block, D represents a document, MaxLength represents the maximum length of an information block, MinLength represents the minimum length of an information block, and OverLap represents the text length that is allowed to overlap between multiple information blocks.

[0157] For example, taking the document of road hazard inspection data D as an example, the block segmentation strategy SplitStrategy is used to obtain the knowledge segmentation result CHUNKS, which can be expressed as {INF1, INF2…INFX}. Among them, {} is the knowledge segmentation result of a document, and INFX is an information block. The electronic device summarizes the knowledge segmentation results CHUNKS of all documents and constructs the original knowledge base KNR of road hazard inspection.

[0158] INFX=SplitStrategy(D,MaxLength=512,MinLength=256,OverLap=64

[0159] Among them, INFX corresponds to the knowledge segment of data D, MaxLength indicates the maximum length of each segment, MinLength indicates the minimum length of each segment, and OverLap indicates that multiple INFs are allowed to overlap.

[0160] Figure 4 The document segmentation diagram provided in the embodiment of the present application is as follows: Figure 4 As shown, document 1 is divided into information block INF1, information block INF2 and information block INF3.

[0161] Furthermore, in the embodiment of the present application, for each information block, the electronic device also determines the summary contained in the information block, and stores the information block and the summary in the knowledge base accordingly. In addition, the electronic device also determines the title corresponding to each information block, and stores each information block, title and summary in the knowledge base accordingly.

[0162] In order to improve the effectiveness of retrieval, reduce the impact of redundant knowledge, and thus improve the accuracy of large language model retrieval question and answer, based on the above embodiments, in the embodiment of the present application, the method further includes:

[0163] According to a preset document segmentation rule, each information block is segmented into each document segment;

[0164] For each document fragment, identify the first keyword contained in the document fragment and determine the first feature vector corresponding to the document fragment; save the document fragment, the first keyword and the first feature vector in correspondence; and save the correspondence between the document fragment and the information block corresponding to the document fragment.

[0165] In an embodiment of the present application, after the electronic device saves each information block in the knowledge base, the electronic device will also divide each information block into multiple document segments and save them in the knowledge base.

[0166] Specifically, the electronic device divides each information block into each document segment according to a preset document segment segmentation rule.

[0167] For example, taking the information block INF as an example, the document segmentation rule is adopted to obtain the document segmentation result {Chs1, Chs2…Chs10}, where {} is the knowledge segmentation result of an information block, and ChsX is the document segment.

[0168] In addition, for each document fragment, the electronic device will also identify the first keyword contained in the document fragment and determine the first feature vector corresponding to the document fragment; save the document fragment, the first keyword and the first feature vector accordingly; and save the correspondence between the document fragment and the information block corresponding to the document fragment.

[0169] Specifically, the electronic device constructs a bge vector retrieval method and a Keybert keyword extraction method, uses the bge vector retrieval to process the document fragment ChsX to obtain the first feature vector Vec, and uses the Keybert keyword extraction to obtain the first keyword Words.

[0170] In the embodiment of the present application, the knowledge base adopts a two-level knowledge element storage method, wherein the first level is the information block layer (including serial number, title and summary), and the second level is the knowledge element storage layer (including document fragments, keywords and vectors). In addition, the first level can also store document types.

[0171] Figure 5 A schematic diagram of the storage of the knowledge base provided in the embodiment of the present application, such as the Figure 5 As shown, the knowledge base includes but is not limited to the first level and the second level, wherein the sequence number, title, content and summary of the information block are stored in the first level, and the document fragments, keywords and vectors are stored in the second level. Moreover, each information block in the first level corresponds to a second level, and the document fragments stored in the second level are each document fragment obtained by segmenting the information block.

[0172] In order to improve the effectiveness of retrieval, reduce the impact of redundant knowledge, and thus improve the accuracy of large language model retrieval question and answer, based on the above embodiments, in the embodiment of the present application, the determination of the summary contained in the information block includes:

[0173] According to a preset summary format rule, determining a subtext in the information block whose format complies with the summary format rule, and determining the subtext as the summary;

[0174] If the subtext does not exist, the information block is input into a large language model so that the large language model generates a summary of the information block, and an output of the large language model is determined as the summary.

[0175] In the embodiment of the present application, multiple summary extraction strategies are designed in the electronic device to determine the summary corresponding to each information block. For example, the electronic device can use a regular correspondence extraction method to determine the summary contained in the information block; the electronic device can also use a large language model to determine the summary contained in the information block.

[0176] Specifically, the electronic device is preset with summary format rules. For each information block, the electronic device can identify the subtext in the information block whose format complies with the summary format rule according to the summary format rule, and determine the subtext as the summary of the information block. If the electronic device determines that there is no subtext in the information block whose format complies with the summary format rule, the electronic device inputs the information block into the large language model, so that the large language model generates a summary for the information block, and determines the output of the large language model as the summary corresponding to the information block.

[0177] In addition, in the embodiment of the present application, the electronic device may not search based on the summary format rule, but directly input the information block into the large language model to generate a summary.

[0178] The large language model used by the electronic device for summary generation and the large language model used for question answering may be the same model or different models.

[0179] In the embodiment of the present application, the electronic device may determine the summary based on the following formula:

[0180]

[0181] Among them, ExtractOrGenernal represents the summary extraction strategy, content is the input content, Extract is the regular object extraction method (extraction based on summary format rules), and Genernal is the object generation method that applies a large language model (such as Tongyi Qianwen).

[0182] In addition, in the embodiment of the present application, the process of the electronic device determining the title of the information block is similar to the process of determining the summary of the information block. Specifically, the electronic device determines the subtext in the information block that conforms to the title format rule according to the preset title format rule, and determines the subtext as the title; if the subtext does not exist, the information block is input into the large language model, so that the large language model generates a title for the information block, and the output of the large language model is determined as the title.

[0183] The process of the electronic device determining the title of the information block is similar to the process of determining the summary of the information block, which will not be described in detail here, and the process of determining the summary can be referred to.

[0184] In order to improve the effectiveness of retrieval, reduce the impact of redundant knowledge, and thus improve the accuracy of large language model retrieval question and answer, based on the above embodiments, in the embodiment of the present application, the method further includes:

[0185] The reply text is displayed, and the knowledge text is displayed.

[0186] In the embodiment of the present application, after the electronic device determines the reply text of the question text, the electronic device displays the reply text and displays the knowledge text used to generate the reply text.

[0187] Specifically, in the embodiment of the present application, the basic structure and content of the answering system can also be constructed by HTML, and CSS can be used for page beautification and layout design. Among them, the answering system uses JavaScript to realize the interactive docking of the enhanced search question-answering method for road hidden danger inspection. When the user clicks a button defined by HTML, JavaScript will capture the click event and execute the enhanced search question-answering method for road hidden danger inspection. The reply result includes the question-answering result, reference table information, document type display, and knowledge element display part.

[0188] Figure 6 The interactive diagram of the response system provided in the embodiment of the present application is shown in the figure. Figure 6 As shown, the answering system generates and displays a reply text based on the question and answer method provided in this application for the question text input by the user, and displays the information source under the reply text, wherein the information source is the knowledge text used by the answering system to determine the reply text.

[0189] Furthermore, in the embodiment of the present application, when the knowledge text is displayed below the reply text, it may be the title of the information block corresponding to the displayed knowledge text, and the document type corresponding to the knowledge text may also be displayed. When the user clicks on the title of a certain knowledge text, the electronic device will display the specific reference target document fragment corresponding to the title when the reply text was generated, indicating the knowledge text based on which the large language model generated the reply.

[0190] Figure 7 A schematic diagram of displaying the knowledge text provided in the embodiment of the present application, such as the Figure 7 As shown, the electronic device displays the title of the information block corresponding to each knowledge text and the corresponding document type. For example, Title 1 is a national standard document; Title 2 is an industry standard document.

[0191] Figure 8A schematic diagram showing details of the knowledge text provided in the embodiment of the present application, such as the Figure 8 As shown, when the user clicks on Title 1 and the electronic device displays the specific reference target document fragment 1 corresponding to Title 1 when the reply text is generated.

[0192] Fig. 9 A schematic diagram of an enhanced search and answer process for road hazard inspection provided in an embodiment of the present application, such as the Fig. 9 As shown, the process includes: obtaining road hazard inspection data (document 1, document 2, document n) from a data source stored in the traffic field; segmenting the road hazard inspection data to construct a road hazard inspection original knowledge base, in which each information block is stored; segmenting the information block in the road hazard inspection original knowledge base to obtain each document fragment, and using a hierarchical storage method to obtain a road hazard inspection knowledge base; if a user question is received, determining the vectorization result (second feature vector) and keyword result (second keyword) of the user question; based on the vectorization result and keyword result, retrieving the original result (target document fragment) in the road hazard inspection knowledge base; using a back-check model to enhance the original result to obtain an increased result (correlation category corresponding to the target document fragment); constructing prompt words based on the enhanced result and the user question, and obtaining a road hazard inspection response result based on a large language model.

[0193] Based on the above embodiments, Fig.10 A schematic diagram of the structure of a retrieval question-answering device applied to a large language model provided in an embodiment of the present application, the device comprising:

[0194] The retrieval module 1001 is used to use the TF-IDF retrieval method to retrieve a target information block similar to the question text from the information blocks stored in the knowledge base, wherein each information block contains a plurality of document fragments; based on the first feature vector and the first keyword corresponding to each document fragment contained in the target information block, and the second feature vector and the second keyword of the question text, retrieve a target document fragment similar to the question text from each document fragment contained in the target information block;

[0195] The processing module 1002 is used to use the target document fragment as the knowledge text used to reply to the question text, construct prompt words, and call the large language model to generate a reply text according to the prompt words.

[0196] In a possible implementation, the retrieval module 1001 is specifically used to determine each second keyword contained in the question text; determine the score value of each second keyword in each information block according to the number of times each second keyword appears in the abstract corresponding to each information block of the knowledge base; for each information block, determine the sum of the score values ​​of each second keyword corresponding to the information block, and determine the sum as the keyword score value corresponding to the information block; and determine the first preset number of information blocks with the largest corresponding keyword score values ​​as the target information blocks.

[0197] In a possible implementation, the retrieval module 1001 is also used to obtain, for each information block, a target document type corresponding to the information block stored in the knowledge base; determine a target preset weight corresponding to the target document type based on a correspondence between pre-saved document types and preset weights; and update the score value of each second keyword in the information block based on the target preset weight; wherein the document types include national standard documents, industry standard documents, guideline documents and landmark documents.

[0198] In a possible implementation, the retrieval module 1001 is specifically used to determine, for each document fragment, the vector similarity between the second feature vector and the target first feature vector corresponding to the document fragment based on the cosine similarity algorithm; determine the first number of keywords contained in the intersection of the first keyword and the second keyword; determine the ratio of the first number to the second number of the first keyword as the keyword similarity between the document fragment and the question text; determine the similarity between the document fragment and the question text based on the vector similarity and the keyword similarity; and determine the second preset number of document fragments with the largest corresponding similarity as the target document fragment.

[0199] In a possible implementation, the processing module 1002 is also used to input the target document fragment and the question text into a trained back-checking model to obtain a relevance score between the target document fragment and the question text output by the back-checking model; and determine a target relevance category corresponding to the target document fragment based on the relevance score and a preset score range corresponding to each relevance category.

[0200] In a possible implementation, the processing module 1002 is specifically used to obtain a pre-saved prompt word template, wherein the prompt word template contains each first field for writing the knowledge text of each relevance category, and a second field for writing the question text; determine the target first field corresponding to the target relevance category, write the target document fragment into the target first field, and write the question text into the second field; and determine the completed prompt word template as the prompt word.

[0201] In a possible implementation, the processing module 1002 is further used to obtain each original document from a preset data source; divide each original document into each information block according to a preset information block segmentation rule; for each information block, determine the summary contained in the information block, and save the information block and the summary correspondingly.

[0202] In a possible implementation, the processing module 1002 is also used to divide each information block into each document fragment according to a preset document fragment segmentation rule; for each document fragment, identify the first keyword contained in the document fragment, and determine the first feature vector corresponding to the document fragment; save the document fragment, the first keyword and the first feature vector in correspondence; and save the correspondence between the document fragment and the information block corresponding to the document fragment.

[0203] In a possible implementation, the processing module 1002 is further used to determine, according to preset summary format rules, a subtext in the information block whose format complies with the summary format rules, and determine the subtext as the summary; if the subtext does not exist, input the information block into a large language model so that the large language model generates a summary for the information block, and determine the output of the large language model as the summary.

[0204] In a possible implementation, the device further includes:

[0205] The display module 1003 is used to display the reply text and the knowledge text.

[0206] Based on the above embodiments, the present application also provides an electronic device, Fig.11 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application is shown in FIG. Fig.11 As shown, it includes: a processor 1101, a communication interface 1102, a memory 1103 and a communication bus 1104, wherein the processor 1101, the communication interface 1102, and the memory 1103 communicate with each other through the communication bus 1104;

[0207] The memory 1103 stores a computer program. When the program is executed by the processor 1101, the processor 1101 executes the steps of the retrieval question answering method applied to a large language model as provided in the above embodiments.

[0208] Since the principle of solving the problem by the above electronic device is similar to the retrieval question answering method applied to a large language model, the implementation of the above electronic device can refer to the embodiment of the method, and the repeated parts will not be repeated.

[0209] The communication bus mentioned in the above electronic device can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus. The communication interface 1102 is used for communication between the above electronic device and other devices. The memory may include a random access memory (RAM) and may also include a non-volatile memory (NVM), such as at least one disk storage. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.

[0210] The above-mentioned processor can be a general-purpose processor, including a central processing unit, a network processor (Network Processor, NP), etc.; it can also be a digital signal processing processor (Digital Signal Processing, DSP), an application-specific integrated circuit, a field programmable gate array or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component, etc.

[0211] On the basis of the above embodiments, an embodiment of the present invention further provides a computer-readable storage medium, which stores a computer program executable by a processor. When the program runs on the processor, the processor implements the steps of the retrieval question and answer method applied to a large language model as provided in the above embodiments.

[0212] Since the principle of solving the problem by the above-mentioned computer-readable storage medium is similar to the retrieval question-answering method applied to a large language model, the implementation of the above-mentioned computer-readable storage medium can refer to the embodiment of the method, and the repeated parts will not be repeated.

[0213] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0214] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0215] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0216] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0217] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A retrieval question answering method applied to a large language model, characterized in that: The method comprises: Using the TF-IDF retrieval method, the target information blocks similar to the question text are retrieved from the information blocks stored in the knowledge base, where each information block contains multiple document fragments; Retrieve a target document segment similar to the question text from each document segment included in the target information block according to the first feature vector and the first keyword corresponding to each document segment included in the target information block, and the second feature vector and the second keyword of the question text; The target document fragment is used as the knowledge text used to reply to the question text, and prompt words are constructed. The large language model is called according to the prompt words to generate the reply text.

2. The method according to claim 1, characterized in that The TF-IDF retrieval method is used to retrieve target information blocks similar to the question text from the information blocks stored in the knowledge base, including: determining each second keyword contained in the question text; Determine a score value of each second keyword in each information block according to the number of times each second keyword appears in the abstract corresponding to each information block of the knowledge base; For each information block, determine the sum of the score values ​​of each second keyword corresponding to the information block, and determine the sum as the keyword score value corresponding to the information block; A first preset number of information blocks having the largest corresponding keyword scores are determined as the target information blocks.

3. The method according to claim 2, characterized in that After determining the score value of each second keyword in each information block according to the number of times each second keyword appears in the abstract corresponding to each information block of the knowledge base, the method further includes: For each information block, obtain the target document type corresponding to the information block stored in the knowledge base; determine the target preset weight corresponding to the target document type based on the correspondence between the pre-saved document type and the preset weight; update the score value of each second keyword in the information block based on the target preset weight; wherein the document types include national standard documents, industry standard documents, guideline documents and landmark documents.

4. The method according to claim 1, characterized in that: The step of retrieving a target document segment similar to the question text from each document segment included in the target information block according to the first feature vector and the first keyword corresponding to each document segment included in the target information block, and the second feature vector and the second keyword of the question text includes: For each document fragment, determine the vector similarity between the second feature vector and the target first feature vector corresponding to the document fragment according to the cosine similarity algorithm; determine the first number of keywords included in the intersection of the first keyword and the second keyword; determine the ratio of the first number to the second number of the first keyword as the keyword similarity between the document fragment and the question text; determine the similarity between the document fragment and the question text according to the vector similarity and the keyword similarity; A second preset number of document segments with the largest corresponding similarities are determined as the target document segments.

5. The method according to claim 1, characterized in that After using the target document segment as the knowledge text used in replying the question text and before constructing the prompt word, the method further includes: Inputting the target document segment and the question text into a trained back-check model, and obtaining a relevance score between the target document segment and the question text output by the back-check model; The target relevance category corresponding to the target document segment is determined according to the relevance score and a preset score interval corresponding to each relevance category.

6. The method according to claim 5, characterized in that The construction prompt words include: Acquire a pre-saved prompt word template, wherein the prompt word template includes each first field for writing the knowledge text of each relevance category, and a second field for writing the question text; Determine a target first field corresponding to the target relevance category, write the target document fragment into the target first field, and write the question text into the second field; The completed prompt word template is determined as the prompt word.

7. The method according to claim 1, characterized in that The construction process of the knowledge base includes: Obtain each original document from a preset data source; According to a preset information block segmentation rule, each original document is segmented into each information block; For each information block, a summary contained in the information block is determined, and the information block and the summary are stored in correspondence with each other.

8. The method according to claim 7, characterized in that The method further comprises: According to a preset document segmentation rule, each information block is segmented into each document segment; For each document fragment, identify the first keyword contained in the document fragment and determine the first feature vector corresponding to the document fragment; save the document fragment, the first keyword and the first feature vector in correspondence; and save the correspondence between the document fragment and the information block corresponding to the document fragment.

9. The method according to claim 7, characterized in that: Determining the summary contained in the information block includes: According to a preset summary format rule, determining a subtext in the information block whose format complies with the summary format rule, and determining the subtext as the summary; If the subtext does not exist, the information block is input into a large language model so that the large language model generates a summary of the information block, and an output of the large language model is determined as the summary.

10. The method according to any one of claims 1 to 9, characterized in that: The method further comprises: The reply text is displayed, and the knowledge text is displayed.

Images