Question and answer method based on large language model and related equipment

Abstract

The invention provides a question and answer method based on a large language model and related equipment, and relates to the technical field of artificial intelligence, and the method comprises the steps: obtaining an original question, obtaining a plurality of thinking chain prompts in different directions, carrying out the rewriting processing of the original question according to each thinking chain prompt, obtaining a plurality of rewritten questions in different directions, and performing vector representation on the original question, the first rewritten question and the plurality of second rewritten questions, performing fusion to obtain a fusion vector, and generating a question answer according to the vector representation, the fusion vector and the question and answer large language model. The output of a rewriting question is optimized by utilizing a thinking chain technology, processing is carried out through thinking chains in multiple directions, final fusion is carried out to improve the accuracy of question and answer retrieval, and on the basis of reserving an original question, topics and contextual information, which may appear in different angles, of the question are considered, and the recall rate is improved.

Description

Background Art

[0002] With the development of artificial intelligence (AI) technology, using large language models to improve quality and efficiency is also an important part of enterprises exploring growth momentum. Therefore, the application of AI question and answer has also become an important part of technological development.

[0003] At present, "retrieval-reading-generation" based on large language models and retrieval-enhanced generation has become a standard paradigm for vertical field applications. This type of large model combines the key information of the vertical domain knowledge base and powerful natural language processing capabilities, aiming to provide efficient and accurate solutions, but its application is challenging.

[0004] Therefore, there is an urgent need for an efficient and accurate question-answering method based on a large language model.

[0005] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present disclosure, and therefore may include information that does not constitute the prior art known to ordinary technicians in the field. Summary of the invention

[0006] The present disclosure provides a question-answering method and related devices based on a large language model, which improve the accuracy of retrieved answers to questions to at least a certain extent, consider different angles of topics and contextual information that may appear, and improve the recall rate.

[0007] Other features and advantages of the present disclosure will become apparent from the following detailed description, or may be learned in part by the practice of the present disclosure.

[0008] In a first aspect, an embodiment of the present disclosure provides a question answering method based on a large language model, the method comprising:

[0009] Get the original question;

[0010] By rewriting the large language model, we obtain multiple thought chain prompts in different directions. According to each thought chain prompt, we rewrite the original question in parallel to obtain multiple rewritten questions in different directions.

[0011] The original problem and multiple rewriting problems in different directions are represented by vectors respectively, and then fused to obtain a fused vector;

[0012] Generate answers to questions based on vector representation, fused vectors, and the question-answering language model.

[0013] In a possible embodiment, the thought chain prompt includes: a first thought chain prompt and a second thought chain prompt;

[0014] By rewriting the large language model, we obtain multiple thought chain prompts in different directions. According to each thought chain prompt, we rewrite the original question in parallel to obtain multiple rewritten questions in different directions, including:

[0015] Obtain a first thought chain prompt through the first rewriting large language model, perform a first rewriting process on the original question, and identify the user's intention to obtain a first rewritten question;

[0016] The second thought chain prompt is obtained through the second rewriting large language model to perform a second rewriting process on the original question, so as to obtain second rewriting questions under multiple different subject scenarios; wherein the first rewriting process and the second rewriting process are parallel processes.

[0017] In a possible embodiment, obtaining a first thought chain prompt through a first rewriting large language model, performing a first rewriting process on the original question, and identifying the user's intention to obtain a first rewritten question includes:

[0018] Get session memory data from the session memory module;

[0019] Generate a first thinking chain prompt according to the pre-configured first prompt information, the session memory data and the preset output format of the first rewriting question;

[0020] Through the first rewriting large language model with a small number of parameters, the first thought chain prompt and the original question are obtained, the user intention is identified and the original question is rewritten to obtain the first rewritten question.

[0021] In a possible embodiment, the second rewriting large language model is used to obtain a second thought chain prompt to perform a second rewriting process on the original question, and a second rewriting question under multiple different subject scenarios is obtained, including:

[0022] According to the preset second prompt information, a second thought chain prompt is generated; the prompt word is used to prompt the second rewriting process to rewrite the original question into multiple different subject scenes;

[0023] Through the second rewriting large language model with a small number of parameters, the second thinking chain prompts and the original question are obtained, and the original question is rewritten into multiple different subject scenarios to obtain multiple second rewritten questions.

[0024] In a possible embodiment, the vector representation includes a dense vector representation; the dense vector representation includes: an original dense vector and a first dense vector;

[0025] The original problem and multiple rewriting problems in different directions are represented by vectors respectively and fused to obtain fused vectors, including:

[0026] Performing dense vector representation on the original problem, the first rewriting problem, and the plurality of second rewriting problems to obtain an original dense vector, a first dense vector, and a plurality of second dense vectors;

[0027] determining an average dense vector of the plurality of second dense vectors;

[0028] Fusing the original dense vector with the average dense vector to obtain a first fused vector;

[0029] The first dense vector is fused with the average dense vector to obtain a second fused vector.

[0030] In a possible embodiment, the vector representation includes a sparse vector representation; the sparse vector representation includes: an original sparse vector and a first sparse vector;

[0031] The original problem and multiple rewriting problems in different directions are represented by vectors respectively and fused to obtain fused vectors, including:

[0032] Performing sparse vector representation on the original problem, the first rewriting problem, and the plurality of second rewriting problems to obtain an original sparse vector, a first sparse vector, and a plurality of second sparse vectors;

[0033] Fusing the original sparse vector with the plurality of second sparse vectors to obtain a third fused vector;

[0034] The first sparse vector is fused with the plurality of second sparse vectors to obtain a fourth fused vector.

[0035] In a possible embodiment, generating a question answer based on the vector representation, the fusion vector, and the question-answering language model includes:

[0036] Similarity calculation is performed in the knowledge base based on the vector representation and the fusion vector, and multiple pieces of knowledge are retrieved;

[0037] Input multiple pieces of knowledge and conversation memory data into the question-answering language model to generate answers to questions.

[0038] In a possible embodiment, the vector representation includes: a dense vector representation and a sparse vector representation; the dense vector representation includes: an original dense vector and a first dense vector; the sparse vector representation includes: an original sparse vector and a first sparse vector; the fused vector includes: a first fused vector and a second fused vector corresponding to the dense vector representation, and a third fused vector and a fourth fused vector corresponding to the sparse vector representation;

[0039] Generate answers to questions based on vector representation, fusion vectors, and the question-answering language model, including:

[0040] similarity calculation is performed in the knowledge base according to the original dense vector, the first dense vector, the first fused vector, and the second fused vector, and a plurality of first knowledge items are retrieved and sorted by similarity scores;

[0041] Performing similarity calculation in the knowledge base according to the original sparse vector, the first sparse vector, the third fused vector, and the fourth fused vector, retrieving a plurality of second knowledge items sorted by similarity scores;

[0042] Determine a combined score of each piece of knowledge according to the ranking of each piece of knowledge in the first piece of knowledge and the second piece of knowledge;

[0043] According to the combination score sorting, the third knowledge ranked in the front preset position is obtained;

[0044] The third knowledge and conversation memory data are input into the question-answering language model to generate answers to questions.

[0045] In a possible embodiment, rewriting the training process of a large language model includes:

[0046] Build a training dataset;

[0047] Based on supervised fine-tuning, an untrained rewritten large language model with a small number of parameters is trained according to the training data set to obtain a rewritten large language model that meets the preset conditions;

[0048] Test the rewritten large language model to obtain the test results output by the question-answering large language model;

[0049] An evaluation is performed based on the test results. If the precision or recall rate of the test results meets the preset threshold, a trained rewritten large language model is obtained.

[0050] In a second aspect, an embodiment of the present disclosure provides a question-answering device based on a large language model, comprising:

[0051] An acquisition unit, used to acquire the original question;

[0052] A rewriting unit is used to obtain multiple thinking chain prompts in different directions, and rewrite the original question according to each thinking chain prompt to obtain multiple rewritten questions in different directions;

[0053] A fusion unit, used for respectively representing the original problem, the first rewriting problem and the plurality of second rewriting problems by vectors, and fusing them to obtain a fused vector;

[0054] The generation unit is used to generate answers to questions based on the vector representation, fusion vectors, and the question-answering language model.

[0055] In a third aspect, an embodiment of the present disclosure provides an electronic device, comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to execute the method in the first aspect above by executing the executable instructions.

[0056] In a fourth aspect, an embodiment of the present disclosure provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the method in the first aspect described above is implemented.

[0057] In a fifth aspect, according to another aspect of the present disclosure, a computer program product or a computer program is also provided, the computer program product or the computer program comprising computer instructions, the computer instructions being stored in a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs any of the above methods.

[0058] The embodiments of the present disclosure provide a question-answering method based on a large language model and related equipment, the method comprising: obtaining an original question, obtaining a plurality of thought chain prompts in different directions, rewriting the original question according to each thought chain prompt, obtaining a plurality of rewritten questions in different directions, respectively vectorizing the original question, the first rewritten question, and the plurality of second rewritten questions, and fusing them to obtain a fused vector, and generating an answer to the question according to the vector representation, the fused vector, and the question-answering large language model. The output of the rewritten question is optimized using thought chain technology, and is processed through thought chains in multiple directions, and finally fused to improve the accuracy of question-answering answer retrieval, while retaining the original question, considering different angles, topics, and contextual information that may appear in the question, and improving the recall rate.

[0059] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] The accompanying drawings herein are incorporated into the specification and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the specification are used to explain the principles of the present disclosure. Obviously, the accompanying drawings described below are only some embodiments of the present disclosure, and for ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without creative work.

[0061] Figure 1 A flowchart of a question-answering method based on a large language model in an embodiment of the present disclosure is shown;

[0062] Figure 2 A flowchart of a rewriting problem in an embodiment of the present disclosure is shown;

[0063] Figure 3 A flowchart of obtaining a first rewriting problem in an embodiment of the present disclosure is shown;

[0064] Figure 4 A flowchart of obtaining a second rewriting problem in an embodiment of the present disclosure is shown;

[0065] Figure 5 A flowchart of dense vector fusion in an embodiment of the present disclosure is shown;

[0066] Figure 6 A flow chart of sparse vector fusion in an embodiment of the present disclosure is shown;

[0067] Figure 7 A flow chart of a hybrid search question answer in an embodiment of the present disclosure is shown;

[0068] Figure 8 A flowchart of a large language model training process rewriting in an embodiment of the present disclosure is shown;

[0069] Fig. 9 An architectural diagram of a question-answering system in an embodiment of the present disclosure is shown;

[0070] Fig.10 A structural schematic diagram of a question-answering device based on a large language model in an embodiment of the present disclosure is shown;

[0071] Fig.11 A schematic structural diagram of an electronic device in an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0072] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that the disclosure will be more comprehensive and complete and to fully convey the concepts of the example embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0073] In addition, the accompanying drawings are only schematic illustrations of the present disclosure and are not necessarily drawn to scale. The same reference numerals in the figures represent the same or similar parts, and their repeated description will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software form, or implemented in one or more hardware modules or integrated circuits, or implemented in different networks and / or processor devices and / or microcontroller devices.

[0074] Explain the nouns that appear.

[0075] Pre-training: Usually unsupervised, generally using a large amount of unlabeled data to perform unsupervised pre-training on the model.

[0076] Supervised Fine Tuning (SFT): Supervised fine tuning is performed on a pre-trained model. Typically, the model is pre-trained using a large unlabeled dataset, and then supervised fine-tuned using a labeled dataset related to the target task. In supervised fine-tuning, the model is trained in a supervised learning manner. This means that the parameters of the model are adjusted using labeled input samples and the corresponding expected outputs. The goal is to adapt the model to a specific task by fine-tuning the weights and parameters of the model on the specific task.

[0077] Retrieval-Augmented Generation (RAG): is a concept that provides external knowledge sources to large models. It enables large language models to generate accurate, contextual answers that are based on knowledge bases to reduce model hallucinations. The general approach to retrieval-augmented generation is to connect an external knowledge base to an existing large model, so that the content of the answers given by the large model after learning the knowledge base is more relevant to the knowledge base and closer to actual business scenarios.

[0078] Chain-of-Thought (COT) is an improved prompt word technology that requires the model to gradually decompose a complex problem into sub-problems and solve them one by one, thereby enhancing the model's common sense and reasoning ability.

[0079] By acquiring and updating security knowledge from external databases, it is challenging to apply. This is mainly manifested in that professional vocabulary and sentences appear more frequently in the context of specific fields. The distribution of their vocabulary is different from the general vocabulary distribution used in the training phase of the pre-trained model. How to efficiently and accurately retrieve relevant information in the knowledge base and how to make the large model understand user questions and generate effective content under the premise of limited context length are difficult problems in the application of large models. This requires not only an efficient retrieval model and a large language model, but also reasonable representation and indexing of queries and documents. The question-and-answer method in related technologies is costly to implement, has poor transferability, and has limited improvement in the overall evaluation effect.

[0080] Therefore, based on the above problems, the inventors considered how to rewrite questions, how to process the rewritten questions, and how to link with other parts to obtain accurate answers to questions. The disclosed embodiment shows a question-answering method and related equipment based on a large language model, the method comprising: obtaining the original question, obtaining a plurality of thinking chain prompts in different directions, rewriting the original question according to each thinking chain prompt, obtaining a plurality of rewritten questions in different directions, respectively vectorizing the original question, the first rewritten question, and the plurality of second rewritten questions, and fusing them to obtain a fused vector, and generating answers to questions based on the vector representation, the fused vector, and the question-answering large language model. The output of the rewritten question is optimized using thinking chain technology, and is processed through thinking chains in multiple directions, and finally fused to improve the accuracy of question-answering answer retrieval, while retaining the original question, considering different angles, topics, and contextual information that may appear in the question, and improving the recall rate.

[0081] The present exemplary implementation is described in detail below with reference to the accompanying drawings and embodiments.

[0082] First, an embodiment of the present disclosure provides a question-answering method based on a large language model. The method can be executed by any electronic device with computing and processing capabilities. In the following process, the electronic device is taken as a server as an example.

[0083] Figure 1 A flowchart of a question-answering method based on a large language model in an embodiment of the present disclosure is shown. Figure 1 As shown, the question answering method based on a large language model provided in the embodiment of the present disclosure includes the following steps:

[0084] S102: Obtain the original question.

[0085] In a possible embodiment, the original question may be a natural language query question provided by the user. For example, q0 represents the original question, which may be: requiring to monitor the information of all persons entering the room, and to issue early warnings to those who often drink and cause trouble.

[0086] S104: Obtain multiple thought chain prompts in different directions by rewriting the large language model, rewrite the original question according to each thought chain prompt, and obtain multiple rewritten questions in different directions.

[0087] In a possible embodiment, the multiple different directions in S104 refer to multiple angles of thought chain prompts.

[0088] Use COT to reason about the intermediate steps, and give rewriting questions by commanding the large language model to think in different ways. Design appropriate thinking chain prompts and decompose them into multiple intermediate steps according to the problem attributes and rewriting task requirements.

[0089] S106: The original problem and the rewritten problems in different directions are represented by vectors respectively, and fused to obtain a fused vector.

[0090] S108: Generate answers to questions based on the vector representation, fusion vectors, and the question-answering language model.

[0091] In a possible embodiment, the method of retrieving knowledge and generating answers to questions may include: performing similarity calculations in a knowledge base based on vector representations and fusion vectors, retrieving multiple pieces of knowledge, inputting the multiple pieces of knowledge and conversation memory data into a question-and-answer large language model, and generating answers to questions.

[0092] Furthermore, after obtaining a large amount of similar knowledge through similarity calculation, the score of each similar knowledge can be calculated and sorted, and the K pieces with the highest scores can be used as the output multiple pieces of knowledge. The multiple pieces of knowledge and conversation memory data can be input into the question-answering language model for retrieval to generate answers to questions.

[0093] Through the above method, the thought chain technology is used to optimize the output of the rewritten question, and it is processed through thought chains in multiple directions, and finally integrated to improve the accuracy of question and answer retrieval. On the basis of retaining the original question, different angles, topics and contextual information that may appear in the question are considered to improve the recall rate.

[0094] In a possible embodiment, multiple chains of thinking prompts can be used to rewrite the problem in multiple directions. The embodiment of the present disclosure shows thinking chain prompts in two directions. There may be other ways to rewrite the original problem at the same time, which will not be repeated here.

[0095] Figure 2 A flowchart of a rewriting problem in an embodiment of the present disclosure is shown. Figure 2 As shown, the following steps are included:

[0096] S202: Obtain a first thought chain prompt through a first rewriting large language model, perform a first rewriting process on the original question, and identify the user's intention to obtain a first rewritten question.

[0097] S204: Obtain a second chain of thought prompts through a second rewriting large language model to perform a second rewriting process on the original question to obtain second rewriting questions under multiple different subject scenarios, wherein the first rewriting process and the second rewriting process are parallel processes.

[0098] Among them, for the first thought chain prompt, the user's question can be rewritten using session memory to obtain a more complete and accurate rewritten question. At the same time, the prompt also commands the rewriting of the large language model to extract the user's intention.

[0099] Among them, for the second thinking chain prompt, you can ask to generate multiple rewriting questions with different theme scenarios to avoid the problem of homogeneity.

[0100] Through the above method, the large language model is rewritten and the questions are rewritten in parallel, which improves the recall rate while balancing the retrieval cost. Each request passes through two models in parallel during the rewriting phase, which will not cause very high reasoning costs and query link delays. It can also obtain multiple rewritten questions from different perspectives and balance the retrieval cost.

[0101] In one possible embodiment, Figure 3 A flowchart of obtaining a first rewriting problem in an embodiment of the present disclosure is shown, including the following steps:

[0102] S302: Acquire session memory data from the session memory module.

[0103] S304: Generate a first thinking chain prompt according to the pre-configured first prompt information, the session memory data and the preset output format of the first rewriting question.

[0104] S306: Obtain the first thought chain prompt and the original question through the first rewriting large language model with a small parameter amount, identify the user intention and rewrite the original question to obtain the first rewritten question.

[0105] The session memory module can be pre-built, and the creation method can include: creating one or more context session variables for storing the context information of the conversation. The variable object can be a dictionary containing key-value pairs related to the conversation, such as "user_query", "LLM_answer", etc., that is, "user-question" and "question-answer large language model-question answer".

[0106] Exemplarily, the first thought chain prompt may include: system prompt (system_prompt) "You are good at identifying intent and rewriting the user's original question. If the user's question is incomplete, contains pronouns, etc., infer based on the following information (session memory data) and rewrite the user's question to make it more complete and specific, while extracting the user's intent."

[0107] Furthermore, other restrictions can be imposed to make the output of the rewritten large language model more convenient to use. For example, the first thought chain prompt can also include: Attention, please think step by step. Directly output the final rewritten question and the extracted user intent. Prohibit the output of your reasoning process. Do not directly respond to the user's question.

[0108] Furthermore, the output format may be limited, such as "rewrite question, user intent, input", etc. The output format is not limited to what is included.

[0109] Exemplarily, the first rewriting problem can be expressed as: new , then q new It may include: Question: The user hopes to have a product that can monitor the information of all people entering, and can identify those who often drink and cause trouble and issue early warnings; User intent: product recommendation.

[0110] In one possible embodiment, Figure 4 A flowchart of obtaining a second rewriting problem in an embodiment of the present disclosure is shown, including the following steps:

[0111] S402: Generate a second thought chain prompt according to the preset second prompt information, where the prompt words are used for the second rewriting processing prompt to rewrite the original question into a plurality of different subject scenarios.

[0112] S404: Obtain the second thought chain prompt and the original question through the second rewriting large language model with a small parameter amount, rewrite the original question into multiple different subject scenarios, and obtain multiple second rewritten questions.

[0113] Exemplarily, the second thought chain prompt may include: system prompt (system_prompt) "You are good at rephrasing the user's original question. Rewrite the user's original question from different, possible subject usage scenarios.

[0114] You can also add prompt words, for example, note: Please think step by step. Directly output the final rewritten rewrite question. Do not output your reasoning process. Do not directly respond to user questions.

[0115] Furthermore, the output format may be limited: "Rewrite question 1; rewrite question 2; rewrite question 3, input".

[0116] For example, multiple second rewriting questions may be output through the second thought prompt chain. The specific number may be limited in the prompt. If not limited, the number will be random. For example, if the second thought prompt chain outputs three rewriting questions, q1, q2, and q3 represent rewriting question 1, rewriting question 2, and rewriting question 3.

[0117] Q1 may include: How to monitor the information of all people entering and provide early warning to those who often drink and cause trouble?

[0118] Q2 may include: What methods are there to record and analyze the information of people entering the venue and issue early warnings when people who often drink and cause trouble are detected?

[0119] Q3 may include: How to set up a system that can monitor the information of all people entering and leaving, and identify and warn those who often cause drunken nuisances?

[0120] Through the above method, the parallel rewriting of the session memory module and the thought chain technology is combined. The COT method is used to optimize the rewriting of the large language model output during reasoning. In this technology, the original question of the user is not directly input to the retriever in the traditional way. Instead, the rewriting LLM is used to generate a rewritten and standardized rewritten question through the COT method, adding intent recognition and context information. This helps the retriever recall the product information that is most likely to be needed by the user for security knowledge that is more likely to generate correct evaluation results. At the same time, each request passes through two models in parallel during the rewriting stage, and there will be no very large reasoning cost and query link delay problems, which balances the retrieval cost.

[0121] In a possible embodiment, in S106, the rewriting problem in two directions may be taken as an example to perform fusion to obtain a fusion vector.

[0122] The original problem and the rewriting problems in multiple directions are respectively represented by vectors, and one or more vector representations may be used. The vector representations may include multiple types, and two are used as examples in the embodiments of the present disclosure, for example, dense vector representation and sparse vector representation. In the following embodiments, the rewriting problems are taken as the first rewriting problem and the second rewriting problem as examples.

[0123] Figure 5 A flowchart of dense vector fusion in an embodiment of the present disclosure is shown. Figure 5 As shown, the following steps are included:

[0124] S502: Perform dense vector representation on the original problem, the first rewriting problem, and the plurality of second rewriting problems to obtain an original dense vector, a first dense vector, and a plurality of second dense vectors.

[0125] In a possible embodiment, q0, q new , q1, q2 and q3 are represented by dense vectors and can be processed using a dense vector model to obtain a high-dimensional vector.

[0126] The original dense vector, the first dense vector, and the multiple second dense vectors can be expressed as: V(q0), V(q new ), V(q1), V(q2) and V(q3).

[0127] S504: Determine an average dense vector of the plurality of second dense vectors.

[0128] In one possible embodiment, the average of all the second dense vectors V(q1), V(q2) and V(q3) is taken to obtain an average vector for representing the overall semantic information of the second rewriting problem rewritten through the second rewriting large language model and the second thinking chain prompt, that is, the average dense vector.

[0129] In another possible embodiment, the average dense vector may be determined using formula (1), as follows:

[0130]

[0131] Among them, V avg (q) represents the average dense vector; V(q k ) represents the kth second dense vector, taking k=3 as an example.

[0132] S506: Fuse the original dense vector with the average dense vector to obtain a first fused vector.

[0133] S508: Fusing the first dense vector with the average dense vector to obtain a second fused vector.

[0134] In a possible embodiment, a fusion method of S506 and S508 may be adopted, or other fusion methods may be adopted. The specific fusion method is not limited in the embodiments of the present disclosure.

[0135] V(q0) and V(q new ) and V avg (q) The purpose of this step is to combine the original semantic information of the original question with the contextual information of the generated topic query to obtain a more accurate query expression.

[0136] The first fusion vector and the second fusion vector are determined by formulas (2) and (3), as shown below:

[0137] V fusion (q0) = α·V(q0) + β·V avg (q) Formula (2)

[0138] V fusion (q i )=α·V(q new )+β·V avg (q) Formula (3)

[0139] Among them, V fusion (q0) represents the first fusion vector; V fusion (q i ) represents the second fusion vector; α and β represent adjustment parameters, which are used to balance the weight between the original dense vector and the average dense vector of the retrieval results.

[0140] Figure 6 A flow chart of sparse vector fusion in an embodiment of the present disclosure is shown. Figure 6 As shown, the following steps are included:

[0141] S602: Perform sparse vector representation on the original problem, the first rewritten problem, and the plurality of second rewritten problems to obtain an original sparse vector, a first sparse vector, and a plurality of second sparse vectors.

[0142] In a possible embodiment, a bag-of-words model may be used to represent the original question, the first rewritten question, and the plurality of second rewritten questions in a sparse vector form.

[0143] The original sparse vector can be expressed as: LS(w|q0), the first sparse vector can be expressed as: LS(w|q new ).

[0144] The second sparse vectors take three as an example, and are expressed as: LS(w|D qi ), i=1, 2, 3.

[0145] S604: Fusing the original sparse vector with the multiple second sparse vectors to obtain a third fused vector.

[0146] S606: Fusing the first sparse vector with the plurality of second sparse vectors to obtain a fourth fused vector.

[0147] The third fusion vector and the fourth fusion vector can be determined by the following formulas (4) and (5), as shown below:

[0148] LS(w∣R0)=β1·LS(w∣q0)+(1-β1)·LS(w∣D qi ) Formula (4)

[0149] LS(w|R new )=β1·LS(w|q new )+(1-β1)·LS(w|D qi ) Formula (5)

[0150] Among them, β1 represents the adjustment parameter, LS(w|R0) represents the third fusion vector; LS(w|R new ) represents the fourth fused vector.

[0151] Among them, w represents a word in the input text or vector, w∈W θ , W θ Represents a feature set; LS(w|R0) represents the feature set W θ The original sparse representation is determined on the feature dimension w of .

[0152] Through step S108, it is possible to adopt Figure 5 or Figure 6 One of the implementations in the above example can be used to retrieve the answer to the question. Figure 5 and Figure 6to retrieve the answer to the question.

[0153] Figure 7 A flow chart of a hybrid search question answer in an embodiment of the present disclosure is shown. Figure 7 As shown, the following steps are included:

[0154] S702: Perform similarity calculation in the knowledge base according to the original dense vector, the first dense vector, the first fused vector, and the second fused vector, and retrieve multiple first knowledge items sorted by similarity scores.

[0155] S704: Perform similarity calculation in the knowledge base according to the original sparse vector, the first sparse vector, the third fused vector, and the fourth fused vector, and retrieve multiple pieces of second knowledge sorted by similarity scores.

[0156] S706: Determine a combined score of each piece of knowledge according to the ranking of each piece of knowledge in the first knowledge and the second knowledge.

[0157] In a possible embodiment, the first knowledge and the second knowledge retrieved by S702 and S704 can be retrieved in parallel, and the ranking scores of the retrieval results obtained by the two similarity calculations performed in parallel can be merged, and the n retrieval results obtained by the retrieval strategy of each knowledge document can be sorted by relevance, and the inverse ranking scores of different retrieval strategies are added to generate a combined score for each document.

[0158] The combined score can be determined by formula (6) as follows:

[0159]

[0160] Among them, d represents any piece of knowledge, D represents the knowledge base, k is a constant, r1(d) represents the position of knowledge d in the first knowledge, and r2(d) represents the position of knowledge d in the second knowledge; α1 is a custom weight. For the same piece of knowledge, the weight in the ranking score calculation of the first knowledge is α1, and the weight in the ranking score calculation of the second knowledge is (1-α1).

[0161] S708: Sort by combining the scores to obtain the third piece of knowledge ranked in a preset front position.

[0162] In a possible embodiment, the ranking and sorting are performed according to the combined scores. The obtained list is the fusion ranking, and the K pieces of knowledge with the highest combined scores are selected, and then rearranged through the rerank model to obtain the final ranking, and the third piece of knowledge ranked in the front preset position is obtained.

[0163] S710: Input the third knowledge and conversation memory data into the question-answering language model to generate answers to questions.

[0164] In a possible embodiment, the third knowledge is selected and handed over to the big model to generate the final answer, while interacting with the conversation memory module. The conversation memory data in the conversation memory module is used as context input to the big model through historical memory and recalled knowledge documents. After the big model generates the answer to the question, it extracts the information of this round of dialogue and stores it in the conversation memory data.

[0165] In the above way, the original questions are rewritten through group fusion, and the hybrid retrieval is rearranged. On the basis of retaining the original questions, different angles, topics and contextual information that may appear in the questions are considered. Fusion operations are performed on dense and sparse retrieval respectively. For the ranking after hybrid retrieval, an improved inverse sorting fusion method is proposed, which helps to improve the recall rate, thus providing strong support for the practical application in the field of multi-round open question answering.

[0166] In the above method, the rewriting large language model, the retriever in the retrieval process, and the question-answering large language model are used. For the rewriting large language model, since the rewriting method selected in the present disclosure is different from the conventional method, it needs to be trained and tested through supervised fine-tuning before it can be used. For the retriever in the retrieval process and the question-answering large language model, mature models can be used directly.

[0167] In one possible embodiment, Figure 8 A flowchart of a rewriting large language model training process in an embodiment of the present disclosure is shown. Figure 8 As shown, the following steps are included:

[0168] S802: Construct a training data set.

[0169] In a possible embodiment, user questions and contexts in real question-answering scenarios are collected to obtain a user question set Q, an excellent model interface in the relevant technology is selected to rewrite the user question set Q to obtain question pairs Q_r, and question pairs and corresponding questions that cannot be recalled to the correct knowledge blocks are deleted to obtain the final training data set Q1.

[0170] S804: Based on supervised fine-tuning, an untrained rewritten large language model with a small number of parameters is trained according to the training data set to obtain a rewritten large language model that meets preset conditions.

[0171] In a possible embodiment, the purpose of fine-tuning training is to let the rewriting large language model learn how to generate corresponding rewriting questions given the original question. The rewriting question may include rewriting questions in multiple different directions. Different fine-tuning training can be performed for rewriting in different directions, or one fine-tuning method can be used for training. The ultimate goal is to output the answer to the question given the original question, and the recall rate or precision reaches the set threshold.

[0172] The training goal of the rewriting large language model is to maximize the probability of the generated rewriting question, which is expressed by the following formula (7). The loss function formula is as follows:

[0173]

[0174] in, Represents the loss function value; represents the original question q in the input training sample, and the first t-1 question pairs Under the condition of probability.

[0175] Through the above training, the rewriting large language model can initially learn how to generate adapted rewriting questions from the original questions.

[0176] S806: Test the rewritten large language model to obtain a test result output by the question-answering large language model.

[0177] S808: Evaluate according to the test results. If the precision or recall rate of the test results meets a preset threshold, a trained rewritten large language model is obtained.

[0178] In a possible embodiment, after the pre-training is completed, the rewritten large language model is applied to the test set to evaluate the impact of the rewritten large language model on the output of the question-answering large language model.

[0179] The rewriting questions generated by the rewriting language model are input into the downstream retriever and question-answering language model, and the output of the question-answering language model is used as the final evaluation standard. When the test results output by the question-answering language model and the precision or recall rate of the evaluation test results reach the set threshold, the fine-tuning of the rewriting language model is completed.

[0180] In the above training process, the retriever feedback supervision signal can be used to assist in rewriting the large language model for fine-tuning. At the same time, the COT method can be used to optimize and rewrite the output of the large language model during inference.

[0181] Fig. 9 FIG. 4 shows an architecture diagram of a question-answering system in an embodiment of the present disclosure. Fig. 9 As shown, the question-answering system 900 includes: a product knowledge base 901 , a rewriting module 902 , a retriever 903 , a sorting module 904 , an answer generation module 905 and a conversation memory module 906 .

[0182] Among them, the product knowledge base 901 contains a large amount of information data of products in specified business scenarios.

[0183] The rewriting module 902 includes a rewriting large language model with a small number of parameters, which inputs the memory context and the user's original, and generates a rewritten and standardized rewriting question through the COT method. In order to better match the domain knowledge base, supervised fine-tuning is performed before use.

[0184] Retriever 903 is composed of an embedding model, a vector database, and a series of similarity algorithms. The embedding model is used to map text into vectors in a specific semantic space. The vector database is used to store the semantic vectors output by the embedding model and give indexes. The similarity algorithm supports multiple vector representations such as dense vectors or sparse vectors, calculates similarities, and thus recalls relevant knowledge text blocks.

[0185] The ranking module 904 includes a fusion ranking mechanism and a re-ranking model.

[0186] The answer generation module 905 includes a question-answering large language model, which has the functions of understanding, reasoning, and generating human language, can solve various natural language tasks, and can use existing pre-trained general large language models.

[0187] The conversation memory module 906 introduces historical memory to interact with the rewriting module 902 and the answer generation module 905 in the system.

[0188] In a possible embodiment, the question-answering method in the present disclosure can be used as an assistant for segmented professional scenarios, such as a business opportunity consulting assistant, or product matching, etc.

[0189] Under the technical solution of knowledge base mounting, users can update product knowledge in a timely and fast manner. There is no need to adjust the retriever and the large question-and-answer language model, which is conducive to the rapid migration of the business opportunity assistant to different branch businesses. It has the function of context memory to achieve multi-round memory and intelligent question-and-answer functions. The retriever feedback supervision signal is used to assist in rewriting the large language model fine-tuning, and the COT method is used to optimize and rewrite the output of the large language model during reasoning.

[0190] Based on the same inventive concept, the present disclosure also provides a question-answering device based on a large language model, such as the following embodiment. Since the principle of solving the problem in the device embodiment is similar to that in the above method embodiment, the implementation of the device embodiment can refer to the implementation of the above method embodiment, and the repeated parts will not be repeated.

[0191] Fig.10 A structural diagram of a question-answering device based on a large language model in an embodiment of the present disclosure is shown. Fig.10As shown, the device 100 includes: an acquisition unit 1001, used to acquire the original question; a rewriting unit 1002, used to acquire a plurality of thinking chain prompts in different directions, and rewrite the original question according to each thinking chain prompt to obtain a plurality of rewritten questions in different directions; a fusion unit 1003, used to respectively vectorize the original question, the first rewritten question and a plurality of second rewritten questions, and fuse them to obtain a fused vector; a generation unit 1004, used to generate an answer to the question according to the vector representation, the fused vector and the question-answering large language model.

[0192] Those skilled in the art will appreciate that various aspects of the present disclosure may be implemented as systems, methods or program products. Therefore, various aspects of the present disclosure may be specifically implemented in the following forms, namely: complete hardware implementation, complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software, which may be collectively referred to herein as "circuits", "modules" or "systems".

[0193] Refer to the following Fig.11 1100 according to this embodiment of the present disclosure is described. Fig.11 The electronic device 1100 shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0194] like Fig.11 As shown, the electronic device 1100 is in the form of a general computing device. The components of the electronic device 1100 may include but are not limited to: at least one processing unit 1110, at least one storage unit 1120, and a bus 1130 connecting different system components (including the storage unit 1120 and the processing unit 1110).

[0195] The storage unit stores program codes, which can be executed by the processing unit 1110, so that the processing unit 1110 executes the steps of various exemplary embodiments of the present disclosure described in the above “Exemplary Method” section of this specification. For example, the processing unit 1110 can execute the steps of any one of the above method embodiments.

[0196] The storage unit 1120 may include a readable medium in the form of a volatile storage unit, such as a random access storage unit (RAM) 11201 and / or a cache storage unit 11202 , and may further include a read-only storage unit (ROM) 11203 .

[0197] The storage unit 1120 may also include a program / utility 11204 having a set (at least one) of program modules 11205, such program modules 11205 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0198] Bus 1130 may represent one or more of several types of bus structures, including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.

[0199] The electronic device 1100 may also communicate with one or more external devices 1140 (e.g., keyboards, pointing devices, Bluetooth devices, etc.), may also communicate with one or more devices that enable a user to interact with the electronic device 1100, and / or communicate with any device that enables the electronic device 1100 to communicate with one or more other computing devices (e.g., routers, modems, etc.). Such communication may be performed via an input / output (I / O) interface 1150. Furthermore, the electronic device 1100 may also communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) via a network adapter 1160. As shown, the network adapter 1160 communicates with other modules of the electronic device 1100 via a bus 1130. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device 1100, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0200] Through the description of the above implementation, it is easy for those skilled in the art to understand that the example implementation described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the implementation of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (which can be a personal computer, a server, a terminal device, or a network device, etc.) to execute the method according to the implementation of the present disclosure.

[0201] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer program product or a computer program, which includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the method in the above embodiment.

[0202] In an exemplary embodiment of the present disclosure, a computer-readable storage medium is also provided, which may be a readable signal medium or a readable storage medium. A program product capable of implementing the above method of the present disclosure is stored thereon. In some possible implementations, various aspects of the present disclosure may also be implemented in the form of a program product, which includes a program code, and when the program product is run on a terminal device, the program code is used to enable the terminal device to execute the steps according to various exemplary implementations of the present disclosure described in the above “Exemplary Method” section of this specification.

[0203] More specific examples of computer-readable storage media in the present disclosure may include, but are not limited to, an electrical connection having one or more conductors, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0204] In the present disclosure, a computer readable storage medium may include a data signal propagated in baseband or as part of a carrier wave, wherein a readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A readable signal medium may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0205] Alternatively, the program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the foregoing.

[0206] In a specific implementation, the program code for performing the operations of the present disclosure may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., using an Internet service provider to connect through the Internet).

[0207] It should be noted that, although several modules or units of the device for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into multiple modules or units to be embodied.

[0208] In addition, although the steps of the method in the present disclosure are described in a specific order in the drawings, this does not require or imply that the steps must be performed in this specific order, or that all the steps shown must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps, etc.

[0209] Through the description of the above implementation, it is easy for those skilled in the art to understand that the example implementation described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the implementation of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (which can be a personal computer, a server, a mobile terminal, or a network device, etc.) to execute the method according to the implementation of the present disclosure.

[0210] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. The present disclosure is intended to cover any variations, uses or adaptations of the present disclosure, which follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The description and examples are intended to be exemplary only, and the true scope and spirit of the present disclosure are indicated by the appended claims.

Claims

1. A question-answering method based on a large language model, characterized in that: The method comprises: Get the original question; Obtain multiple thought chain prompts in different directions by rewriting the large language model, and rewrite the original question in parallel according to each thought chain prompt to obtain multiple rewritten questions in different directions; Representing the original problem and the rewriting problems in different directions as vectors respectively, and fusing them to obtain a fused vector; Generate an answer to the question based on the vector representation, the fused vector and the question-answering large language model.

2. The method according to claim 1, characterized in that The thought chain prompt includes: a first thought chain prompt and a second thought chain prompt; The method obtains multiple thought chain prompts in different directions by rewriting the large language model, and rewrites the original question in parallel according to each thought chain prompt to obtain multiple rewritten questions in different directions, including: Obtain a first thought chain prompt through a first rewriting large language model, perform a first rewriting process on the original question, and identify the user's intention to obtain a first rewritten question; The second thought chain prompt is obtained through the second rewriting large language model to perform a second rewriting process on the original question, so as to obtain second rewriting questions under multiple different subject scenarios; wherein the first rewriting process and the second rewriting process are parallel processes.

3. The method according to claim 2, characterized in that The first rewriting large language model is used to obtain a first thought chain prompt, the original question is subjected to a first rewriting process, and the user intention is identified to obtain a first rewritten question, including: Get session memory data from the session memory module; Generate a first thought chain prompt according to the pre-configured first prompt information, the session memory data and the preset output format of the first rewriting question; The first thought chain prompt and the original question are obtained through a first rewriting large language model with a small amount of parameters, the user intention is identified and the original question is rewritten to obtain a first rewritten question.

4. The method according to claim 2, characterized in that The second rewriting large language model is used to obtain a second chain of thought prompts to perform a second rewriting process on the original question, thereby obtaining a plurality of second rewriting questions in different subject scenarios, including: According to the preset second prompt information, the second thought chain prompt is generated; the prompt word is used to prompt the second rewriting process to rewrite the original question into multiple different subject scenes; The second thought chain prompt and the original question are obtained through a second rewriting large language model with a small amount of parameters, and the original question is rewritten into multiple different subject scenarios to obtain multiple second rewritten questions.

5. The method according to claim 2, characterized in that: The vector representation includes a dense vector representation; the dense vector representation includes: an original dense vector and a first dense vector; The original problem and the rewriting problems in different directions are respectively represented by vectors, and fused to obtain a fused vector, including: Performing dense vector representation on the original problem, the first rewriting problem, and the plurality of second rewriting problems to obtain an original dense vector, a first dense vector, and a plurality of second dense vectors; determining an average dense vector of the plurality of second dense vectors; Fusing the original dense vector with the average dense vector to obtain a first fused vector; The first dense vector is fused with the average dense vector to obtain a second fused vector.

6. The method according to claim 2, characterized in that The vector representation comprises a sparse vector representation; The sparse vector representation includes: an original sparse vector and a first sparse vector; The original problem and the rewriting problems in different directions are respectively represented by vectors, and fused to obtain a fused vector, including: Performing sparse vector representation on the original problem, the first rewriting problem, and the plurality of second rewriting problems to obtain an original sparse vector, a first sparse vector, and a plurality of second sparse vectors; Fusing the original sparse vector with the plurality of second sparse vectors to obtain a third fused vector; The first sparse vector is merged with the plurality of second sparse vectors to obtain a fourth merged vector.

7. The method according to claim 1, characterized in that The step of generating an answer to a question according to the vector representation, the fusion vector and the question-answering large language model comprises: Performing similarity calculation in a knowledge base according to the vector representation and the fusion vector, and retrieving a plurality of pieces of knowledge; The multiple pieces of knowledge and conversation memory data are input into the question-answering large language model to generate answers to questions.

8. The method according to claim 1, characterized in that: The vector representation includes: a dense vector representation and a sparse vector representation; the dense vector representation includes: an original dense vector and a first dense vector; the sparse vector representation includes: an original sparse vector and a first sparse vector; the fused vector includes: a first fused vector and a second fused vector corresponding to the dense vector representation, and a third fused vector and a fourth fused vector corresponding to the sparse vector representation; The step of generating an answer to a question according to the vector representation, the fusion vector and the question-answering large language model comprises: performing similarity calculation in a knowledge base according to the original dense vector, the first dense vector, the first fused vector, and the second fused vector, and retrieving a plurality of first knowledge items sorted by similarity scores; Performing similarity calculation in a knowledge base according to the original sparse vector, the first sparse vector, the third fused vector, and the fourth fused vector, and retrieving a plurality of second knowledge items sorted by similarity scores; Determine a combined score of each piece of knowledge according to the ranking of each piece of knowledge in the first knowledge and the second knowledge; According to the combination score sorting, the third knowledge ranked in the front preset position is obtained; The third knowledge and conversation memory data are input into the question-answering large language model to generate answers to questions.

9. The method according to claim 1, characterized in that: The training process of rewriting the large language model includes: Build a training dataset; Based on supervised fine-tuning, the untrained rewritten large language model with a small number of parameters is trained according to the training data set to obtain a rewritten large language model that meets preset conditions; Testing the rewritten large language model to obtain a test result output by the question-answering large language model; An evaluation is performed based on the test result, and if the precision or recall rate of the test result meets a preset threshold, a trained rewritten large language model is obtained.

10. A question-answering device based on a large language model, characterized in that: include: An acquisition unit, used to acquire the original question; A rewriting unit, used for obtaining multiple thinking chain prompts in different directions, and rewriting the original question according to each thinking chain prompt to obtain multiple rewritten questions in different directions; A fusion unit, used to represent the original problem, the first rewriting problem and the plurality of second rewriting problems in vector form respectively, and to fuse them to obtain a fused vector; A generating unit is used to generate an answer to a question based on the vector representation, the fused vector and the question-answering large language model.

11. An electronic device, characterized in that: include: processor; as well as A memory, configured to store executable instructions of the processor; The processor is configured to perform the method of any one of claims 1 to 9 by executing the executable instructions.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 9 is implemented.

13. A computer program product comprising: A computer program or instruction, characterized in that when the computer program or instruction is executed by a processor, the method described in any one of claims 1 to 9 is implemented.

Images