Question and answer processing method, device and equipment based on artificial intelligence and storage medium

Abstract

The invention relates to the technical field of artificial intelligence, and discloses an artificial intelligence-based question and answer processing method and device, equipment and a storage medium. Through an RAG-based large language model, performing reply processing on the user question according to a preset knowledge base to obtain an answer to the user question; performing ROUGE calculation processing on the answers and the knowledge base to obtain ROUGE scores between the answers and the knowledge base; and carrying out credibility evaluation on the answer according to the ROUGE score. The method can be applied to the question and answer engine in the field of financial science and technology, the response accuracy, reliability and interpretability of the question and answer engine can be effectively improved, and the illusion phenomenon is reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of artificial intelligence, and in particular, to a question-answering processing method, device, equipment, and storage medium based on artificial intelligence. Background Art

[0002] Intelligent question-answering engines simulate the language understanding and reasoning processes of humans to automatically answer natural language questions raised by users, helping users solve problems, and are widely used in various fields. For example, in the field of financial vehicle insurance applications, intelligent question-answering engines such as vehicle management intelligent customer service or vehicle management intelligent assistants are used to provide instant answers to common questions about vehicle insurance services (such as vehicle insurance purchase, query, accident damage assessment, and claims settlement) for vehicle owners.

[0003] Currently, intelligent question-answering engines rely on external knowledge sources to answer questions. Since the information provided by external knowledge sources is incomplete or even incorrect, it causes intelligent question-answering engines to generate some illogical false or incorrect answers, resulting in an hallucination phenomenon. Summary of the Invention

[0004] The present invention provides a question-answering processing method, device, computer equipment, and storage medium based on artificial intelligence to solve the technical problem that the question-answering engine generates illogical false or incorrect answers and has an hallucination phenomenon.

[0005] In a first aspect, a question-answering processing method based on artificial intelligence is provided, including:

[0006] Receiving a user question;

[0007] Using a large language model based on RAG to perform a reply process on the user question according to a preset knowledge base to obtain an answer to the user question;

[0008] Performing ROUGE calculation processing on the answer and the knowledge base to obtain a ROUGE score between the answer and the knowledge base;

[0009] Evaluating the credibility of the answer according to the ROUGE score.

[0010] In a second aspect, a question-answering processing device based on artificial intelligence is provided, including:

[0011] A question receiving module for receiving a user question;

[0012] A question reply module for performing a reply process on the user question according to a preset knowledge base using a large language model based on RAG to obtain an answer to the user question;

[0013] The ROUGE score calculation module is used to perform ROUGE calculation processing on the answer and the knowledge base to obtain the ROUGE score between the answer and the knowledge base;

[0014] The credibility evaluation module is used to evaluate the credibility of the answer according to the ROUGE score.

[0015] In a third aspect, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the above-mentioned artificial intelligence-based question-answering processing method are implemented.

[0016] In a fourth aspect, a computer-readable storage medium is provided. The computer-readable storage medium stores a computer program, and when the computer program is executed by the processor, the steps of the above-mentioned artificial intelligence-based question-answering processing method are implemented.

[0017] In the solutions implemented by the above-mentioned artificial intelligence-based question-answering processing method, device, computer device, and storage medium, a user question is received; through a large language model based on RAG, the user question is answered according to a preset knowledge base to obtain an answer to the user question; ROUGE calculation processing is performed on the answer and the knowledge base to obtain the ROUGE score between the answer and the knowledge base; the credibility of the answer is evaluated according to the ROUGE score. In the present invention, on the one hand, through a large language model based on RAG, combined with a large amount of knowledge in the knowledge base, it is possible to reasonably reason and summarize the answer to the user question, so as to efficiently and accurately generate a high-quality answer that conforms to logic; on the other hand, by calculating the ROUGE score between the answer and the knowledge base, so as to evaluate the credibility of the answer based on the ROUGE score, it is possible to quantify and identify the reliability of the answer, effectively improve the transparency of the answering engine, enhance the interpretability of the answering engine, and reduce the generation of hallucination phenomena. Description of the Drawings

[0018] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following will briefly introduce the drawings required to be used in the description of the embodiments of the present invention. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0019] Figure 1 is a schematic diagram of an application environment of an artificial intelligence-based question-answering processing method in an embodiment of the present invention;

[0020] Figure 2 is a schematic flowchart of an artificial intelligence-based question-answering processing method in an embodiment of the present invention;

[0021] Figure 3 is Figure 2 a schematic flowchart of a specific implementation manner of step S20 in

[0022] Figure 4 is Figure 2 a schematic flowchart of a specific implementation manner of step S30 in

[0023] Figure 5 a schematic structural diagram of a question - answering processing device based on artificial intelligence in an embodiment of the present invention;

[0024] Figure 6 a schematic structural diagram of a computer device in an embodiment of the present invention;

[0025] Figure 7 is another schematic structural diagram of a computer device in an embodiment of the present invention. Specific Embodiments

[0026] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without making creative efforts fall within the protection scope of the present invention.

[0027] The question - answering processing method based on artificial intelligence provided by the embodiments of the present invention can be applied in an application environment such as Figure 1 . Among them, the client communicates with the server through the network. The server can receive user questions through the client; through a large - language model based on RAG, answer the user questions according to a preset knowledge base to obtain the answers to the user questions; perform ROUGE calculation processing on the answers and the knowledge base to obtain the ROUGE score between the answers and the knowledge base; evaluate the credibility of the answers according to the ROUGE score. In this way, on the one hand, through the large - language model based on RAG and combined with a large amount of knowledge in the knowledge base, it is possible to reasonably reason and summarize the answers to user questions, so as to efficiently and accurately generate high - quality answers that conform to logic; on the other hand, by calculating the ROUGE score between the answers and the knowledge base, in order to evaluate the credibility of the answers based on the ROUGE score, it is possible to quantify and identify the reliability of the answers, effectively improve the transparency of the answering engine, enhance the interpretability of the answering engine, and reduce the generation of hallucination phenomena. Among them, the client can be, but is not limited to, various personal computers, laptop computers, smart phones, tablet computers, and portable wearable devices. The server can be implemented by an independent server or a server cluster composed of multiple servers. The present invention will be described in detail through specific embodiments below.

[0028] Please refer to Figure 2 as shown Figure 2 which is a schematic flowchart of a question-answering processing method based on artificial intelligence provided by an embodiment of the present invention, including the following steps:

[0029] S10: Receive a user's question.

[0030] The question-answering processing method based on artificial intelligence provided by the present invention can be applied to intelligent question-answering engines such as intelligent customer service or intelligent assistants in various application scenarios. It can effectively improve the response accuracy, reliability, and interpretability of the intelligent question-answering engine based on artificial intelligence technology and natural language processing technology, and reduce the hallucination phenomenon.

[0031] For ease of understanding, first, the terms related to the present invention are explained:

[0032] Artificial intelligence (AI): It is a new technical science that studies, develops theories, methods, technologies, and application systems for simulating, extending, and expanding human intelligence; artificial intelligence is a branch of computer science. Artificial intelligence attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a way similar to human intelligence. The research in this field includes robots, speech recognition, image recognition, natural language processing, and expert systems, etc. Artificial intelligence can simulate the information process of human consciousness and thinking. Artificial intelligence also uses digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use knowledge to obtain the best results in terms of theories, methods, technologies, and application systems.

[0033] Natural language processing (NLP): NLP uses computers to process, understand, and apply human languages (such as Chinese, English, etc.). NLP belongs to a branch of artificial intelligence and is an interdisciplinary subject of computer science and linguistics, and is often referred to as computational linguistics. Natural language processing includes syntactic analysis, semantic analysis, discourse understanding, etc. Natural language processing is commonly used in technical fields such as machine translation, handwritten and printed character recognition, speech recognition and text-to-speech conversion, information intention recognition, information extraction and filtering, text classification and clustering, public opinion analysis, and opinion mining. It involves data mining, machine learning, knowledge acquisition, knowledge engineering, artificial intelligence research related to language processing, and linguistic research related to language computing.

[0034] Retrieval-Augmented Generation (RAG): It is an architecture that combines information retrieval with a generation model, aiming to improve the performance of large language models in generation tasks. Especially when answering questions or generating content, it can better utilize the knowledge in external knowledge bases.

[0035] Large language models based on RAG: Refer to large language models that have been pre-trained to combine information retrieval and generation capabilities. Examples of large language models based on RAG include the pre-trained REALM model (using the Transformer architecture of RAG), the Fusion-in-Decoder (FiD) model, the KILT framework model, etc.

[0036] Preset knowledge base: Refers to an external knowledge base that provides reference knowledge documents for generating answers to user questions by large language models based on RAG. There is no limit to the number of preset knowledge bases. It can be one. In order to enhance the comprehensiveness and richness of reference knowledge documents, it can also be multiple.

[0037] Recall-Oriented Understudy for Gisting Evaluation (ROUGE): It is an indicator used to measure the quality of machine-generated text, mainly used to quantitatively evaluate the similarity between machine-generated text and reference text.

[0038] Based on this, the following will detail the artificial intelligence-based question-and-answer processing method provided by the present invention.

[0039] The intelligent question-and-answer engine is usually implemented through a server, which can receive user questions in real time. For example, in the field of financial vehicle insurance applications, users such as car owners often ask questions by phone or online chat. It is necessary to rely on intelligent question-and-answer engines such as vehicle management intelligent customer service or vehicle management intelligent assistants to reply to some vehicle insurance questions of customers in order to provide users with efficient and high-quality vehicle insurance management and service solutions and improve the user experience.

[0040] Exemplarily, user questions are such as "How to settle vehicle insurance premiums in advance?" etc.

[0041] S20: Through a large language model based on RAG, reply to the user question according to the preset knowledge base to obtain the answer to the user question.

[0042] After receiving the user question, the intelligent question-and-answer engine uses a large language model based on RAG to reply to the user question according to a large number of knowledge documents provided by the preset knowledge base, so as to efficiently and accurately generate the answer to the user question.

[0043] Please refer toFigure 3 , in some embodiments, step S20 may include but is not limited to the following steps:

[0044] S21: Use a RAG-based large language model to retrieve and process the user's question according to the knowledge base to obtain the knowledge documents related to the user's question.

[0045] S22: Use a RAG-based large language model to generate an answer based on the user's question and the knowledge documents to obtain the answer.

[0046] For steps S21 - S22, first use a RAG-based large language model to retrieve and process the user's question according to the knowledge base to obtain the knowledge documents related to the user's question. The number of knowledge documents related to the user's question can be one or multiple. Then input the user's question and the knowledge documents into the RAG-based large language model together, so that the RAG-based large language model can perform answer generation processing to obtain the answer to the user's question.

[0047] In this way, by using a RAG-based large language model, first retrieve the knowledge documents related to the user's question from the knowledge base, and then fuse and input the user's question and the knowledge documents into the RAG-based large language model, so that the RAG-based large language model can effectively utilize the information in the knowledge base, reasonably reason and summarize the answer to the user's question, make up for the defect of the inherent lack of knowledge of the RAG-based large language model, and thus generate a high-quality answer that conforms to logic.

[0048] In step S21 of some embodiments, a RAG-based large language model can be used to perform semantic recognition processing on the user's question to obtain the semantic vector of the user's question; use a RAG-based large language model to perform semantic similarity retrieval processing in the knowledge base according to the semantic vector to obtain the knowledge documents.

[0049] In step S21, in order to convert the user's question into a form that can be understood and processed by the RAG-based large language model, a RAG-based large language model can be used to first perform semantic recognition processing on the user's question to obtain the semantic vector of the user's question, where the semantic vector can represent the semantic information of the user's question.

[0050] Then use a RAG-based large language model to perform semantic similarity retrieval in the knowledge base by means of ElasticSearch or approximate nearest neighbor (ANN) search, so as to retrieve the knowledge documents that are as semantically relevant to the user's question as possible, ensure that the retrieved knowledge documents are most helpful to the user's question, and at the same time improve the speed and efficiency of the retrieval.

[0051] In this way, it is possible to quickly and effectively retrieve the knowledge document in the knowledge base that is most semantically relevant to the user's question, providing an accurate, comprehensive, and rich reference basis for the large language model based on RAG to generate user answers.

[0052] In step S22 of some embodiments, the user question and the knowledge document can be fused to obtain context information; the context information is input into the large language model based on RAG; the context information is encoded to obtain context vectors; through the attention mechanism, autoregressive answer generation processing is performed on the context vectors to obtain answers.

[0053] In step S22, the user question and the knowledge document can be fused to obtain context information. For example, the user question and the knowledge document are concatenated to obtain the context information of the long text, ensuring that all information of the user question and the knowledge document can be considered simultaneously during the process of the large language model based on RAG generating answers.

[0054] Taking the context information as the input of the large language model based on RAG, the large language model based on RAG will encode the context information to obtain context vectors, and then use the attention mechanism to capture the global dependencies between different word vectors in the context vectors to obtain global dependency features. Thus, based on the global dependency features, through the autoregressive generation method, the answer to the user question is generated.

[0055] In this way, the large language model based on RAG not only depends on its own pre-trained knowledge but also can effectively utilize the information of the knowledge document, realizing the effective integration of the semantics of the user question and the semantics of the knowledge document, thereby generating more accurate answers.

[0056] For example: The user asks the vehicle management intelligent customer service in a chat way on the "Vehicle Insurance Butler Platform" the question "How to settle the vehicle insurance premium in advance?" The vehicle management intelligent customer service uses the large language model based on RAG to retrieve from the knowledge base specific to the financial vehicle insurance field the knowledge documents related to the user's question, such as "Standard Operation Guide for Early Settlement of Premiums", "Standard Operation Guide for Early Settlement of Vehicle Insurance Premiums", and "Electronic Signing - Operation Guide";

[0057] "User question + Standard Operation Guide for Early Settlement of Premiums + Standard Operation Guide for Early Settlement of Vehicle Insurance Premiums + Electronic Signing - Operation Guide" is input into the large language model based on RAG for answer generation processing, and the answer to the user question is obtained as follows:

[0058] "The following is the detailed operation guide for early settlement of vehicle insurance premiums:

[0059] ① Preparation matters:

[0060] · Log in to the "Vehicle Insurance Butler Platform" to apply for early settlement and electronically sign the settlement letter.

[0061] · Use the pre-deposited mobile phone number as the login account. For corporate customers, use the mobile phone number of the legal person or authorized person.

[0062] · If it is a corporate customer and the signatory is the legal person or authorized person, the company seal must be completed before electronic signing.

[0063] · If the current legal person of the enterprise is inconsistent with the legal person category recorded in the system, the latest legal person information needs to be updated, otherwise electronic signing may not be possible.

[0064] ② Operation steps:

[0065] · After entering the [My Orders] page, confirm that the amount calculation is correct.

[0066] · After clicking OK, proceed to the next step, then initiate the application and make a payment to the corporate account. Note that customers with overdue payments must initiate the application process on the same day because the penalty amount changes daily.

[0067] · For projects that require paper documents, while uploading the transfer voucher, it is also necessary to upload the letter and fill in the mailing information.

[0068] ③ Approval process:

[0069] · After submitting the application, you can view the approval status on the [My Orders] page.

[0070] · The approval status includes: Approved (waiting for the settlement materials to be mailed), Returned (you can cancel the application or modify the materials and resubmit), Rejected (the project is changed to an applicable status, and it needs to be recalculated and submitted again), Under Approval (the operation, finance, or asset department is reviewing).

[0071] ④ Handling of special situations:

[0072] · If you receive a return notice, you can click [Modify Materials] to view the reason and make corrections according to the instructions before resubmitting.

[0073] · If you decide not to continue with the settlement, you can directly click [Cancel Application] at the top of the page.

[0074] Please note that the specific operation steps and interfaces may change. It is recommended to check the latest operation guide at any time to ensure accuracy. In addition, if you encounter any technical problems, you can obtain support through the help center within the platform or contact the customer service staff.

[0075] Thus, detailed and accurate answers to user questions can be provided immediately.

[0076] S30: Perform ROUGE calculation and processing on the answer and the knowledge base to obtain the ROUGE score between the answer and the knowledge base.

[0077] After generating an answer to the user's question using a large language model based on RAG, perform ROUGE calculation and processing on the answer and the knowledge base to obtain the ROUGE score between the answer and the knowledge base, so as to quantify the similarity between the answer and the knowledge base.

[0078] Please refer to Figure 4 , in some embodiments, step S30 may include but is not limited to the following steps:

[0079] S31: Calculate the precision and recall of the answer based on the answer and the knowledge document.

[0080] S32: Perform ROUGE calculation and processing based on the precision and recall to obtain the ROUGE score.

[0081] For steps S31 - S32, in order to facilitate the ROUGE calculation and processing of the answer and the knowledge base, the precision and recall of the answer can be calculated first based on the answer and the knowledge document involved in the user's question, and then ROUGE calculation and processing can be performed based on the precision and recall to obtain the ROUGE score, which is used as the ROUGE score between the answer and the knowledge base.

[0082] Among them, the formula for calculating the ROUGE score is as follows:

[0083]

[0084] Among them, s represents the ROUGE score, P represents precision, and R represents recall.

[0085] In this way, using the precision and recall of the answer as indicators to calculate the ROUGE score can significantly improve the calculation speed.

[0086] In step S31 of some embodiments, the length of the longest common subsequence between the knowledge document and the answer can be calculated through common subsequence length calculation and processing of the knowledge document and the answer; based on the length and the total number of words in the answer, precision calculation and processing of the answer can be performed to obtain the precision; based on the length and the total number of words in the knowledge document, recall calculation and processing of the answer can be performed to obtain the recall.

[0087] In step S31, the precision and recall of the answer can be calculated based on the longest common subsequence (LCS) between the knowledge document and the answer.

[0088] Specifically, perform a calculation process for the common subsequence length of the knowledge document and the answer to obtain the length of the longest common subsequence between the knowledge document and the answer, denoted as LCS(M, N).

[0089] Then, based on the length of the longest common subsequence and the total number of words in the answer, perform a precision calculation process on the answer to obtain the precision of the answer. The calculation formula is as follows:

[0090]

[0091] Among them, M represents the knowledge document, N represents the answer, and |N| represents the total number of words in the answer.

[0092] It is also possible to perform a recall calculation process on the answer based on the length of the longest common subsequence and the total number of words in the knowledge document to obtain the recall of the answer. The calculation formula is as follows:

[0093]

[0094] Among them, |M| represents the total number of words in the knowledge document.

[0095] In this way, calculating the precision and recall of the answer based on the longest common subsequence between the knowledge document and the answer can improve the calculation speed and ensure the accuracy of the precision and recall at the same time.

[0096] In step S31 of some embodiments, it is possible to perform a calculation process for the number of shared n-grams of the knowledge document and the answer to obtain the number of shared n-grams between the knowledge document and the answer; based on the number of shared n-grams and the number of n-grams of the answer, perform a precision calculation process on the answer to obtain the precision; based on the number of shared n-grams and the number of n-grams of the knowledge document, perform a recall calculation process on the answer to obtain the recall.

[0097] In step S31, it is possible to calculate the precision and recall of the answer based on the number of shared n-grams between the knowledge document and the answer, where n represents the size of the n-grams to be calculated.

[0098] Specifically, perform a calculation process for the number of shared n-grams of the knowledge document and the answer to obtain the number of shared n-grams between the knowledge document and the answer, denoted as |M∩N|.

[0099] Then, based on the number of shared n-grams and the number of n-grams of the answer, perform a precision calculation process on the answer to obtain the precision of the answer. The calculation formula is as follows:

[0100]

[0101] Among them, |N′| represents the number of n-grams in the answer.

[0102] It is also possible to calculate the recall rate of the answer based on the number of shared n-grams and the number of n-grams in the knowledge document, and obtain the recall rate of the answer. The calculation formula is as follows:

[0103]

[0104] Among them, |M′| represents the number of n-grams in the knowledge document.

[0105] In this way, calculating the precision rate and recall rate of the answer based on the number of shared n-grams between the knowledge document and the answer can improve the calculation speed and ensure the accuracy of the precision rate and recall rate.

[0106] S40: Evaluate the credibility of the answer according to the ROUGE score.

[0107] Finally, evaluate the credibility of the generated answer according to the ROUGE score between the generated answer and the knowledge base.

[0108] For example, compare the ROUGE score between the generated answer and the knowledge base with a preset score threshold. When the ROUGE score between the generated answer and the knowledge base is greater than or equal to the score threshold, it means that the generated answer is highly consistent with the information in the knowledge base, and it is determined that the credibility of the answer is credible; when the ROUGE score between the generated answer and the knowledge base is less than the score threshold, it means that the generated answer is logically inconsistent with the information in the knowledge base, and it is determined that the credibility of the answer is not credible.

[0109] In this way, quantifying and evaluating the credibility of the answer through the ROUGE score makes the response of the intelligent question-answering engine more transparent, enhances the interpretability of the generated answer, thereby improving the reliability of the answer, avoiding incorrect or false answers, and helping to eliminate the hallucination phenomenon.

[0110] After step S30 of some embodiments, the knowledge base can also be ROUGE-tagged.

[0111] For example, assume there are i knowledge bases, denoted as k_i, the answer generated by the RAG-based large language model is denoted as N, and the ROUGE score between the answer and each knowledge base is denoted as s_i = rouge(k_i, N). The s_i = rouge(k_i, N) of each knowledge base name can be marked after the generated answer, and distinguished by the shade of color. For example, the deeper the color of the knowledge base name with a higher score. Thus, it can help users understand and clarify which knowledge base the answer generated by the RAG-based large language model mainly depends on, and when the colors of all knowledge base names are not deep, it can serve as a prominent reminder for users to pay special attention to the credibility of the answer.

[0112] In this way, it can help users distinguish the quality of the answer, enhancing the reliability, transparency, and user trust of the question-answering engine.

[0113] It can be seen that in the above solution, on the one hand, by calculating the ROUGE score between the answer and the knowledge base to quantitatively evaluate the credibility of the answer, compared with directly calculating the semantic similarity between the answer and the knowledge base for evaluation, the calculation speed is much faster, which can significantly improve the efficiency of credibility evaluation. On the other hand, during the process of the RAG-based large language model retrieving knowledge documents related to the user's question from the knowledge base, semantic similarity has already been utilized, and the semantic similarity between each knowledge document and the answer generated by the RAG-based large language model is relatively high. If semantic similarity is still used to evaluate credibility, the discrimination of credibility will be insufficient. By calculating the ROUGE score to quantitatively evaluate the credibility of the answer, it can be clearly distinguished, and the result of credibility evaluation is more accurate.

[0114] In summary, the above solution method can automatically generate answers and evaluate credibility, reducing the need for manual intervention, and improving the response processing efficiency and user experience of the entire question-answering engine.

[0115] The question-answering processing method based on artificial intelligence provided by the above embodiment receives a user's question; through a RAG-based large language model, it processes the user's question according to a preset knowledge base to obtain an answer to the user's question; performs ROUGE calculation processing on the answer and the knowledge base to obtain the ROUGE score between the answer and the knowledge base; and evaluates the credibility of the answer based on the ROUGE score. In this way, on the one hand, through the RAG-based large language model, combined with a large amount of knowledge in the knowledge base, it can reasonably reason and summarize the answer to the user's question, so as to efficiently and accurately generate a high-quality answer that conforms to logic; on the other hand, by calculating the ROUGE score between the answer and the knowledge base to evaluate the credibility of the answer based on the ROUGE score, it can quantify and identify the reliability of the answer, effectively improve the transparency of the response engine, enhance the interpretability of the response engine, and reduce the generation of hallucination phenomena.

[0116] It should be understood that the magnitudes of the sequence numbers of the steps in the above embodiments do not imply the order of execution. The order of execution of each process should be determined by its function and internal logic, and should not constitute any limitation to the implementation process of the embodiments of the present invention.

[0117] In one embodiment, a question-answering processing device based on artificial intelligence is provided. The question-answering processing device based on artificial intelligence corresponds one-to-one with the question-answering processing method based on artificial intelligence in the above embodiment. As Figure 5 shown, the question-answering processing device based on artificial intelligence includes a question receiving module 101, a question answering module 102, a ROUGE score calculation module 103, and a credibility evaluation module 104. The detailed description of each functional module is as follows:

[0118] The question receiving module 101 is used to receive user questions;

[0119] The question answering module 102 is used to answer and process the user questions based on the preset knowledge base through a large language model based on RAG to obtain the answers to the user questions;

[0120] The ROUGE score calculation module 103 is used to perform ROUGE calculation processing on the answers and the knowledge base to obtain the ROUGE score between the answers and the knowledge base;

[0121] The credibility evaluation module 104 is used to evaluate the credibility of the answers according to the ROUGE score.

[0122] In one embodiment, the question answering module 102 is specifically used for:

[0123] Retrieving and processing the user questions according to the knowledge base through the large language model based on RAG to obtain the knowledge documents related to the user questions;

[0124] Generating answers according to the user questions and the knowledge documents through the large language model based on RAG to obtain the answers.

[0125] In one embodiment, the ROUGE score calculation module 103 is specifically used for:

[0126] Calculating the precision rate and recall rate of the answers according to the answers and the knowledge documents;

[0127] Performing ROUGE calculation processing according to the precision rate and the recall rate to obtain the ROUGE score.

[0128] In one embodiment, the ROUGE score calculation module 103 is further used for:

[0129] Perform a common subsequence length calculation process on the knowledge document and the answer to obtain the length of the longest common subsequence between the knowledge document and the answer;

[0130] Perform a precision calculation process on the answer according to the length and the total number of words in the answer to obtain the precision;

[0131] Perform a recall calculation process on the answer according to the length and the total number of words in the knowledge document to obtain the recall;

[0132] In one embodiment, the ROUGE score calculation module 103 is further configured to:

[0133] Perform a shared n-grams quantity calculation process on the knowledge document and the answer to obtain the quantity of shared n-grams between the knowledge document and the answer;

[0134] Perform a precision calculation process on the answer according to the quantity of shared n-grams and the quantity of n-grams in the answer to obtain the precision;

[0135] Perform a recall calculation process on the answer according to the quantity of shared n-grams and the quantity of n-grams in the knowledge document to obtain the recall;

[0136] In one embodiment, the question answering module 102 is further configured to:

[0137] Perform a semantic recognition process on the user question through the RAG-based large language model to obtain the semantic vector of the user question;

[0138] Perform a semantic similarity retrieval process in the knowledge base according to the semantic vector through the RAG-based large language model to obtain the knowledge document.

[0139] In one embodiment, the question answering module 102 is further configured to:

[0140] Perform a fusion process on the user question and the knowledge document to obtain context information;

[0141] Input the context information into the RAG-based large language model;

[0142] Perform an encoding process on the context information to obtain a context vector;

[0143] Perform an autoregressive answer generation process on the context vector through an attention mechanism to obtain the answer.

[0144] The present invention provides a question - answering processing device based on artificial intelligence. On the one hand, through a large - language model based on RAG and in combination with a large amount of knowledge in the knowledge base, it can reasonably infer and summarize the answers to user questions, thereby efficiently and accurately generating high - quality logical answers. On the other hand, by calculating the ROUGE score between the answer and the knowledge base, in order to evaluate the credibility of the answer based on the ROUGE score, it can quantify and identify the reliability of the answer, effectively improve the transparency of the answering engine, enhance the interpretability of the answering engine, and reduce the generation of hallucination phenomena.

[0145] For the specific limitations of the question - answering processing device based on artificial intelligence, reference can be made to the limitations of the intelligent question - answering method in the above text, which will not be elaborated here. Each module in the above - mentioned question - answering processing device based on artificial intelligence can be implemented in whole or in part by software, hardware, and their combinations. The above - mentioned modules can be embedded in the processor of the computer device in hardware form or be independent of it, or can be stored in the memory of the computer device in software form, so that the processor can call and execute the operations corresponding to the above - mentioned modules.

[0146] In one embodiment, a computer device is provided. This computer device can be a server, and its internal structure diagram can be as Figure 6 shown. The computer device includes a processor, a memory, a network interface, and a database connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes non - volatile and / or volatile storage media and internal memory. The non - volatile storage media stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non - volatile storage media. The network interface of the computer device is used to communicate with an external client through a network connection. When the computer program is executed by the processor, it realizes the functions or steps on the server side of a question - answering processing method based on artificial intelligence.

[0147] In one embodiment, a computer device is provided. This computer device can be a client, and its internal structure diagram can be as Figure 7 shown. The computer device includes a processor, a memory, a network interface, a display screen, and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes non - volatile storage media and internal memory. The non - volatile storage media stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non - volatile storage media. The network interface of the computer device is used to communicate with an external server through a network connection. When the computer program is executed by the processor, it realizes the functions or steps on the client side of a question - answering processing method based on artificial intelligence.

[0148] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, the following steps are implemented:

[0149] Receive a user question;

[0150] Through a RAG-based large language model, process the user question according to a preset knowledge base to obtain an answer to the user question;

[0151] Perform ROUGE calculation processing on the answer and the knowledge base to obtain the ROUGE score between the answer and the knowledge base;

[0152] Evaluate the credibility of the answer according to the ROUGE score.

[0153] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the following steps are implemented:

[0154] Receive a user question;

[0155] Through a RAG-based large language model, process the user question according to a preset knowledge base to obtain an answer to the user question;

[0156] Perform ROUGE calculation processing on the answer and the knowledge base to obtain the ROUGE score between the answer and the knowledge base;

[0157] Evaluate the credibility of the answer according to the ROUGE score.

[0158] It should be noted that for the functions or steps that can be achieved by the above computer-readable storage medium or computer device, reference can be made to the relevant descriptions on the server side and the client side in the foregoing method embodiments. To avoid repetition, they will not be described in detail here.

[0159] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, storage, database, or other medium used in the embodiments provided by the present invention can include non-volatile and / or volatile memories. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and Rambus dynamic RAM (RDRAM), etc.

[0160] Those skilled in the art can clearly understand that, for the convenience and simplicity of description, only the above division of each functional unit and module is used as an example. In actual applications, the above functions can be allocated to different functional units and modules according to needs, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0161] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements for some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included in the protection scope of the present invention.

Claims

1. A question-answering processing method based on artificial intelligence, characterized in that: include: Receive user questions; By generating a large language model of RAG based on retrieval enhancement, the user's question is answered according to a preset knowledge base to obtain an answer to the user's question; Performing text summary index ROUGE calculation processing on the answer and the knowledge base to obtain a ROUGE score between the answer and the knowledge base; The answer is evaluated for credibility based on the ROUGE score.

2. The question-answer processing method according to claim 1, wherein: The method of answering the user's question based on the RAG-based large language model and the preset knowledge base to obtain the answer to the user's question includes: By using the RAG-based large language model, searching and processing the user question according to the knowledge base, and obtaining the knowledge document related to the user question; The answer is obtained by performing answer generation processing according to the user question and the knowledge document through the RAG-based large language model.

3. The question-answer processing method according to claim 2, wherein: Performing ROUGE calculation processing on the answer and the knowledge base to obtain a ROUGE score between the answer and the knowledge base includes: Calculate the precision and recall of the answer based on the answer and the knowledge document; ROUGE calculation processing is performed according to the precision and the recall to obtain the ROUGE score.

4. The question-answer processing method according to claim 3, wherein: The calculating the precision and recall of the answer according to the answer and the knowledge document comprises: Calculating the length of a common subsequence between the knowledge document and the answer to obtain the length of the longest common subsequence between the knowledge document and the answer; According to the length and the total number of words in the answer, the accuracy of the answer is calculated to obtain the accuracy; The recall rate of the answer is calculated based on the length and the total number of words in the knowledge document to obtain the recall rate.

5. The question-answer processing method according to claim 3, wherein: The calculating the precision and recall of the answer according to the answer and the knowledge document comprises: Calculating the number of shared n-grams between the knowledge document and the answer to obtain the number of n-grams shared between the knowledge document and the answer; According to the number of the shared n-grams and the number of the n-grams of the answer, the precision rate of the answer is calculated to obtain the precision rate; The recall rate of the answer is calculated based on the number of the shared n-grams and the number of n-grams of the knowledge document to obtain the recall rate.

6. The question-answer processing method according to claim 2, wherein: The method of searching and processing the user question according to the knowledge base by using the RAG-based large language model to obtain the knowledge document related to the user question includes: Performing semantic recognition processing on the user question through the RAG-based large language model to obtain a semantic vector of the user question; By using the RAG-based large language model, semantic similarity retrieval processing is performed in the knowledge base according to the semantic vector to obtain the knowledge document.

7. The question-answer processing method according to claim 2, wherein: The step of generating an answer according to the user question and the knowledge document by using the RAG-based large language model to obtain the answer includes: Fusing the user question and the knowledge document to obtain context information; Inputting the context information into the RAG-based large language model; Encoding the context information to obtain a context vector; The context vector is processed by autoregression to generate an answer through an attention mechanism to obtain the answer.

8. A question-answering processing device based on artificial intelligence, characterized in that: include: A question receiving module is used to receive user questions; A question answering module is used to answer the user's question according to a preset knowledge base through a large language model based on RAG to obtain an answer to the user's question; A ROUGE score calculation module, used for performing ROUGE calculation processing on the answer and the knowledge base to obtain a ROUGE score between the answer and the knowledge base; A credibility evaluation module is used to evaluate the credibility of the answer according to the ROUGE score.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the question and answer processing method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the question and answer processing method according to any one of claims 1 to 7 are implemented.

Images