Question answering method and question answering system

Abstract

This disclosure proposes a method and a system for question answering. The method includes: obtaining a question from a user; executing a matching algorithm based on the question to determine whether the question matches one of multiple predefined questions in a frequently asked database; if the question matches a first predefined question, outputting a first predefined answer corresponding to the first predefined question; if the question does not match any predefined question, inputting the question into a language model to generate a response; outputting the generated response; and updating the frequently asked database based on the generated response.

Description

Technical Field

[0001] This disclosure relates to a question-answering method and system that combines discriminative artificial intelligence and generative artificial intelligence. Prior Technology

[0002] With the rapid development of generative artificial intelligence (AI) technologies, especially large language models (LLMs), intelligent customer service systems are increasingly applying these technologies to improve users' problem-solving capabilities. However, existing generative methods typically require significant computational resources and costs, particularly in scenarios using paid LLMs. For high-frequency query scenarios (such as intelligent customer service systems), this can lead to unnecessary waste of computational resources and increased operating costs.

[0003] Traditional intelligent customer service systems primarily rely on rule-based retrieval techniques or discriminative AI. These methods have low computational costs, can quickly provide predefined answers, and are suitable for structured and predictable question scenarios. However, when the questions asked by users are not covered in the database, the response capabilities of such systems are limited, and they cannot provide sufficient flexibility to handle varied or unseen questions.

[0004] Therefore, the main challenge of existing technologies lies in how to effectively combine discriminative AI and generative LLM in intelligent customer service systems, thereby balancing the accuracy, flexibility, and computational cost of responses. Summary of the Invention

[0005] To address the aforementioned issues, this disclosure proposes a question-answering method and system that can accelerate the response process and continuously update the database.

[0006] This disclosure proposes a question-answering method applicable to computer systems. The question-answering method includes: obtaining a first question from a user; performing a pairing algorithm based on the first question to determine whether the first question is paired with one of multiple preset questions in a frequently asked database; if the first question is paired with a first preset question, outputting a first preset answer corresponding to the first preset question; if the first question is not paired with a preset question, inputting the first question into a language model to obtain a first generated answer; outputting the first generated answer; and updating the frequently asked database based on the first generated answer.

[0007] From another perspective, the embodiments disclosed herein propose a question-and-answer system, including memory and a processor electrically connected to each other. The processor executes instructions in the memory to complete the question-and-answer method described above.

[0008] To make the above-mentioned features and advantages disclosed herein more apparent and understandable, specific embodiments are described below in conjunction with the accompanying drawings for detailed explanation. Simple Explanation of the Diagram

[0009] Figure 1 is a schematic diagram illustrating a question-and-answer system according to one embodiment. Figure 2 is a schematic diagram illustrating the operation of a question-and-answer system according to one embodiment. Figure 3 is a flowchart illustrating the updating of a frequently queried database according to one embodiment. Figure 4 is a schematic diagram illustrating a dialogue record according to one embodiment. Implementation

[0010] Some embodiments disclosed herein will be described in detail below with reference to the accompanying drawings. Component symbols used in the following description are considered identical or similar when appearing in different drawings. These embodiments are only a part of this disclosure and do not reveal all possible implementations of this disclosure. More precisely, these embodiments are merely examples of the systems and methods within the scope of this patent application.

[0011] The terms "first," "second," etc., used in this article do not specifically refer to order or sequence; they are merely used to distinguish elements or operations described using the same technical terms.

[0012] Figure 1 is a schematic diagram illustrating a question-and-answer system according to an embodiment. Referring to Figure 1, the question-and-answer system 120 can be implemented as a mobile phone, personal computer, tablet computer, server, wearable device, home appliance, or any electronic device with computing capabilities. The question-and-answer system 120 includes a processor 121 and a memory 122, which are electrically connected to each other. The processor 121 may include a central processing unit, microprocessor, microcontroller, deep-learning processing unit (DPU), neural network processing unit (NPU), tensor processing unit (TPU), application-specific integrated circuits (ASIC), programmable logic device (PLD), etc. The memory 122 may include random access memory, read-only memory, or flash memory, which stores multiple instructions. The processor 121 executes the instructions in the memory 122 to complete a question-and-answer method.

[0013] User 110 can input a question 111 through any suitable interface device, such as a microphone, keyboard, touch screen, mouse, etc. After the processor 121 completes the question-answering method, it generates an answer 112, which is then provided to user 110 through a suitable interface device, such as a screen, speaker, etc. In some embodiments, the question-answering system 120 may be a chatbot, an artificial intelligence agent, a customer service system, etc., and this disclosure is not limited thereto.

[0014] Figure 2 is a schematic diagram illustrating the operation of a question-and-answer system according to one embodiment. Referring to Figure 2, the question-and-answer system 120 first obtains a question 111 from a user. In step 201, a pairing algorithm is executed based on this question 111 to determine whether the question is paired with one of multiple preset questions in the frequently asked database 210. For example, these preset questions are about financial services and may include questions such as "How are the charges calculated?", "What should I do if I forget my password?", and "What should I do if I lose my credit card?". Each preset question corresponds to a preset answer. In some embodiments, multiple preset questions may also correspond to the same preset answer; for example, "I want to report a lost credit card" and "What should I do if I lose my credit card?" may correspond to the same preset answer.

[0015] The pairing algorithm described above can employ any rule-based algorithm or machine learning algorithm. For example, a machine learning algorithm could include a neural network where question 111 is converted into multiple tokens, each a vector, which are then input into the neural network. There are at least two ways to design the neural network. In the first approach, the neural network outputs multiple confidence values, each corresponding to a preset question and representing the similarity between the preset question and question 111. In the second approach, the neural network outputs a feature vector, and each preset question can also be input into the neural network to obtain a corresponding feature vector. Here, the similarity between two feature vectors can be calculated to obtain a confidence value; this similarity could be, for example, cosine similarity or Euclidean distance (inversely proportional to the confidence value). Using a neural network can also be called discriminative AI. Regardless of the approach used, the confidence value between question 111 and each preset question can be calculated. Next, the maximum value of these confidence values ​​is calculated, and it is determined whether this maximum value is greater than a confidence threshold. If the maximum value is greater than the confidence threshold, then question 111 is matched with the preset question corresponding to the maximum value; if the maximum value is less than or equal to the confidence threshold, then question 111 is not matched with any preset question.

[0016] When question 111 is matched with a preset question (also known as the first preset question), the preset answer (also known as the first preset answer) corresponding to the first preset question can be obtained from the frequently asked question database 210, and this first preset answer 221 can be output to the user 110.

[0017] When question 111 is not matched with any preset question, in step 202, a language model is invoked, and question 111 is input into this language model to obtain a generated answer (also known as the first generated answer). Then, the first generated answer 222 is output to the user 110. This language model may be, for example, the GPT (Generative Pretrained Transformer) series, the BERT (Bidirectional Encoder Representations from Transformers) series, the LLaMA (Large Language Model Meta AI) series, etc., and this disclosure is not limited thereto. This language model may be deployed locally in the question answering system 120, or it may be deployed in the cloud and invoked by the question answering system 120 using an application interface (API). Next, in step 203, the first generated answer may be tested. This testing may include correctness testing, reliability testing, accuracy testing, etc., and this disclosure is not limited thereto. In some embodiments, question 111 and the first generated answer 222 may be input together into the language model, and the language model may determine whether such a question and answer are correct, reliable, or accurate through appropriate prompts. The language models used in steps 202 and 203 can be the same or different. After passing the detection, in step 204, the frequently asked questions database 210 is updated based on the first generated answer 222. This may involve adding a pair between question 111 and the first generated answer 222, modifying a preset answer in the frequently asked questions database 210, mapping question 111 to an existing preset answer, merging question 111 with an existing preset question, merging the first generated answer 222 with an existing preset answer, and so on. If the detection fails, the frequently asked questions database 210 will not be updated.

[0018] In some embodiments, the language model generates a first generated answer 222 based on a retrieval augmented generation (RAG) algorithm. In other words, the language model can query various databases or web pages based on question 111 and merge or summarize the results to obtain the first generated answer 222.

[0019] In some embodiments, the question-and-answer system 120 performs the step of updating the frequently asked database 210 in a non-real-time state. The purpose of this is to provide answers to the user 110 immediately, and then update the database only when the connection between the user 110 and the question-and-answer system 120 is lost, or when the question-and-answer system 120 has idle resources.

[0020] In the above approach, a matching algorithm is prioritized to attempt to find matching preset questions and answers from the frequently asked query database 210. The language model is only invoked when no matching is found. This approach avoids frequent calls to the language model, reducing computational resources and API usage costs. Furthermore, the generated answers can be used to update the frequently asked query database 210, thus increasing the probability of successful matching for similar questions in the future. In other words, the frequency of use of the frequently asked query database 210 increases as the system runs, and the language model is only invoked when a previously unseen question is received. This not only reduces costs but also increases response speed.

[0021] Figure 3 is a flowchart illustrating the updating of a frequently asked query database according to one embodiment. Referring to Figure 3, in step 301, the aforementioned question 111 (hereinafter referred to as q1) and the first generated answer 222 (hereinafter referred to as a1) are first obtained.

[0022] In step 302, all preset questions, question q1, and appropriate prompts are input into the language model to determine whether question q1 is similar to one of the preset questions. Step 302 differs from the pairing algorithm described above; this step utilizes the language model's higher comprehension ability to determine if there are similar preset questions. If no similar preset question is found in the frequently asked query database 210, in step 303, question q1 and the generated answer a1 are combined into a pair (q1, a1) and added to the frequently asked query database 210.

[0023] In some embodiments, before adding a pair (q1, a1) to the frequently queried database 210, it can be determined whether the pair (q1, a1) passes a check. As mentioned above, this check may include one or more of correctness checks, reliability checks, and accuracy checks, and this check is also performed by the language model. The pair (q1, a1) is added to the frequently queried database 210 only if it passes the check. If the pair (q1, a1) fails the check, it will not be added to the frequently queried database 210.

[0024] If the result of step 302 is yes (assuming question q1 is similar to the preset question q2), then in step 304, question q1, the preset question q2, and an appropriate prompt are input into the language model to determine whether question q1 and the preset question q2 belong to a higher-level relationship. This higher-level relationship refers to one question being broader than another, or both being different sub-questions about the same topic. For example, one question is "Can I bring outside food into this attraction?" and the other is "Can I bring pets into this attraction?" Since both questions are about the regulations of the attraction, they belong to a higher-level relationship. In some embodiments, the prompt used is, for example, "Are these two questions related to a higher-level relationship or can they be combined?"

[0025] If the judgment result of step 304 is negative, in step 305, the pairing (q2, a2) to which the preset question q2 belongs is updated according to question q1, where a2 is the preset answer corresponding to the preset question q2. For example, the preset answer a2 can be used to replace the generated answer a1, so that question q1 also corresponds to the preset answer a2. In this way, both question q1 and the preset question q2 will correspond to the preset answer a2. When the user asks other questions similar to question q1 next time, there is a chance to directly use the preset answer a2 to reply to the user.

[0026] If the result of step 304 is yes, in step 306, question q1, preset question q2, and appropriate prompts are input into the language model to generate a new question q_new. Furthermore, generated answer a1, preset answer a2, and appropriate prompts are input into the language model to generate a new answer a_new. Here, the prompt used is, for example, "Please combine these two questions (answers) according to their semantics." For example, the two questions are "Can I bring outside food to this attraction?" and "Can I bring pets to this attraction?", and the merged new question is "Can I bring outside food or pets to this attraction?" Next, in step 307, the language model determines whether the new question q_new and the new answer a_new pass the test. If they pass, in step 308, the new question q_new and the new answer a_new form a pair (q_new, a_new) to be added to the frequently asked question database 210, and this pair (q_new, a_new) replaces the original preset question q2 and preset answer a2. In this context, as users ask relevant or related questions, the question-answering system 120 can continuously update the question database 210, merging new information into the existing preset questions and preset answers.

[0027] If the result of step 307 is negative, in step 309, the context of question q1 is obtained, and question q1 and the context are merged to generate a new question q3. For example, Figure 4 is a schematic diagram illustrating a dialogue record according to an embodiment. The content of dialogue 401 is "What conditions are required for a return?", and the content of dialogue 402 is "What if the product is damaged?". When dialogue 402 is provided as a question to the question answering system 120, a suitable preset question cannot be found based solely on the content of dialogue 402, and even if a similar preset question is found, it cannot pass the detection (the result of step 307 is negative). Therefore, dialogue 401 can be obtained as the context of dialogue 402. In some embodiments, dialogues 402 and 402 can both be input into the language model, and the language model will output the merged question q3. For example, the merged question q3 is "What conditions are required for a return if the product is damaged?" Next, the question is input into the language model to obtain the generated answer a3.

[0028] Next, returning to step 302, replace question q1 described above with question q3, replace generated answer a1 with generated answer a3, and repeat steps 302-307 until the detection is passed in step 307. If the detection is still not passed when step 307 is executed a second time, a previous dialogue can be obtained as context in step 309. In some embodiments, if there is no previous dialogue that can be used as context, different questions and answers can be generated in step 306 by relying on the randomness of the language model. In some embodiments, a limit on the number of repetitions can be set. When the number of times the detection is not passed reaches this limit, the process is terminated, that is, the latest questions and answers will not be updated to the frequently asked questions database 210.

[0029] The frequently asked questions database 210 can be updated in different ways using the process shown in Figure 3. Step 303 adds a new pairing, step 305 adds a new question and assigns it to an existing answer, and step 308 merges the new question (answer) with the existing question (answer). After the update, the pairs stored in the frequently asked questions database 210 can cover more questions, and new information can be added to existing answers. In some experiments, this update can make the process shown in Figure 2 faster, and because the original preset questions in the question database 210 are updated, accuracy can also be improved.

[0030] Although this disclosure has been disclosed above with reference to embodiments, it is not intended to limit this disclosure. Anyone skilled in the art may make some modifications and refinements without departing from the spirit and scope of this disclosure. Therefore, the scope of protection of this disclosure shall be determined by the appended claims.

[0031] 110: User 111: Question 112: Answer 120: Question and Answer System 121: Processor 122: Memory Steps 201~204, 301~309: 210: Frequently consult the database 221: Preset Answer 222: Generate an answer 401, 402: Dialogue

Claims

1. A question-answering method, applicable to a computer system, the question-answering method comprising: obtaining a first question from a user; performing a pairing algorithm based on the first question to determine whether the first question is paired with one of a plurality of preset questions in a frequently asked database; if the first question is paired with a first preset question among the preset questions, outputting a first preset answer corresponding to the first preset question; if the first question is not paired with any of the preset questions, inputting the first question into a language model to obtain a first generated answer; outputting the first generated answer; and updating the frequently asked database based on the first generated answer, wherein the step of updating the frequently asked database based on the first generated answer includes: (a) Input the preset questions and the first question into the language model to determine whether the first question is similar to one of the preset questions; (b) If the first question is not similar to one of the preset questions, combine the first question and the first generated answer into a first pair to add to the frequently asked database; (c) If the first question is similar to a second preset question of the preset questions, input the first question and the second preset question into the language model to determine whether the first question and the second preset question belong to a superordinate relationship; and (d) If the first question and the second preset question do not belong to the superordinate relationship, update the pair to which the second preset question belongs based on the first question.

2. The question-answering method as described in claim 1, wherein the step of performing the pairing algorithm based on the first question to determine whether the first question is paired with one of multiple preset questions in the frequently asked database includes: Calculate a confidence value between the first question and each of the preset questions; Calculate the maximum value of these confidence values; determine whether the maximum value is greater than a confidence threshold; if the maximum value is greater than the confidence threshold, determine that the first question is matched with the preset question corresponding to the maximum value; and if the maximum value is less than or equal to the confidence threshold, determine that the first question is not matched with these preset questions.

3. The question-answering method as described in claim 1, wherein before adding the first pair to the frequently asked database, the question-answering method further includes: The language model determines whether the first question and the first generated answer pass a test. If the first question and the first generated answer pass the detection, the first pair is added to the frequently asked questions database; and if the first question and the first generated answer fail the detection, the first pair is not added to the frequently asked questions database.

4. The question-answering method as described in request item 1 further includes: (e) If the first question and the second preset question belong to the superordinate relationship, input the first question and the second preset question into the language model to generate a first new question, and input the first generated answer and the second preset answer corresponding to the second preset question into the language model to generate a first new answer; (f) The language model determines whether the first new question and the first new answer pass a detection; and (g) If the first new question and the first new answer pass the detection, combine the first new question and the first new answer into a second pair to add to the frequently asked database, and replace the second preset question and the second preset answer with the second pair.

5. The question-answering method as described in request item 4 further includes: (h) If the first new question and the first new answer fail the detection, obtain a context corresponding to the first question, and merge the first question and the context to generate a second question; And (i) input the second question into the language model to obtain a second generated answer.

6. The question-answering method as described in request item 5 further includes: Replace the first new question with the second new question, and the first new answer with the second new answer, to repeat steps (a) to (i).

7. The question-answering method as described in Request 1, wherein the language model generates the first generated answer based on a retrieval-enhanced generation algorithm.

8. The question-answering method as described in request item 1, wherein the step of updating the frequently asked database based on the first generated answer is performed in a non-real-time state.

9. A question-and-answer system, comprising: A memory module that stores multiple instructions; and a processor, electrically connected to the memory, for executing instructions to complete multiple steps: obtaining a first question from the user; executing a pairing algorithm based on the first question to determine whether the first question is paired with one of a number of preset questions in a frequently asked database; if the first question is paired with a first preset question among the preset questions, outputting a first preset answer corresponding to the first preset question; if the first question is not paired with any of the preset questions, inputting the first question into a language model to obtain a first generated answer; and outputting the first generated answer. The method further includes updating the frequently asked questions database based on the first generated answer, wherein the steps of updating the frequently asked questions database based on the first generated answer include: (a) inputting the preset questions and the first question into the language model to determine whether the first question is similar to one of the preset questions; (b) if the first question is not similar to one of the preset questions, combining the first question and the first generated answer into a first pair to add to the frequently asked questions database; (c) if the first question is similar to a second preset question of the preset questions, inputting the first question and the second preset question into the language model to determine whether the first question and the second preset question belong to a superordinate relationship; and (d) if the first question and the second preset question do not belong to the superordinate relationship, updating the pair to which the second preset question belongs based on the first question.

10. The question-answering system as described in claim 9, wherein the step of performing the matching algorithm based on the first question to determine whether the first question is matched to one of a plurality of preset questions in the frequently asked database includes: Calculate a confidence value between the first question and each of the preset questions; Calculate the maximum value of these confidence values; determine whether the maximum value is greater than a confidence threshold; if the maximum value is greater than the confidence threshold, determine that the first question is matched with the preset question corresponding to the maximum value; and if the maximum value is less than or equal to the confidence threshold, determine that the first question is not matched with these preset questions.

11. The question-answering system as described in claim 9, wherein the steps further include, before adding the first pair to the frequently asked database: The language model determines whether the first question and the first generated answer pass a test. If the first question and the first generated answer pass the detection, the first pair is added to the frequently asked questions database; and if the first question and the first generated answer fail the detection, the first pair is not added to the frequently asked questions database.

12. The question-answering system as described in claim 9, wherein the steps further include: (e) If the first question and the second preset question belong to the superordinate relationship, input the first question and the second preset question into the language model to generate a first new question, and input the first generated answer and the second preset answer corresponding to the second preset question into the language model to generate a first new answer; (f) The language model determines whether the first new question and the first new answer pass a detection; and (g) If the first new question and the first new answer pass the detection, combine the first new question and the first new answer into a second pair to add to the frequently asked database, and replace the second preset question and the second preset answer with the second pair.

13. The question-answering system as described in claim 12, wherein the steps further include: (h) If the first new question and the first new answer fail the detection, obtain a context corresponding to the first question, and merge the first question and the context to generate a second question; And (i) input the second question into the language model to obtain a second generated answer.

14. The question-answering system as described in claim 13, wherein the steps further include: Replace the first new question with the second new question, and the first new answer with the second new answer, to repeat steps (a) to (i).

15. The question-answering system as described in claim 9, wherein the language model generates the first generated answer based on a retrieval-enhanced generation algorithm.

16. The question-answering system as described in claim 9, wherein the step of updating the frequently asked database based on the first generated answer is performed in a non-real-time state.

Images