An optimization method for large language model question answering system

Abstract

The present invention discloses an optimization method for a large language model question-answering system, which relates to the technical field of natural language processing, including obtaining questions raised by a questioner, obtaining question information, structuring the question information, and submitting it to an answerer designated by the questioner, wherein the answerer includes a large language model and a human, and the question information includes current question content, historical question-answering content, and identity information of the questioner; generating an answer result Ans_1 through a FAQ module or a large language model; the optimization method for the large language model question-answering system can effectively avoid the large language model hallucination through the mechanism of answerer revision in step S3, that is, the phenomenon that the large language model may generate content inconsistent with real-world facts or user input when generating text, so as to ensure the professionalism and accuracy of answers in important application scenarios.

Description

Technical Field

[0001] The present invention relates to the technical field of natural language processing, and in particular to an optimization method for a large language model question answering system. Background Art

[0002] With the rapid development of artificial intelligence technology, large language models (LLMs) have become one of the core technologies in the field of natural language processing. These models can capture rich language rules and knowledge information by training on large-scale text data, thus having powerful natural language understanding and generation capabilities, and are widely used in various question-answering systems.

[0003] However, although large language models have achieved remarkable results in many natural language processing tasks, their performance in question-answering systems, especially in answering professional questions, still needs to be improved. This is because traditional question-answering systems usually rely on manually designed rules or specific knowledge bases to answer questions, while large language models try to automatically learn the ability to answer questions from massive amounts of text data. This difference leads to some challenges for large language models in question-answering systems, such as the accuracy and consistency of answers and the ability to handle complex questions.

[0004] In order to solve the above problems, using large language models to build efficient and accurate question-answering systems has become an important research goal. In the current related technologies, there are two main methods for optimizing the output of large language models: 1. A method for optimizing prompts based on user feedback, such as the invention patent CN117573846B, which automatically optimizes the output of large language models through multiple rounds of prompt word optimization; 2. A method based on domain knowledge base transfer learning, such as the invention patent application CN118070870A, which introduces domain expert knowledge data sets, and integrates domain knowledge into the preliminary power professional large language model through transfer learning. Among them, the first type of method mainly relies on the feedback of the questioner, and the questioner generally only has the ability to evaluate the relevance of the answer to the question, and does not have the ability to evaluate the accuracy of the answer to professional questions; and the domain expert knowledge data set described in the second type of method is not easy to produce. At the same time, the above methods cannot solve the problem that the large language model stubbornly insists on its own opinions, repeatedly makes the same wrong answers to multiple questions on the same question, and cannot revise the answers in time. Summary of the invention

[0005] The purpose of the present invention is to provide an optimization method for a large language model question answering system to solve the above-mentioned deficiencies in the prior art.

[0006] In order to achieve the above object, the present invention provides the following technical solution: an optimization method for a large language model question answering system, comprising the following steps:

[0007] S1. Obtain the question raised by the questioner, obtain question information, structure the question information, and submit it to the answerer designated by the questioner. The answerer includes a large language model and a human. The question information includes the current question content, historical question and answer content, and the identity information of the questioner.

[0008] S2, generate the answer result Ans_1 through the FAQ module or the large language model;

[0009] S3, the respondent revises Ans_1 to obtain the answer result Ans_2, and returns Ans_2 as the final answer to the questioner, wherein the revision includes modifying the content of Ans_1 and confirming the content of Ans_1;

[0010] S4, obtaining the feedback of the questioner on Ans_2 by tagging Ans_2, combining the question raised by the questioner, Ans_2, and the feedback of the questioner on Ans_2 to form new {question; answer; feedback} data, obtaining FAQ data, and recording the FAQ data into the FAQ module for updating the FAQ module;

[0011] S5. Use FAQ data to fine-tune the large language model through supervised learning fine-tuning and reinforcement learning fine-tuning.

[0012] Furthermore, the questioner includes a user and a large language model. When the questioner is a large language model, the large language model automatically generates a question to obtain question information.

[0013] Furthermore, the S2 specifically includes the following steps:

[0014] Check whether there is an answer to the question in the FAQ module;

[0015] If yes, the result in the FAQ module is used as the answer result Ans_1 of the large language model;

[0016] If not, the structured question information is used as the input of the large language model to generate the answer result Ans_1.

[0017] Furthermore, the S3 specifically includes the following steps:

[0018] S3.1, determine whether the respondent has revised the answer result Ans_1;

[0019] S3.2, if yes, get the answer result Ans_2 and return it to the questioner;

[0020] S3.3, otherwise, wait for the set time. If the respondent does not respond within the set time, Ans_1 is directly copied to Ans_2 as the final answer and submitted to the questioner. If the respondent responds within the set time, return to S3.1.

[0021] Furthermore, the label includes a qualitative label and a quantitative label, the qualitative label includes affirmation and negation, and the quantitative label is a score within a set score range.

[0022] Furthermore, the updating of the FAQ module further comprises the following steps:

[0023] Use the large language model to abstract the questioner's questions and corresponding answers to obtain the corresponding question templates and answer templates;

[0024] The question template, answer template and the questioner's feedback on Ans_2 are combined to form new {question template; answer template; feedback} data to obtain FAQ template data, and the FAQ model data is recorded into the FAQ module for updating the FAQ module.

[0025] Further, the querying of whether there is an answer to the question in the FAQ module comprises the following steps:

[0026] Query the questioner's question in the FAQ data;

[0027] If the query is unsuccessful, the large language model is used to abstract the questioner's question and obtain the corresponding question template;

[0028] Searching the FAQ template data for the question template;

[0029] If the query is successful, then {question template; answer template} that solves the problem is added to the input prompt word of the large language model, so that the large language model can refer to the answer template to answer.

[0030] Furthermore, the supervised learning tuning includes the following steps:

[0031] If the feedback is a qualitative label, filter out {question; answer} pairs with positive feedback from the FAQ data; if the feedback is a quantitative label, filter out {question; answer} pairs with feedback scores exceeding a preset threshold from the FAQ data;

[0032] Construct a supervised learning dataset based on the selected {question; answer} pairs;

[0033] Fine-tune large language models using supervised learning datasets.

[0034] Furthermore, the reinforcement learning fine-tuning includes the following steps:

[0035] Based on the {question; answer; feedback} dataset, a reward model is trained to provide feedback scores on the answer performance of the large language model based on the question input;

[0036] Based on the reinforcement learning algorithm, the reward model is used to provide feedback and score on the answer output of the large language model, which is used to fine-tune the large language model to maximize the expected feedback score.

[0037] Further, the tuning is started in a periodic or quantitative manner, comprising the following steps:

[0038] Perform timing, and when the timing reaches the set time, start a new round of tuning and reset the timing; or measure the collected FAQ data, and when the measurement reaches the set amount, start a new round of tuning and reset the measurement;

[0039] After each round of tuning, the tuned large language model weights are used to update the currently used large language model weights, and the FAQ data saved in the FAQ module is cleared.

[0040] 1. Compared with the prior art, the optimization method of a large language model question-answering system provided by the present invention can effectively avoid the large language model hallucination through the respondent revision mechanism in step S3, that is, the large language model may generate content inconsistent with real-world facts or user input when generating text, so as to ensure the professionalism and accuracy of answers in important application scenarios such as medical consultation, legal consultation and other scenarios.

[0041] 2. Compared with the prior art, the optimization method of a large language model question-answering system provided by the present invention can quickly save the correct answer after the respondent makes a single revision through the updating and retrieval answer mechanism, and make the correct answer through the FAQ retrieval mechanism in the subsequent answers to the same question through the FAQ of the above steps S2 to S4, which can effectively solve the problems that often occur in the existing large language model application scenarios, such as stubbornly insisting on one's own opinions, repeatedly giving the same wrong answers to multiple questions of the same question, and failing to revise the answers in time.

[0042] 3. Compared with the prior art, the optimization method of a large language model question answering system provided by the present invention can achieve continuous tuning and improvement of the large language model and question answering system through the S5 tuning mechanism. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0044] Figure 1 A diagram of the method steps provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0045] In order to enable those skilled in the art to better understand the technical solution of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings.

[0046] In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", "clockwise", "counterclockwise" and the like indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the referred device or element must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as limiting the present invention.

[0047] In addition, the terms "first" and "second" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, "multiple" means two or more, unless otherwise clearly and specifically defined. In addition, the terms "installed", "connected" and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal connection of two components. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0048] Example embodiments will be described more fully below with reference to the accompanying drawings, but the example embodiments may be embodied in different forms and should not be construed as limited to the embodiments set forth herein. On the contrary, the purpose of providing these embodiments is to make the present disclosure thorough and complete and to enable those skilled in the art to fully understand the scope of the present disclosure.

[0049] In the absence of conflict, the various embodiments of the present disclosure and the various features therein may be combined with each other.

[0050] As used herein, the term "and / or" includes any and all combinations of one or more of the associated listed items.

[0051] The terms used herein are only used to describe specific embodiments and are not intended to limit the present disclosure. As used herein, the singular forms "a", "an" and "the" are also intended to include the plural forms, unless the context clearly indicates otherwise. It will also be understood that when the terms "comprising" and / or "made of" are used in this specification, the presence of the features, wholes, steps, operations, elements and / or components is specified, but the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or groups thereof is not excluded.

[0052] The embodiments described herein may be described with reference to plan views and / or cross-sectional views by means of idealized schematic diagrams of the present disclosure. Therefore, the example illustrations may be modified according to manufacturing techniques and / or tolerances. Therefore, the embodiments are not limited to the embodiments shown in the accompanying drawings, but include modifications of the configurations formed based on the manufacturing process. Therefore, the regions illustrated in the accompanying drawings have schematic properties, and the shapes of the regions shown in the figures illustrate the specific shapes of the regions of the elements, but are not intended to be limiting.

[0053] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by those of ordinary skill in the art. It will also be understood that terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant art and the present disclosure, and will not be interpreted as having an idealized or overly formal meaning unless explicitly defined as such herein.

[0054] See also Figure 1 , an optimization method for a large language model question answering system, comprising the following steps:

[0055] S1, obtain the question raised by the questioner and obtain the question information;

[0056] Specifically, it includes: the questioner asks questions through voice or text, structures the question information, and submits it to a specific respondent, where the respondent can be a large language model or a human; in one of the embodiments of the present invention, in the medical consultation system, the patient can ask questions to the digital doctor or to the professional doctor.

[0057] Furthermore, the question information includes the current question content and the historical question and answer content; further preferably, the identity information of the questioner can also be added, including but not limited to age, gender, hobbies, etc.; further preferably, the identity information of the respondent can also be structured into the question information, including but not limited to main fields, years of practice, etc.; in the above-mentioned consultation system embodiment, the question information includes the patient's age, gender, previous medical history, the department consulted, the level of the doctor being questioned, as well as the current question and historical questions and answers of the current round.

[0058] Further preferably, the identity of the questioner can also be a large language model to automatically generate questions; in the above embodiment, the doctor can also trigger the digital patient to ask questions to him in the consultation system.

[0059] Furthermore, a specific respondent is a user who is assigned an respondent identity by the question-and-answer system, and may be a manual respondent or a large language model respondent; further preferably, a manual respondent is a user who has been verified by the question-and-answer system to have corresponding capabilities; in the above-mentioned consultation system embodiment, the manual respondent is a doctor from multiple departments registered with the hospital and certified by the hospital, and the large language model respondent is a large language model trained by the medical knowledge base.

[0060] S2, generates the answer result Ans_1 through the FAQ module or the large language model;

[0061] Specifically, they include:

[0062] S2.1. First, check whether there is an answer to the question in the FAQ (frequently asked questions) module;

[0063] S2.2. If it exists, the result in the FAQ module is used as the answer result Ans_1 of the large language model;

[0064] S2.3, otherwise, use the structured question information as the input of the large language model to generate the answer result Ans_1;

[0065] In the above-mentioned medical consultation system embodiment, for the question raised by the patient, the question is first queried from the FAQ module; if the question is not recorded in the FAQ module, the large language model is used to generate the answer result Ans_1; if the question is recorded in the FAQ module, the answer recorded in the FAQ module is used as the answer result Ans_1, and the large language model does not need to generate an answer again;

[0066] Furthermore, the FAQ module is a module in the question-and-answer system that records questions, corresponding answers, and the questioner's feedback on the answers. It is a pre-configured module based on the existing technology; the large language model is a pre-trained large language model.

[0067] S3, the respondent revises Ans_1 to obtain the answer Ans_2, and returns Ans_2 to the questioner as the final answer;

[0068] Specifically, they include:

[0069] S3.1, determine whether the respondent has revised the answer result Ans_1;

[0070] S3.2, if yes, get the answer result Ans_2 and return it to the questioner;

[0071] S3.3. Otherwise, if the respondent does not respond within the time limit set by the respondent, such as when the respondent leaves, Ans_1 is directly copied to Ans_2 as the final answer and submitted to the questioner. If the respondent responds within the time limit set by the respondent, the process returns to S3.1.

[0072] In an embodiment of the teaching question-and-answer system of the present invention, the teacher modifies the solution given by the large language model to become the final solution; if the student chooses to ask questions to the large language model teacher, or chooses to ask questions to the manual teacher, but the manual teacher leaves or does not respond to the operation within the specified time, the teaching question-and-answer system will directly copy Ans_1 to Ans_2 as the final solution and submit it to the student.

[0073] Furthermore, the revision can be empty, that is, there is no modification. When the revision is empty, the respondent can copy Ans_1 to Ans_2 by directly confirming Ans_1, or change punctuation and / or text content, etc.;

[0074] In an embodiment of the teaching question-answering system of the present invention, the student is the questioner and the teacher is the answerer. When the teacher finds that the answer given by the large language model is wrong or has a better solution, the teacher can make corrections or supplements. If the teacher approves the solution given by the large language model, the result of the large language model can also be directly sent to the student. For example, assuming that the subject the student asks is mathematics, question Q1 is "There are several chickens and rabbits in the same cage. Counting from the top, there are thirty-five heads; counting from the bottom, there are ninety-four feet. How many chickens and rabbits are there in the cage?" The large language model gives the following answer A1:

[0075] "According to the standard solution to the chicken-rabbit problem, we can set the number of chickens to The number of rabbits is . Given that the total number of heads is 35 and the total number of feet is 94, then , According to the previous formula, we can get , substituting the result into the following formula, we can get , and then substitute back into the previous formula, we get ; So there are 23 chickens and 12 rabbits. "

[0076] Teacher 1 approves this answer and directly sends the result to the student without correction; Teacher 2 supplements this answer with the following answer A2:

[0077] "According to the standard solution to the chicken-rabbit problem, we can set the number of chickens to The number of rabbits is The total number of heads is 35 and the total number of feet is 94. , The first calculation method, according to the previous formula, can be obtained , substituting the result into the following formula, we can get , and then substitute back into the previous formula, we get , so there are 23 chickens and 12 rabbits. The second calculation method is to simplify the latter formula to , and then subtract the simplified equation from the previous one, we get , and then substitute into the previous formula to get ; So there are 23 chickens and 12 rabbits. "

[0078] S4, obtain the questioner’s feedback on Ans_2 and update the FAQ module;

[0079] Further, obtaining the feedback of the questioner on Ans_2 includes: marking Ans_2 with a label; preferably, the label can be qualitative, such as: affirmative / negative; or quantitative, such as marking a score value, such as a score on a 5-point scale (i.e., 0-5 points) or a 100-point scale (0-100 points); for example, for the "chicken and rabbit in the same cage" problem in the above embodiment, under the 5-point scale, the student thinks that the result given by teacher 1 is correct, but the solution is not rich enough, and gives 3 points; while the result given by teacher 2 is correct, and more than one solution is given, and gives 4 points;

[0080] Furthermore, updating the FAQ module means combining the question raised by the questioner, Ans_2, and the questioner's feedback on Ans_2 to form a new {question; answer; feedback} data, and recording it into the FAQ module; in the above-mentioned teaching question and answer system embodiment, if Q1 is not in the record of the FAQ module, a new {Q1; A2; 4 points} data will be formed.

[0081] Further optionally, updating the FAQ module may also include: using the large language model to abstract the questioner's question and the corresponding answer to obtain a template of the question and a template of the corresponding answer, and then combining the question template, the answer template and the questioner's feedback on Ans_2 to form a new {question template; answer template; feedback} FAQ template data, and recording it in the FAQ module; further, when performing FAQ query in S2, if the questioner's question is not successfully queried in the FAQ data, then using the large language model to abstract the questioner's question to obtain a template of the question, and then querying the FAQ template data, if the query is successful, the input prompt word of the large language model is Optimized rewriting: Add {question template; answer template} to solve this type of problem in the prompt word, so that the large language model can refer to the answer template to answer; for example, in the above teaching question-answering system embodiment, by saving the answer template (i.e., the solution idea or answer steps) of the "chicken and rabbit in the same cage" problem, in the subsequent use of the system, when a student asks the chicken and rabbit in the same cage problem again, even if the number in the question has changed and cannot be successfully queried in the FAQ data, the template of the problem obtained by abstract extraction can be matched to the answer template of the "chicken and rabbit in the same cage" type of problem in the FAQ template data, thereby guiding the large language model to answer by using the solution idea or answer steps in the template through the prompt word;

[0082] S5, use FAQ data to tune the large language model;

[0083] Specifically, FAQ data is the {question; answer; feedback} data saved in the FAQ module, and the tuning includes: 1) supervised learning fine-tuning (finetune); 2) reinforcement learning fine-tuning;

[0084] Further, the method of supervised learning tuning is:

[0085] S511, filter out {question; answer} pairs with good feedback results from the FAQ data;

[0086] Further, a good feedback result refers to a {question; answer} pair with a "positive" feedback in the qualitative scoring, or a {question; answer} pair with a feedback score exceeding a preset threshold (such as a score greater than or equal to 4 points in a 5-point scoring system, or a score greater than or equal to 80 points in a 100-point scoring system) in the quantitative scoring; in the teaching question and answer system embodiment of the present invention, a 5-point scoring standard is adopted to screen question and answer pairs with feedback scores of 4 and 5 from the teaching question and answer FAQ data;

[0087] S512, constructing a supervised learning dataset using the screened {question; answer} pairs; in the above embodiment, we construct a supervised learning dataset using the questions in the screened question-answer pairs as data and the answers as labels;

[0088] S513, fine-tuning the large language model through supervised learning using the supervised learning data set. For example, fine-tuning methods such as instruction fine-tuning, adapter tuning, LoRa (Low-Rank Adaptation), etc. can be used to fine-tune the large language model.

[0089] Further, the method of reinforcement learning fine-tuning is:

[0090] S521, using the {question; answer; feedback} data set, train a reward model (Reward Model). The goal of the reward model is to imitate human feedback, that is, to provide feedback and scoring for the answer performance of the large language model based on the question input; in the above embodiment, the questions and answers in the FAQ {question; answer; feedback} data set are used as data, and the feedback is used as a label. First, feature engineering is performed on the questions and answers. The feature engineering includes constructing the text into word vectors, calculating the semantic similarity of the text, extracting features such as part of speech, numbers, operators, etc. in the text, and using a deep learning network to train a reward model for teaching question-answering feedback.

[0091] S522, using a reinforcement learning algorithm, the large language model is fed back and scored through a reward model to maximize the expected feedback score. For example, a reinforcement learning algorithm such as PPO (Proximal Policy Optimization) or A3C (Asynchronous Advantage Actor-Critic) can be used to fine-tune the large language model.

[0092] Furthermore, each round of tuning can be performed regularly (such as every 30 days) and / or quantitatively (such as every time 10,000 FAQ data are accumulated). Specifically, a timing is performed, and when the timing reaches the set time, a new round of tuning is started, and the timing is reset; or the collected FAQ data is measured, and when the measurement reaches a set number, a new round of tuning is started, and the measurement is reset; in the embodiment of the teaching question and answer system of the present invention, a quantitative method is adopted, and a round of tuning is started every time 10,000 "affirmative" question and answer pairs are accumulated.

[0093] Furthermore, in each round of tuning, supervised learning fine-tuning and reinforcement learning fine-tuning can be performed in any order, or only one of them can be performed; for the above embodiment, in one round of tuning, the supervised learning dataset is first used to perform supervised learning fine-tuning on the large language model, and then the {question; answer; feedback} dataset is used to train the reward model, and the reward model is used to perform reinforcement learning on the large prediction model.

[0094] Furthermore, after each round of tuning, the tuned large language model weights are used to update the currently used large language model weights, and the FAQ data saved in the FAQ module is cleared.

[0095] The above description is only by way of illustration of certain exemplary embodiments of the present invention. It is undoubted that those skilled in the art can modify the described embodiments in various ways without departing from the spirit and scope of the present invention. Therefore, the above drawings and descriptions are illustrative in nature and should not be construed as limiting the scope of protection of the claims of the present invention.

Claims

1. A method for optimizing a large language model question answering system, characterized in that: The following steps are involved: S1. Obtain the question raised by the questioner, obtain question information, structure the question information, and submit it to the answerer designated by the questioner. The answerer includes a large language model and a human. The question information includes the current question content, historical question and answer content, and the identity information of the questioner. S2, generate answer result Ans_1 through FAQ module or large language model; S3, the respondent revises Ans_1 to obtain the answer result Ans_2, and returns Ans_2 as the final answer to the questioner, wherein the revision includes modifying the content of Ans_1 and confirming the content of Ans_1; S4, obtaining the feedback of the questioner on Ans_2 by tagging Ans_2, combining the question raised by the questioner, Ans_2, and the feedback of the questioner on Ans_2 to form new {question; answer; feedback} data, obtaining FAQ data, and recording the FAQ data into the FAQ module for updating the FAQ module; S5. Use FAQ data to fine-tune the large language model through supervised learning fine-tuning and reinforcement learning fine-tuning; The S2 specifically includes the following steps: Check whether there is an answer to the question in the FAQ module; If yes, the result in the FAQ module is used as the answer result Ans_1 of the large language model; If not, the structured question information is used as the input of the large language model to generate the answer result Ans_1; The updating FAQ module further comprises the following steps: Use the large language model to abstract the questioner's questions and corresponding answers to obtain the corresponding question templates and answer templates; Combine the question template, the answer template and the questioner's feedback on Ans_2 to form new {question template; answer template; feedback} data to obtain FAQ template data, and record the FAQ template data into the FAQ module for updating the FAQ module; The querying of whether there is an answer to the question in the FAQ module comprises the following steps: Query the questioner's question in the FAQ data; If the query is unsuccessful, the large language model is used to abstract the questioner's question and obtain the corresponding question template; Searching for the question template in FAQ template data; If the query is successful, then {question template; answer template} that solves the problem is added to the input prompt word of the large language model, so that the large language model can refer to the answer template to answer.

2. The optimization method for a large language model question answering system according to claim 1, characterized in that: The questioner includes a user and a large language model. When the questioner is a large language model, the large language model automatically generates a question to obtain question information.

3. The optimization method of a large language model question answering system according to claim 1, characterized in that: The S3 specifically includes the following steps: S3.1, determine whether the respondent has revised the answer result Ans_1; S3.2, if yes, get the answer result Ans_2 and return it to the questioner; S3.3, otherwise, wait for the set time. If the respondent does not respond within the set time, Ans_1 is directly copied to Ans_2 as the final answer and submitted to the questioner. If the respondent responds within the set time, return to S3.

1.

4. The optimization method of a large language model question answering system according to claim 1, characterized in that: The labels include qualitative labels and quantitative labels. The qualitative labels include affirmation and negation, and the quantitative labels are scores within a set score range.

5. The optimization method for a large language model question answering system according to claim 4, characterized in that: The supervised learning tuning includes the following steps: If the feedback is a qualitative label, filter out {question; answer} pairs with positive feedback from the FAQ data; if the feedback is a quantitative label, filter out {question; answer} pairs with feedback scores exceeding a preset threshold from the FAQ data; Construct a supervised learning dataset based on the selected {question; answer} pairs; Fine-tune large language models using supervised learning datasets.

6. The optimization method of a large language model question answering system according to claim 1, characterized in that: The reinforcement learning fine-tuning includes the following steps: Based on the {question; answer; feedback} dataset, a reward model is trained to provide feedback scores on the answer performance of the large language model based on the question input; Based on the reinforcement learning algorithm, the reward model is used to provide feedback and score on the answer output of the large language model, which is used to fine-tune the large language model to maximize the expected feedback score.

7. The optimization method of a large language model question answering system according to claim 1, characterized in that: The tuning is started in a regular or quantitative manner and includes the following steps: Perform timing, and when the timing reaches the set time, start a new round of tuning and reset the timing; or measure the collected FAQ data, and when the measurement reaches the set amount, start a new round of tuning and reset the measurement; After each round of tuning, the tuned large language model weights are used to update the currently used large language model weights, and the FAQ data saved in the FAQ module is cleared.

Images