Question and answer model training method and question and answer method

Abstract

The invention provides a question and answer model training method and a question and answer method, and relates to the technical field of artificial intelligence, in particular to the technical field of natural language understanding and deep learning. A specific embodiment of the question and answer model training method comprises the steps of obtaining a sample session, wherein the sample session comprises a sample question and a sample answer; inputting the sample session into a retrieval enhancement generation model, and outputting a prediction evidence and a prediction answer; based on the sample answer, performing multi-dimensional evaluation on the prediction evidence and the prediction answer to generate a verifiable reward; and adjusting parameters of the retrieval enhancement generation model based on the verifiable rewards, and iteratively training and generating a question and answer model.

Description

Technical Field

[0001] This disclosure relates to the field of artificial intelligence technology, and more particularly to the field of natural language understanding and deep learning technology. Background Technology

[0002] In enterprise knowledge service scenarios, RAG (Retrieval-Augmented Generation) systems have become a core technological support for products such as intelligent customer service platforms and enterprise-level knowledge assistants. Their core objective is to provide users with accurate and efficient knowledge-based Q&A services through the collaboration of retrieval and generation. Currently, mainstream RAG systems generally employ static similarity retrieval mechanisms and offline fine-tuning modes to build their technical frameworks. While these systems are practical for handling general domain Q&A needs, they lack effective data utilization mechanisms to construct verifiable monitoring signals in the face of complex and ever-changing business contexts within enterprise scenarios, as well as the massive amounts of business data generated across the entire "self-service—human customer service—remote support" chain. This makes it difficult to achieve continuous adaptation and dynamic optimization to business scenarios.

[0003] The technical implementation paths of existing enterprise intelligent customer service and RAG systems have formed relatively mature industry practices, mainly including three core solutions: First, based on vector retrieval or keyword retrieval technology, relevant document information is retrieved from data sources such as FAQ (Frequently Asked Questions) databases and professional knowledge bases, and then the retrieved information is integrated and generated through a large language model to form the final answer; Second, for specific industry or enterprise-specific business scenarios, the basic model is supervised and fine-tuned through a small amount of manually labeled question and answer data to achieve model adaptation and performance improvement in specific domains; Third, some systems introduce coarse-grained feedback data such as user click-through rates and satisfaction ratings as optimization criteria to make simple adjustments to the search result ranking rules or model parameters to adapt to the needs of actual use scenarios. Summary of the Invention

[0004] This disclosure provides a question-answering model training method, question-answering method, apparatus, device, storage medium, and program product.

[0005] In a first aspect, embodiments of this disclosure propose a question-answering model training method, comprising: acquiring sample sessions, the sample sessions including sample questions and sample answers; inputting the sample sessions into a retrieval enhancement generation model, outputting predicted evidence and predicted answers; performing multi-dimensional evaluation on the predicted evidence and predicted answers based on the sample answers, generating verifiable rewards; adjusting the parameters of the retrieval enhancement generation model based on the verifiable rewards, and iteratively training the question-answering model.

[0006] Secondly, embodiments of this disclosure propose a question-answering method, comprising: receiving a target question input by a user; inputting the target question into a question-answering model and outputting a target answer, wherein the question-answering model is trained using the method described in the first aspect.

[0007] Thirdly, embodiments of this disclosure propose a question-answering model training apparatus, comprising: an acquisition module configured to acquire sample sessions, the sample sessions including sample questions and sample answers; a retrieval module configured to input the sample sessions into a retrieval-enhanced generative model and output predicted evidence and predicted answers; an evaluation module configured to perform multi-dimensional evaluation of the predicted evidence and predicted answers based on the sample answers and generate verifiable rewards; and a training module configured to adjust the parameters of the retrieval-enhanced generative model based on the verifiable rewards and iteratively train the question-answering model.

[0008] Fourthly, embodiments of this disclosure provide a question-answering device, comprising: a receiving module configured to receive a target question input by a user; and a question-answering module configured to input the target question into a question-answering model and output a target answer, wherein the question-answering model is trained using the device described in the third aspect.

[0009] Fifthly, embodiments of this disclosure provide an electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the methods described in the first or second aspect.

[0010] In a sixth aspect, embodiments of this disclosure provide a non-transitory computer-readable storage medium storing computer instructions for causing a computer to perform the methods described in the first or second aspect.

[0011] In a seventh aspect, embodiments of this disclosure provide a computer program product including a computer program that, when executed by a processor, implements the method as described in the first or second aspect.

[0012] The key or essential features of the embodiments disclosed herein are not intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0013] Other features, objects, and advantages of this disclosure will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings. The drawings are provided for a better understanding of the invention and are not intended to limit the scope of this disclosure. Wherein: Figure 1 This is a flowchart of an embodiment of the question-answering model training method according to the present disclosure; Figure 2 This is a flowchart of yet another embodiment of the question-answering model training method according to the present disclosure; Figure 3 This is a flowchart of one embodiment of the question-and-answer method according to this disclosure; Figure 4 This is a schematic diagram of the structure of an embodiment of the question-answering model training device according to the present disclosure; Figure 5 This is a schematic diagram of the structure of an embodiment of the question-and-answer device according to the present disclosure; Figure 6 This is a block diagram of an electronic device used to implement the question-answering model training method of the embodiments of this disclosure. Detailed Implementation

[0014] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0015] It should be noted that, unless otherwise specified, the embodiments and features described in this disclosure can be combined with each other. This disclosure will now be described in detail with reference to the accompanying drawings and embodiments.

[0016] Figure 1 A flow 100 of an embodiment of a question-answering model training method according to the present disclosure is shown. The question-answering model training method may include the following steps: Step 101: Obtain a sample session.

[0017] In this embodiment, the entity executing the question-answering model training method can obtain sample sessions.

[0018] Sample conversations can be obtained from end-to-end business data within an enterprise's existing systems, including but not limited to: user chat logs with self-service chatbots, chat logs with human customer service representatives, remote or on-site support tickets, and final solution records. Sample conversations can include sample questions and sample answers. Sample questions may contain industry jargon, abbreviations, error codes, internal system names, or other enterprise-specific expressions, while sample answers represent the effective solutions or outcomes for those questions.

[0019] In some embodiments, sample sessions may include, but are not limited to, at least one of the following: self-service closed-loop sessions, human closed-loop sessions, and rejection-response sessions. Specifically, a self-service closed-loop session (S1 type) can refer to a problem that closes at the self-service stage after multiple rounds of questioning without being transferred to a human agent, and is considered a model-solvable problem and an example of a valid solution; a human closed-loop session (S2 type) can refer to a problem where, after the user transfers to a human agent, the agent provides an answer solely through text or knowledge documents, and is considered a problem that can be solved by optimizing the RAG; a rejection-response session (S3 type) can refer to a problem that ultimately requires remote support, complex troubleshooting, or high-risk operations, and is considered a rejection-response session that guides escalation. These category labels can be used as data filtering criteria to select core data that is more valuable for model training.

[0020] Step 102: Input the sample session into the retrieval enhancement generative model and output the predicted evidence and predicted response.

[0021] In this embodiment, the aforementioned execution entity can input sample sessions into a retrieval-enhanced generative model, outputting predicted evidence and predicted responses. Predicted evidence can be relevant information retrieved and filtered by the model from various data sources to support the response. Predicted responses can be responses to the sample questions generated by the model based on the predicted evidence.

[0022] In some embodiments, the retrieval enhancement generative model may include a query understanding sub-model, a retrieval planning sub-model, an evidence evaluation sub-model, a response generation sub-model, and a decision-making sub-model. Each sub-model is an artificial intelligence model with specified capabilities, supporting individual or joint training to collaboratively complete the entire process from question understanding to response generation. Specifically, the query understanding sub-model is responsible for parsing and standardizing special expressions in sample questions; the retrieval planning sub-model is responsible for formulating retrieval strategies and executing retrieval; the evidence evaluation sub-model is responsible for screening high-value evidence; the response generation sub-model is responsible for generating structured responses; and the decision-making sub-model is responsible for judging the validity of responses and determining the output method.

[0023] Step 103: Based on the sample responses, perform multi-dimensional evaluation of the predicted evidence and predicted responses to generate verifiable rewards.

[0024] In this embodiment, the aforementioned execution entity can perform multi-dimensional evaluation of the predicted evidence and predicted answers based on sample responses and the category labels of the sample sessions, generating verifiable rewards. These rewards can be fine-grained feedback signals that quantify the effectiveness of model behavior, meeting the requirements of verifiability and traceability, and are used to guide model optimization strategies.

[0025] Evaluation dimensions may include, but are not limited to: the format and standardization of predicted answers, the retrieval contribution of predicted evidence, and the accuracy and fidelity of predicted answers. Different evaluation dimensions correspond to different types of rewards, which are ultimately integrated into a total verifiable reward through preset weighting coefficients, ensuring that the reward signal comprehensively reflects the model's performance at each stage.

[0026] Step 104: Adjust the parameters of the retrieval enhancement generative model based on verifiable rewards, and iteratively train the question-answering model.

[0027] In this embodiment, the aforementioned execution entity can adjust the parameters of the retrieval enhancement generative model based on verifiable rewards, and iteratively train the question-answering model. The core logic of model parameter adjustment can be to minimize the loss function using gradient descent, where the loss function is essentially the negative of the expected reward; that is, minimizing the loss maximizes the cumulative reward.

[0028] During the iterative training process, the initial stage uses filtered historical S1, S2, and S3 session data for offline reinforcement learning to train the initial strategy; subsequently, online data can be combined for continuous optimization to ensure that the model can continuously adapt to changes in the enterprise's business context and improve its performance in complex scenarios.

[0029] This disclosure provides a question-answering model training method. By hierarchically labeling and precisely filtering sample conversations, combined with multi-dimensional verifiable reward signals, the method drives the retrieval enhancement generation model to perform iterative optimization. This effectively improves the model's adaptability to special expressions within enterprises, as well as the accuracy and fidelity of its answers. At the same time, it achieves safe and controllable handling of high-risk questions, solving the problems of insufficient retrieval and unbalanced rejection strategies in traditional RAG systems in enterprise scenarios.

[0030] Continue to refer to Figure 2 This illustrates a flow 200 of yet another embodiment of the question-answering model training method according to the present disclosure. The question-answering model training method may include the following steps: Step 201: Obtain a sample session.

[0031] In this embodiment, the specific operation of step 201 has been described. Figure 1 The steps in step 101 of the illustrated embodiment are described in detail and will not be repeated here.

[0032] Step 202: Input the sample question into the query understanding sub-model and output the normalized query representation.

[0033] In this embodiment, the aforementioned execution entity can input sample questions into the query understanding sub-model and output standardized query representations and candidate semantic tags. The query understanding sub-model can be built upon existing models, fine-tuned by supplementing it with industry-specific samples such as jargon, abbreviations, and error codes, and combined with enterprise configuration information and a search database to achieve accurate understanding and standardized transformation of sample questions.

[0034] In some embodiments, the sample question undergoes format conversion to generate a standardized query representation. Typically, special expressions in the sample question, such as industry jargon, abbreviations, error codes, and internal system names, are parsed and converted into a generalized, standardized query representation that the model can understand. Corresponding candidate semantic tags (such as permission issues, process inquiries, troubleshooting, etc.) are then generated to provide a basis for subsequent retrieval planning. The training method for this sub-model is flexible; it can be trained independently or jointly with other sub-models, depending on the overall training requirements.

[0035] Step 203: Input the normalized query representation into the retrieval planning sub-model and output candidate evidence.

[0036] In this embodiment, the aforementioned execution entity can input standardized query representations and candidate semantic labels into the retrieval planning sub-model and output candidate evidence. The core function of the retrieval planning sub-model is to formulate an efficient retrieval plan, ensuring accurate retrieval of relevant information from various data sources.

[0037] In some embodiments, based on the normalized query representation, a retrieval data source is first selected according to priority, and then a retrieval strategy is selected based on the selected retrieval data source to perform the retrieval and obtain candidate evidence.

[0038] Typically, data source priorities are determined based on normalized query representations and candidate semantic tags, and data sources (including but not limited to FAQ databases, knowledge bases, configuration databases, and historical work order databases) are selected according to priority. For example, the priority rules can be set by default as follows: FAQ database (fastest response time, most concise answers) → Knowledge base (containing detailed information such as SOP (Standard Operating Procedure) documents and product manuals) → Configuration database (storage system configurations, permission rules, etc.) → Historical work order database (containing scenario-based cases). These priorities can be dynamically adjusted based on candidate semantic tags; for example, troubleshooting questions can increase the priority of the configuration database and historical work order database.

[0039] After selecting the data source, a corresponding retrieval strategy (including but not limited to multi-round query rewrite, Top-K dynamic adjustment, and adaptive filtering) is chosen based on the characteristics of the data source and the query complexity. Among these, multi-round query rewrite can be applied to fuzzy and incomplete complex problems by generating subqueries to gradually refine the retrieval scope; Top-K dynamic adjustment can set different numbers of returned results according to the query type to balance accuracy and comprehensiveness; adaptive filtering can filter irrelevant or expired documents based on document metadata (such as update time and related scenarios).

[0040] After performing the search operation, if the search results do not meet the completeness requirements (such as missing key information), a multi-hop search is triggered. By extracting key entities and logical relationships from the previous round of results, a subquery is generated to continue supplementing the search until sufficient information is obtained or the preset search round limit is reached. Finally, all search results are summarized to obtain candidate evidence.

[0041] Step 204: Input the candidate evidence into the evidence evaluation sub-model and output the predicted evidence.

[0042] In this embodiment, the aforementioned executing entity can input candidate evidence into the evidence evaluation sub-model and output predicted evidence. The evidence evaluation sub-model can filter high-value evidence through multi-dimensional evaluation, providing reliable support for subsequent response generation.

[0043] In some embodiments, candidate evidence is subjected to relevance assessment, consistency assessment, and denoising assessment to generate assessment results; based on the assessment results, predictive evidence is selected from the candidate evidence. Specifically, relevance assessment measures the degree of match between candidate evidence and the sample question; consistency assessment determines whether the content direction of candidate evidence is consistent with the bias of the sample question; and denoising assessment removes distractors, irrelevant content, and duplicate information from the candidate evidence.

[0044] Step 205: Input the predicted evidence into the response generation sub-model and output candidate answers.

[0045] In this embodiment, the aforementioned executing entity can input predictive evidence into the response generation sub-model and output candidate answers. The core objective of the response generation sub-model is to generate answers that conform to enterprise standards, are clearly structured, and have accurate content based on predictive evidence.

[0046] In some embodiments, the predictive evidence is formatted to generate candidate responses. Typically, based on corporate guidelines, valid information from the predictive evidence is integrated to generate structured candidate responses that include step-by-step instructions, precautions, risk warnings, and cited sources. The generation process strictly adheres to corporate-defined format constraints to ensure the responses are standardized and readable, while avoiding sensitive promises or non-compliant statements.

[0047] Step 206: Input the predicted evidence and predicted response into the decision sub-model and output the decision result.

[0048] In this embodiment, the aforementioned executing entity can input predictive evidence and predictive responses into the decision sub-model and output decision results. The decision sub-model is responsible for judging the validity of the predictive responses, determining the final output method, and ensuring that the model can make reasonable decisions in complex scenarios.

[0049] In some embodiments, the predicted evidence and predicted answers are evaluated for sufficiency, answer uncertainty estimation, and similar session category label distribution to generate a decision result. Typically, the predicted evidence is evaluated for sufficiency by verifying the matching degree between the candidate answer's basis and the search content through methods such as text vector matching and literal matching; if the matching degree is low, the evidence is deemed insufficient. The candidate answers are evaluated for answer uncertainty by calculating the uncertainty of the answer; if the uncertainty exceeds a threshold, a secondary evaluation is triggered. Simultaneously, the category label distribution of similar sessions is referenced, combined with the handling methods of historical cases, to assist in decision-making. The decision result may include, but is not limited to, at least one of the following: continue searching (when evidence is insufficient and the maximum number of searches has not been reached), answer directly (when evidence is sufficient and uncertainty is below the threshold), politely refuse to answer (when there is no valid search information or it is a high-risk question), suggest transferring to human assistance (when a human text or knowledge document answer is required), or suggest transferring to remote support (when complex troubleshooting or high-risk operations are required).

[0050] Step 207: Generate a predicted answer based on the candidate answers and the decision results.

[0051] In this embodiment, the aforementioned execution entity can generate a predicted answer based on candidate answers and decision results. Specifically, if the decision result is a direct answer, the candidate answer is used as the predicted answer; if the decision result is a polite refusal, a suggestion to transfer to human assistance, or a suggestion to transfer to remote support, a corresponding guiding statement is generated as the predicted answer; if the decision result is to continue searching, the process returns to step 203 and is re-executed until the final predicted answer is generated.

[0052] Step 208: Perform format validation on the predicted answer and generate a format reward.

[0053] In this embodiment, the aforementioned executing entity can perform format verification on the predicted answer and generate a format reward. This format reward can be an important component of the verifiable reward, used to ensure the structured and compliant nature of the predicted answer.

[0054] The validation process can employ a rule-based approach supplemented by a lightweight model. Rule-based validation may include, but is not limited to: text and image layout, hyperlink formatting, line break standards, integrity of source code identifiers, sensitive information avoidance, and proper ending conventions. The lightweight model can utilize a rendered visual model to evaluate the aesthetics and readability of the format, compensating for the limitations of rules in evaluating unstructured formats. Based on the validation results, corresponding reward scores are assigned: full marks are awarded for complete compliance, partial marks for minor formatting flaws, and negative marks for serious formatting non-compliance.

[0055] Step 209: Evaluate the effectiveness of the prediction evidence and generate a gain reward.

[0056] In this embodiment, the aforementioned executing entity can evaluate the effectiveness of the retrieval contribution of predicted evidence and generate a gain reward. This gain reward can be used to quantify the contribution of the retrieval action to the effectiveness of the final answer, guiding the optimization of the retrieval strategy.

[0057] The evaluation process can be based on the relevance, consistency, and denoising evaluation results of the evidence evaluation sub-model, as well as the evidence sufficiency evaluation results of the decision-making sub-model. It can also determine the contribution of each retrieval action by predicting the source retrieval rounds of documents cited in the answers, statistically analyzing the retrieval rounds with the highest citation ratio. Retrieval strategies that guide the correct answer fragments of standard solutions or S1 or S2 types (manifested as increased coverage of standard solution fragments, increased relevance, and a decrease in the proportion of noisy documents) are positively rewarded; retrieval strategies that repeatedly recall irrelevant documents and cause context pollution are negatively rewarded. The calculation of coverage, relevance, and the proportion of noisy documents uses existing mature algorithms, which will not be elaborated here.

[0058] Step 210: Evaluate the quality of the predicted answers and generate quality rewards.

[0059] In this embodiment, the aforementioned executing entity can perform a quality assessment on the accuracy and fidelity of the predicted answer and generate a quality reward. This quality reward can serve as a key signal guiding the model to improve the core quality of its answers.

[0060] In some embodiments, if the sample session is a self-service closed-loop question answering or human-assisted closed-loop question answering, the predicted evidence and predicted answer are input into the performance evaluation model, which outputs a performance evaluation score. Based on the performance evaluation score, a quality reward is generated. Typically, if the sample session is a self-service closed-loop question answering or human-assisted closed-loop question answering, the sample question, sample answer, predicted evidence, and predicted answer can be input into a high-performance evaluation model (which may use an existing state-of-the-art model or a proprietary model trained on proprietary data). Evaluation scores are generated from dimensions such as accuracy, completeness, consistency, and no illusions. A quality reward is generated based on the evaluation score, with high-similarity answers receiving high positive rewards and answers that contradict the sample answer or lack key information being penalized.

[0061] In some embodiments, if the sample session is a question-and-answer type that should be rejected, a negative reward is given to the predicted answer that provides a definitive answer, and a positive reward is given to the predicted answer that rejects the question. Typically, if the sample session is a question-and-answer type, a high negative reward can be given to the predicted answer that generates a specific action plan or a false definitive answer, and a positive reward can be given to the predicted answer that reasonably rejects the question or guides escalation (the reward weight is lower than the positive reward of S1 / S2 type samples).

[0062] In some embodiments, an evidence acceptance rate is generated based on predicted evidence and predicted responses, and a quality reward is generated based on the evidence acceptance rate. Typically, the evidence acceptance rate is generated by calculating the number of citations and the accuracy of citations of the predicted evidence by the predicted responses. If each paragraph of the predicted response can provide the corresponding search segment location and the citations are accurate and have high coverage, it is judged as a high acceptance rate and is given a positive reward; if the acceptance rate is low or the citations are incorrect, a negative reward is given, thereby further improving the fidelity of the response through this indicator.

[0063] Step 211: Adjust the parameters of the retrieval enhancement generative model based on verifiable rewards, and iteratively train the question-answering model.

[0064] In this embodiment, the specific operation of step 211 has been described. Figure 1 Step 104 in the illustrated embodiment is described in detail and will not be repeated here.

[0065] This disclosure provides a question-answering model training method that achieves refined processing of the entire process from question understanding, retrieval planning, evidence screening to answer generation and decision-making through the collaborative work of multiple sub-models. Combined with multi-dimensional verifiable reward signals, it ensures that the model can continuously optimize its strategies and provide more accurate, secure and reliable question-answering services in enterprise internal knowledge service scenarios.

[0066] Further reference Figure 3 The diagram illustrates a flow 300 of an embodiment of a question-and-answer method according to the present disclosure. The question-and-answer method may include the following steps: Step 301: Receive the target question input by the user.

[0067] In this embodiment, the entity executing the question-and-answer method can receive the target question input by the user. The target question can originate from enterprise users and may contain industry jargon, abbreviations, error codes, internal system names, and other special expressions, covering various enterprise business scenarios such as permission inquiries, process handling, and troubleshooting.

[0068] Step 302: Input the target question into the question-answering model and output the target answer.

[0069] In this embodiment, the aforementioned execution entity can input the target question into the question-answering model and output the target answer. The question-answering model can utilize collaborative processing of multiple sub-models to ensure a fast and accurate response to the target question, while also providing appropriate handling for complex or high-risk questions.

[0070] Question answering models can utilize Figure 1 or Figure 2 The described method is trained, and its processing flow is consistent with the logic of the training phase: First, the query understanding sub-model parses the special expressions in the target question to generate a standardized query representation and candidate semantic labels; then, the retrieval planning sub-model formulates a retrieval plan based on the results, recalls relevant information from various data sources, and supplements and improves it through multi-hop retrieval; the evidence evaluation sub-model filters high-value predictive evidence; the response generation sub-model generates structured candidate answers; and the decision-making sub-model combines the sufficiency of evidence, the uncertainty of the answer, and the distribution of similar case labels to determine the final output method and generate the target answer.

[0071] In some embodiments, in response to determining that the question-answering model meets a preset safety threshold, the question-answering model is incrementally updated with a preset step size. Typically, after the model is launched, new session data is continuously sampled and evaluated. In response to determining that the running trajectory of the question-answering model meets the preset safety threshold (e.g., the evaluation model score meets the standard and does not involve high risk), a low learning rate is used to perform small incremental updates on the question-answering model to ensure the stability of model iteration and avoid large deviations.

[0072] In some embodiments, two question-answering models are replicated, serving as a baseline model and a reinforcement learning model, respectively. The baseline model provides question-answering services. The reinforcement learning model is iteratively updated at preset time intervals, and the updated reinforcement learning model replaces the baseline model. Typically, two question-answering models are replicated, serving as both the baseline and reinforcement learning models. The baseline model stably provides question-answering services, ensuring business continuity. The reinforcement learning model is iteratively updated based on new session data. During the update process, A / B testing or bandit methods can be used to gradually scale up and verify the effectiveness. Once the evaluation shows satisfactory results, the updated reinforcement learning model replaces the baseline model, achieving seamless online model updates and ensuring the model continuously adapts to changes in the enterprise's business context, thereby improving service quality.

[0073] This disclosure provides a question-answering method that effectively improves the response capability to complex questions from enterprise users by employing a question-answering model trained through reinforcement learning. The accuracy, fidelity, and compliance of the answers are superior, and the method also has continuous self-evolution capabilities, enabling it to provide high-quality knowledge question-answering services to enterprises in a long-term and stable manner.

[0074] Further reference Figure 4As an implementation of the methods shown in the above figures, this disclosure provides an embodiment of a question-answering model training device, which is similar to... Figure 1 Corresponding to the method embodiments shown, this device can be specifically applied to various electronic devices.

[0075] like Figure 4 As shown, the question-answering model training device 400 of this embodiment may include: an acquisition module 401, a retrieval module 402, an evaluation module 403, and a training module 404. The acquisition module 401 is configured to acquire sample sessions, which include sample questions and sample answers; the retrieval module 402 is configured to input the sample sessions into a retrieval-enhanced generative model and output predicted evidence and predicted answers; the evaluation module 403 is configured to perform multi-dimensional evaluation of the predicted evidence and predicted answers based on the sample answers and generate verifiable rewards; the training module 404 is configured to adjust the parameters of the retrieval-enhanced generative model based on the verifiable rewards and iteratively train the question-answering model.

[0076] In this embodiment, the specific processing of the acquisition module 401, retrieval module 402, evaluation module 403, and training module 404 in the question-answering model training device 400, and the resulting technical effects, can be found by referring to [reference needed]. Figure 1 The relevant descriptions of steps 101-104 in the corresponding embodiments will not be repeated here.

[0077] In some optional implementations of this embodiment, the sample session includes at least one of the following: self-service closed-loop session, manual closed-loop session, and response rejection session.

[0078] In some optional implementations of this embodiment, the retrieval enhancement generative model includes a query understanding sub-model, a retrieval planning sub-model, an evidence evaluation sub-model, a response generation sub-model, and a decision-making sub-model.

[0079] In some optional implementations of this embodiment, the retrieval module 402 is further configured to: input the sample question into the query understanding sub-model and output a normalized query representation; input the normalized query representation into the retrieval planning sub-model and output candidate evidence; input the candidate evidence into the evidence evaluation sub-model and output predicted evidence; input the predicted evidence into the response generation sub-model and output candidate answers; input the predicted evidence and predicted answers into the decision sub-model and output a decision result; and generate a predicted answer based on the candidate answers and the decision result.

[0080] In some optional implementations of this embodiment, the retrieval module 402 is further configured to: perform format conversion on the sample question to generate a standardized query representation.

[0081] In some optional implementations of this embodiment, the retrieval module 402 is further configured to: first select a retrieval data source according to priority based on the normalized query representation, and then select a retrieval strategy based on the selected retrieval data source to perform the retrieval and obtain candidate evidence.

[0082] In some optional implementations of this embodiment, the retrieval module 402 is further configured to: perform relevance assessment, consistency assessment and denoising assessment on candidate evidence, and generate assessment results; Based on the evaluation results, predictive evidence is selected from the candidate evidence.

[0083] In some optional implementations of this embodiment, the retrieval module 402 is further configured to: perform format conversion on the predicted evidence and generate candidate answers.

[0084] In some optional implementations of this embodiment, the retrieval module 402 is further configured to: perform evidence sufficiency, answer uncertainty estimation and similar session category label distribution on the predicted evidence and predicted answer, and generate a decision result, which includes at least one of the following: continue retrieval, answer directly, refuse to answer, transfer to human, and transfer to remote.

[0085] In some optional implementations of this embodiment, the evaluation module 403 is further configured to: perform format validation on the predicted answer and generate a format reward; and / or evaluate the validity of the predicted evidence and generate a gain reward; and / or evaluate the quality of the predicted answer and generate a quality reward.

[0086] In some optional implementations of this embodiment, the evaluation module 403 is further configured to: if the sample session is a self-service closed-loop question answering or a manual closed-loop question answering, input the predicted evidence and predicted answer into the performance evaluation model, output a performance evaluation score, and generate a quality reward based on the performance evaluation score; and / or if the sample session is a question answering that should be rejected, assign a negative reward to the predicted answer for a definitive answer and a positive reward to the predicted answer for a rejected answer; and / or generate an evidence adoption rate based on the predicted evidence and predicted answer, and generate a quality reward based on the evidence adoption rate.

[0087] Further reference Figure 5 As an implementation of the methods shown in the above figures, this disclosure provides an embodiment of a question-and-answer device, which is similar to... Figure 3 Corresponding to the method embodiments shown, this device can be specifically applied to various electronic devices.

[0088] like Figure 5As shown, the question-answering device 500 of this embodiment may include a receiving module 501 and a question-answering module 502. The receiving module 501 is configured to receive a target question input by a user; the question-answering module 502 is configured to input the target question into a question-answering model and output a target answer. The question-answering model utilizes... Figure 4 The device described was trained.

[0089] In this embodiment, the specific processing of the receiving module 501 and the question-and-answer module 502 in the question-and-answer device 500 and the resulting technical effects can be referred to respectively. Figure 3 The relevant descriptions of steps 301-302 in the corresponding embodiments will not be repeated here.

[0090] In some optional implementations of this embodiment, the question-answering device 500 further includes: a first update module, configured to perform incremental updates of the question-answering model with a preset step size in response to determining that the question-answering model meets a preset security threshold.

[0091] In some optional implementations of this embodiment, the question-answering device 500 further includes: a second update module, configured to copy two question-answering models, respectively serving as a baseline model and a reinforcement learning model, providing question-answering services through the baseline model; iteratively updating the reinforcement learning model every preset time period, and replacing the baseline model with the updated reinforcement learning model.

[0092] The collection, storage, use, processing, transmission, provision, and disclosure of any type of information, such as user personal information, in this technical solution comply with relevant laws and regulations and do not violate public order and good morals.

[0093] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0094] Figure 6 A schematic block diagram of an example electronic device 600 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0095] like Figure 6As shown, device 600 includes a computing unit 601, which can perform various appropriate actions and processes based on a computer program stored in read-only memory (ROM) 602 or a computer program loaded into random access memory (RAM) 603 from storage unit 608. RAM 603 may also store various programs and data required for the operation of device 600. The computing unit 601, ROM 602, and RAM 603 are interconnected via bus 604. Input / output (I / O) interface 605 is also connected to bus 604.

[0096] Multiple components in device 600 are connected to I / O interface 605, including: input unit 606, such as keyboard, mouse, etc.; output unit 607, such as various types of monitors, speakers, etc.; storage unit 608, such as disk, optical disk, etc.; and communication unit 609, such as network card, modem, wireless transceiver, etc. Communication unit 609 allows device 600 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0097] The computing unit 601 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 601 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 601 performs the various methods and processes described above, such as question-answering model training methods. For example, in some embodiments, the question-answering model training method may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 608. In some embodiments, part or all of the computer program may be loaded and / or installed on device 600 via ROM 602 and / or communication unit 609. When the computer program is loaded into RAM 603 and executed by the computing unit 601, one or more steps of the question-answering model training method described above may be performed. Alternatively, in other embodiments, the computing unit 601 may be configured to perform question-answering model training methods by any other suitable means (e.g., by means of firmware).

[0098] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0099] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0100] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0101] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0102] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0103] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, distributed system servers, or servers incorporating blockchain technology.

[0104] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution provided in this disclosure can be achieved, and this is not limited herein.

[0105] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A method for training a question and answer model, comprising: obtaining a sample session, the sample session comprising a sample question and a sample answer; inputting the sample session into a retrieval and generation model to output predicted evidence and a predicted answer; based on the sample answer, performing multi-dimensional evaluation on the predicted evidence and the predicted answer to generate a verifiable reward; based on the verifiable reward, adjusting parameters of the retrieval and generation model to iteratively train a question and answer model.

2. The method of claim 1, wherein, The sample session comprises at least one of the following: a self-closed loop session, a human-closed loop session, and a rejected answer session.

3. The method of claim 1, wherein, The retrieval and generation model comprises a query understanding sub-model, a retrieval planning sub-model, an evidence evaluation sub-model, an answer generation sub-model, and a decision sub-model.

4. The method of claim 3, wherein, The inputting the sample session into a retrieval and generation model to output predicted evidence and a predicted answer comprises: inputting the sample question into the query understanding sub-model to output a normalized query representation; inputting the normalized query representation into the retrieval planning sub-model to output candidate evidence; inputting the candidate evidence into the evidence evaluation sub-model to output the predicted evidence; inputting the predicted evidence into the answer generation sub-model to output a candidate answer; inputting the predicted evidence and the predicted answer into the decision sub-model to output a decision result; based on the candidate answer and the decision result, generating the predicted answer.

5. The method of claim 4, wherein, The inputting the sample question into the query understanding sub-model to output a normalized query representation comprises: performing format conversion on the sample question to generate the normalized query representation.

6. The method of claim 4, wherein, The inputting the normalized query representation into the retrieval planning sub-model to output candidate evidence comprises: based on the normalized query representation, first selecting a retrieval data source according to a priority, and then performing retrieval according to the selected retrieval data source to obtain the candidate evidence.

7. The method of claim 4, wherein, The inputting the candidate evidence into the evidence evaluation sub-model to output the predicted evidence comprises: performing relevance evaluation, consistency evaluation, and noise reduction evaluation on the candidate evidence to generate an evaluation result; based on the evaluation result, filtering the predicted evidence from the candidate evidence.

8. The method of claim 4, wherein, The inputting the predicted evidence into the answer generation sub-model to output a candidate answer comprises: performing format conversion on the predicted evidence to generate the candidate answer.

9. The method of claim 4, wherein, The inputting the predicted evidence and the predicted answer into the decision sub-model to output a decision result comprises: performing evidence sufficiency, answer uncertainty estimation, and similar session category label distribution on the predicted evidence and the predicted answer to generate a decision result, the decision result comprising at least one of the following: continue retrieval, direct answer, reject answer, transfer to human, and transfer to remote.

10. The method of claim 1, wherein, The based on the sample answer, performing multi-dimensional evaluation on the predicted evidence and the predicted answer to generate a verifiable reward comprises: performing format verification on the predicted answer to generate a format reward; and / or performing effectiveness evaluation on the predicted evidence to generate a gain reward; and / or performing quality evaluation on the predicted answer to generate a quality reward.

11. The method of claim 10, wherein, The quality evaluation on the predicted answer generates a quality reward, including: If the sample session is a self-closed loop question and answer or a manual closed loop question and answer, the predicted evidence and the predicted answer are input into a performance evaluation model to output a performance evaluation score, and the quality reward is generated based on the performance evaluation score; and / or If the sample session is a should-reject answer question, a negative reward is given to a predicted answer of a deterministic answer, and a positive reward is given to a predicted answer of a rejection; and / or Based on the predicted evidence and the predicted answer, an evidence adoption rate is generated, and the quality reward is generated based on the evidence adoption rate.

12. A question and answer method, comprising: receiving a target question input by a user; inputting the target question into a question and answer model to output a target answer, the question and answer model being trained by the method of any one of claims 1-11.

13. The method of claim 12, wherein, The method further comprises: in response to determining that the question and answer model meets a preset safety threshold, performing an incremental update of the question and answer model by a preset step size.

14. The method of claim 12, wherein, The method further comprises: copying two question and answer models as a baseline model and a reinforcement learning model respectively, and providing a question and answer service through the baseline model; every preset time period, iteratively updating the reinforcement learning model, and replacing the baseline model with the updated reinforcement learning model.

15. A question and answer model training device, comprising: an acquisition module configured to acquire a sample session, the sample session comprising a sample question and a sample answer; a retrieval module configured to input the sample session into a retrieval and enhancement generation model to output predicted evidence and a predicted answer; an evaluation module configured to perform multi-dimensional evaluation on the predicted evidence and the predicted answer based on the sample answer to generate a verifiable reward; a training module configured to adjust parameters of the retrieval and enhancement generation model based on the verifiable reward to iteratively train a question and answer model.

16. A question and answer device, comprising: a receiving module configured to receive a target question input by a user; a question and answer module configured to input the target question into a question and answer model to output a target answer, the question and answer model being trained by the device of claim 15.

17. An electronic device, comprising: at least one processor; and a memory connected in communication with the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-11 or 12-14.

18. A non-transitory computer-readable storage medium storing computer instructions for causing a computer to perform the method of any one of claims 1-11 or 12-14.

19. A computer program product comprising a computer program which, when executed by a processor, implements the method of any one of claims 1-11 or 12-14.

Images