Question answer generation method and device, equipment and medium

Abstract

The invention relates to the technical field of intelligent decision making, can be applied to business system platforms such as financial science and technology, medical health and the like, and discloses a question answer generation method, device, equipment and medium, and the method comprises the steps: collecting a plurality of question answers generated by questions, collecting the question answers generated by each question into a question and answer sample set, calculating a reward model score and a reference model benchmark score of each question and answer sample in the question and answer sample set one by one, fitting into a confidence coefficient curve, and dividing an answer retention interval; screening answers to the questions according to the answer retention interval, and calculating and updating an answer retention proportion in the question and answer sample set; optimizing the question and answer model according to the answer retention proportion, and generating a plurality of candidate answers of the user question by using the optimized question and answer model; and calculating candidate reward scores, retaining candidate answers corresponding to the candidate reward scores in the answer retaining interval, and performing semantic fusion on the answers to generate a final question answer of the user question. The accuracy of question answer generation can be improved.

Description

Methods, apparatus, equipment and media for generating answers to questions Technical Field

[0001] This invention relates to the field of intelligent decision-making technology, and in particular to a method, apparatus, device, and medium for generating answers to questions. Background Technology

[0002] Policy filtering refers to the mechanism of selecting high-confidence samples to participate in policy updates and filtering low-confidence samples by judging the reliability of reward signals during the alignment process of large language models; answer generation is the process by which the model generates output content that meets the user's requirements based on the optimized policy.

[0003] In the fintech field, when the model generates different portfolio recommendations, it filters out low-confidence samples in the reward model that are inaccurate in predicting market fluctuations, ensuring that the output portfolio recommendations are more in line with actual abnormal preferences. Based on the filtered high-confidence strategy, it outputs investment solutions that are adapted to the user's abnormality level.

[0004] In the healthcare field, low-confidence samples with ambiguous matching of disease characteristics in the filtering reward model are used to output treatment references that conform to disease characteristics based on a reliable strategy.

[0005] Therefore, in existing technologies, answer generation in scenarios such as high-precision anomaly judgment in financial technology and professional medical and health diagnosis and treatment advice suffers from insufficient credibility of output content due to noise and instability of reward signals, and is prone to policy drift or incorrect optimization, resulting in low accuracy when generating answers to questions. Summary of the Invention

[0006] This invention provides a method, apparatus, device, and medium for generating answers to questions, in order to address the technical problem of low accuracy in answer generation.

[0007] Firstly, a question-answer generation method is provided, comprising: collecting multiple question-answer samples generated by a preset question-answering model for a target question within a preset question set, and concatenating the target question with the corresponding multiple question-answer samples to form a question-answer sample set; aggregating the question-answer samples generated for each target question within the preset question set into a question-answer sample set, and calculating the reward model score and reference model baseline score for each question-answer sample within the question-answer sample set; fitting the reward model score and the reference model baseline score into a confidence curve, and dividing the answer retention interval according to the confidence curve; filtering the question answers in the question-answer sample set according to the answer retention interval to form an updated question-answer sample set, and calculating the answer retention ratio in the updated question-answer sample set; optimizing the question-answering model according to the answer retention ratio, and using the optimized question-answering model to generate multiple candidate answers to a user question in a target scenario; calculating the candidate reward scores of the candidate answers, and retaining the candidate answers corresponding to the candidate reward scores in the answer retention interval to obtain a retained answer set; and performing semantic fusion on the answers in the retained answer set to generate the final question answer for the user question.

[0008] Secondly, a question-answer generation device is provided, comprising: a question-answer acquisition module, used to acquire multiple question-answers generated by a preset question-answer model for a target question within a preset question set, and to concatenate the target question with the corresponding multiple question-answers to form a question-answer sample; a question-answer sample set generation module, used to aggregate the question-answer samples generated for each target question within the preset question set into a question-answer sample set, and to calculate the reward model score and the reference model baseline score for each question-answer sample within the question-answer sample set; a confidence curve fitting module, used to fit the reward model score and the reference model baseline score into a confidence curve, and to divide the answer retention interval according to the confidence curve; and to calculate the answer retention ratio. The system comprises the following modules: a question-and-answer model optimization module, which optimizes the question-and-answer model based on the answer retention interval to form an updated question-and-answer sample set and calculates the answer retention ratio in the updated question-and-answer sample set; a question-and-answer model optimization module, which optimizes the question-and-answer model based on the answer retention ratio and uses the optimized question-and-answer model to generate multiple candidate answers to the user's question in the target scenario; a candidate reward score calculation module, which calculates the candidate reward score of the candidate answer and retains the candidate answer corresponding to the candidate reward score in the answer retention interval to obtain a retained answer set; and a question-and-answer fusion module, which performs semantic fusion on the answers in the retained answer set to generate the final question answer to the user's question.

[0009] Thirdly, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the above-described question answer generation method.

[0010] Fourthly, a computer-readable storage medium is provided, which stores a computer program that, when executed by a processor, implements the steps of the above-mentioned question answer generation method.

[0011] In the above-mentioned solution implemented by the question-answer generation method, apparatus, device, and medium, multiple answers generated by the question-answering model for a preset question can be collected through the client to form a question-answering sample set, and the reward score and reference benchmark score of each sample can be calculated. The two are fitted into a confidence curve and a high-confidence answer retention interval is defined. Based on this, samples are screened to obtain an updated set and the answer retention ratio is calculated. After optimizing the question-answering model based on the answer retention ratio, multiple candidate answers to the user's question are generated, their candidate reward scores are calculated, and only candidates falling within the answer retention interval are retained to form a retained answer set. The answers in this set are semantically fused to generate the final question answer. The final question answer is fed back to the client. In this invention, through dynamic interval division and sample screening based on the confidence curve, unreliable noise samples of the reward signal can be accurately filtered out, ensuring that the question-answering model is optimized only on high-quality, high-consistency feedback, thereby significantly improving the model's stability, convergence speed, and final performance on high-precision tasks. The trained answer retention interval serves as a lightweight and reliable online filter, quickly eliminating candidate answers with abnormally high or low scores from the reward model. This effectively shields low-quality or harmful outputs that may arise from input distribution shifts or model uncertainties, improving the security and reliability of the generated results. Semantic fusion preserves the essence of the answer set, ensuring that answers conform to human preferences while maintaining diversity and information completeness. This provides users with accurate and comprehensive final answers in practical applications. Attached Figure Description

[0012] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0013] Figure 1 is a schematic diagram of an application environment for a question-answer generation method according to an embodiment of the present invention; Figure 2 is a schematic flowchart of a question-answer generation method according to an embodiment of the present invention; Figure 3 is a schematic flowchart of a specific implementation of step S2 in Figure 2; Figure 4 is a schematic flowchart of a specific implementation of step S4 in Figure 2; Figure 5 is a schematic structural diagram of a question-answer generation device according to an embodiment of the present invention; Figure 6 is a schematic structural diagram of a computer device according to an embodiment of the present invention; Figure 7 is another schematic structural diagram of a computer device according to an embodiment of the present invention. Detailed Implementation

[0014] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0015] The question-answer generation method provided in this invention can be applied in the application environment shown in Figure 1, where the client communicates with the server via a network. The server can collect multiple answers generated by the question-answering model for a preset question through the client, forming a question-answering sample set, and calculate the reward score and reference baseline score for each sample. The two scores are fitted to a confidence curve, and a high-confidence answer retention interval is defined. Based on this, samples are filtered to obtain an updated set, and the answer retention ratio is calculated. After optimizing the question-answering model based on the answer retention ratio, multiple candidate answers to the user's question are generated, their candidate reward scores are calculated, and only candidates falling within the answer retention interval are retained, forming a retained answer set. Semantic fusion is performed on the answers in this set to generate the final question answer. The final question answer is fed back to the client. In this invention, through dynamic interval division and sample filtering based on the confidence curve, unreliable noise samples with positive reward signals can be accurately filtered out, ensuring that the question-answering model is optimized only on high-quality, highly consistent feedback, thereby significantly improving the model's stability, convergence speed, and final performance on high-precision tasks. The trained answer retention interval serves as a lightweight and reliable online filter, quickly eliminating candidate answers with abnormally high or low scores from the reward model. This effectively shields low-quality or harmful outputs that may arise from input distribution shifts or model uncertainties, improving the security and reliability of the generated results. Semantic fusion preserves the essence of the answer set, ensuring that answers conform to human preferences while maintaining diversity and information completeness, thus providing users with accurate and complete final answers in practical applications. The client can be, but is not limited to, various personal computers, laptops, smartphones, tablets, and portable wearable devices. The server can be implemented using a standalone server or a server cluster consisting of multiple servers. The invention is described in detail below through specific embodiments.

[0016] Please refer to Figure 2, which is a flowchart of a question-answer generation method provided in an embodiment of the present invention, including the following steps: S1, collecting multiple question answers generated by a preset question-answer model for a target question in a preset question set, and concatenating the target question with the corresponding multiple question answers to form a question-answer sample.

[0017] In this embodiment of the invention, the preset question-answering model is a supervised, fine-tuned large language model capable of generating corresponding text answers based on input text questions. It is a strategy model that needs optimization, and its parameters are continuously updated during training to better align with human preferences. The preset question set refers to a dataset containing multiple independent question texts, collected and constructed before training. The question-answering sample refers to a complete text sequence composed of a question text and a corresponding answer text generated by the question-answering model, concatenated according to a predetermined format.

[0018] In detail, a preset set of questions is read from the storage device, and the text of each target question is input sequentially or in parallel into the question-answering model that has been loaded into memory. By calling the forward inference function of the question-answering model and using a decoding strategy with randomness (such as top-p sampling), the same question is generated multiple times independently to obtain a set of candidate answer texts. The system concatenates the question with each candidate answer according to a predefined template (e.g., target question-candidate answer), and stores each concatenated string as an independent question-answering sample.

[0019] In this embodiment of the invention, multiple answers are concatenated into a question-answer sample, which creates a rich and realistic behavior-feedback environment for model alignment training. This allows the training signal to no longer depend on a single output, but to identify a stable and reliable optimization direction among multiple possible solutions to a question.

[0020] In the fintech field, for example, when dealing with the problem of explaining anomalies in a financial derivative, generating multiple versions of explanations can expose differences in the accuracy and compliance of the model's statements. This allows the reward model to learn more accurately the preference for compliance anomaly prompts and avoids training a model that may produce misleading simplified answers.

[0021] In the field of healthcare, when faced with the question of providing initial advice for a certain symptom, generating multiple answers can reveal the different tendencies of the model in terms of the conservatism, comprehensiveness, and safety of the advice. This helps the training process to filter out those answers that are both professional and cautious, reducing the probability of giving absolute or highly abnormal advice.

[0022] S2. Collect the question-and-answer samples generated for each target question in the preset question set into a question-and-answer sample set, and calculate the reward model score and reference model baseline score for each question-and-answer sample in the question-and-answer sample set.

[0023] In this embodiment of the invention, the question-and-answer sample set is a summary of all question-and-answer samples obtained by sampling target questions from all preset question sets. The reward model score is a scalar value output by the reward model after evaluating a single question-and-answer sample, used to quantify the degree to which the answer in that sample conforms to human preferences. The reference model baseline score is another scalar value output by a fixed reference model after evaluating the same question-and-answer sample, used to provide a stable and conservative quality benchmark, typically based on the inherent probability of the model generating the answer or the output of an earlier version of the reward model.

[0024] In detail, all independently generated question-and-answer samples are added to a unified list or tensor data structure, which is the set of question-and-answer samples for the current batch.

[0025] In this embodiment of the invention, as shown in FIG3, the step of calculating the reward model score and reference model baseline score of each question and answer sample in the question and answer sample set includes: S31, text encoding the question and answer sample to obtain a first question and answer sample vector; S32, pooling the question and answer sample vector to obtain a second question and answer sample vector; S33, mapping the second question and answer sample vector to the reward model score and the reference model baseline score using a preset reward model and a preset reference model, respectively.

[0026] In detail, the standardized concatenated text of each question-and-answer sample in the question-and-answer sample set is converted into a token sequence that the model can recognize. This sequence is then input into a pre-trained text encoder (such as the encoder of the Transformer model). The token sequence is semantically encoded using a self-attention mechanism, and the hidden representation sequence {h1,h2,…,h_T} corresponding to each token is output, which is the first question-and-answer sample vector.

[0027] Specifically, the first question-answer sample vector (token-level hidden representation sequence {h1,h2,…,h_T}) is aggregated using average pooling, max pooling, or by extracting the hidden state of the first question-answer sample vector position. This process converts the variable-length sequence features into a fixed-dimensional vector h_agg, which is the second question-answer sample vector.

[0028] Furthermore, the second question-answer sample vector h_agg is input into a small feedforward network (MLP) of the preset reward model, and mapped to a specified interval (such as [-1,1]) through a Tanh or Sigmoid activation function, outputting a scalar reward model score. At the same time, h_agg is input into a preset reference model. If the reference model is a stable version of RM, the score is output according to the same MLP mapping logic. If it is a basic language model, the log-likelihood or perplexity index is calculated and output as the reference model benchmark score. The reward model mapping process combines distribution alignment and correlation constraints with the reference model for secondary calibration to ensure that the reward scores at different training stages are comparable, which is different from the traditional reward model's independent scoring mode and lack of stability verification.

[0029] In this embodiment of the invention, centralized sample management and dual-dimensional score evaluation provide data support for the reliability verification of reward signals, ensuring that subsequent strategy optimization is based solely on high-confidence reward signals.

[0030] In the fintech field, for example, when training a model to generate investment report summaries, the reward model will give a score for the readability and attractiveness of the summary, while the reference model will give a score for its consistency with the facts of the original data. The combination of the two can filter out those samples that seem fluent but deviate from the facts, or are too conservative but accurate, ensuring the direction of model optimization.

[0031] In the healthcare field, responses to patient inquiries can be compared using a dual-score system to filter out those that downplay abnormalities in an attempt to appease patients or are overly technical and cause anxiety. This guides the model to generate responses that are both professional and considerate.

[0032] S3. Fit the reward model score and the reference model baseline score to a confidence curve, and divide the answer retention interval according to the confidence curve.

[0033] In this embodiment of the invention, the confidence curve is a two-dimensional function graph with the reward model score interval as the horizontal axis and the linear correlation strength between the reward score and the reference baseline score within that interval (usually represented by the average confidence coefficient R²) as the vertical axis. It displays the distribution of the confidence level of the reward signal across different reward value segments. The answer retention interval is a continuous score segment on the confidence curve where R² is higher than a threshold θ, and its upper and lower bounds are used as the confidence region for subsequent screening.

[0034] In this embodiment of the invention, fitting the reward model score and the reference model baseline score into a confidence curve includes: for each question-answer sample, calculating the confidence coefficients of multiple question answers in the question-answer sample based on the reward model score and the reference model baseline score; dividing the reward model score into score intervals according to the score distribution of the reward model score using a sliding window; calculating the average confidence coefficient corresponding to the confidence coefficient in each interval of the score interval; and plotting a confidence curve with the score interval as the horizontal axis and the average confidence coefficient as the vertical axis.

[0035] In detail, for each sample in the question-and-answer sample set, its reward model score and the reference model baseline score are extracted, and the determination coefficient, i.e., the confidence coefficient, for a single sample is calculated according to the following formula:

[0036] in, This represents the confidence coefficient. This represents the score of the reward model. Indicates the baseline score of the reference model. Indicates the sample number. This represents the mean of the reward model scores for all samples.

[0037] Specifically, the distribution range (from minimum to maximum) of all reward model scores in the statistical question-and-answer sample set is divided into several continuous and non-overlapping score intervals by sliding the window segment by segment, starting from the minimum score, according to a preset window size (e.g., 0.1 score interval units) and step size. This ensures that each interval contains a certain number of samples (avoiding statistical distortion caused by too few samples). The size and step size of the sliding window can be dynamically adjusted according to the distribution density of the reward scores. The window shrinks and the step size decreases in dense score areas, while the window expands and the step size increases in sparse score areas, reducing the problem of unreasonable interval division caused by the fixed window size in conventional methods.

[0038] Furthermore, for each score interval obtained through the sliding window, all samples whose reward scores fall within that interval are retrieved, and the corresponding confidence coefficients (R² values) are read. The arithmetic mean of these confidence coefficients is calculated to obtain the average confidence coefficient for that score interval. Through local averaging, the noise confidence estimate of a single sample is smoothed, resulting in a stable index representing the overall credibility level of that reward score segment, effectively reducing the impact of random fluctuations.

[0039] Next, a two-dimensional coordinate system is established, with the median (or interval identifier) ​​of each score interval as the horizontal axis and the calculated average confidence coefficient of the corresponding interval as the vertical axis. A series of discrete data points are plotted in the coordinate system, and these data points are connected by a curve fitting algorithm (such as polynomial fitting or spline interpolation) to form a continuous confidence curve.

[0040] Furthermore, a confidence threshold (such as 0.8) can be set, and the confidence curve can be traversed to find all consecutive horizontal axis (reward score) segments where the vertical axis (average confidence coefficient) is higher than the threshold. The reward score ranges corresponding to these segments can be merged to obtain the answer retention interval, such as [Rlow, Rhigh].

[0041] In this embodiment of the invention, the reliability of the reward signal is screened by dynamically fitting the confidence curve and dividing the answer retention interval, ensuring that subsequent training is based only on high-confidence samples, thereby improving the stability and convergence efficiency of model training.

[0042] In the fintech field, for example, when optimizing automated financial statement analysis models, the confidence curve may show that in the upper-middle reward score range, the model's analytical summary has the highest consistency with the factual reference. However, in the very high score range, although the reward model considers the summary to be excellent, the consistency with the reference fact decreases. In this case, the system will automatically classify the upper-middle score range as a reserve interval, prioritizing the use of this portion of the samples to optimize the model, avoiding being misled by high-scoring samples that may contain oversimplification or speculation.

[0043] In the healthcare field, confidence curves in responses to drug interaction queries may reveal a significant discrepancy between the reward model's scoring of "absolute safety claims" and the reference model's assessment (based on a rigorous medical knowledge base), while the scoring of conservative responses such as "listing potential abnormalities and recommending consultation with a doctor" shows high consistency. The system then designates the score range containing conservative responses as a trust zone, ensuring the model aligns with rigorous and safe practices and filtering out overly optimistic responses that may contain medical abnormalities.

[0044] S4. Filter the question answers in the question-and-answer sample set according to the answer retention interval to form an updated question-and-answer sample set, and calculate the answer retention ratio in the updated question-and-answer sample set.

[0045] In this embodiment of the invention, updating the question-and-answer sample set involves filtering out the questions and answers in the original question-and-answer sample set whose reward model scores fall outside the answer retention range. The remaining sample set is then used to drive parameter updates for the question-and-answer model. The answer retention ratio is a scalar value representing the proportion of high-confidence samples that are retained after policy filtering and used for model updates.

[0046] In detail, before model optimization (such as PPO update), samples whose reward signals are considered unreliable (low confidence) are actively removed. This reduces the pollution of policy model gradient updates by noise and misleading signals, and improves the stability of training, convergence speed, and the robustness and reliability of the final model on complex tasks.

[0047] In this embodiment of the invention, the step of filtering the question answers in the question-and-answer sample set according to the answer retention interval to form an updated question-and-answer sample set includes: traversing the reward model scores corresponding to the question answers in the question-and-answer sample set, and determining whether the reward model scores are within the answer retention interval; if they are within the answer retention interval, retaining the question answer corresponding to the reward model score and assigning a preset answer weight to the question answer; if they are not within the answer retention interval, not retaining the question answer corresponding to the reward model score; merging the retained question answers with the corresponding questions in the question-and-answer sample set to generate an updated question-and-answer sample set.

[0048] In detail, the reward model score associated with each sample in the original question-and-answer sample set is read sequentially. For each score, it is checked whether the score simultaneously satisfies the condition of being greater than or equal to the lower bound of the interval and less than or equal to the upper bound of the interval. The judgment result (yes or no) is recorded as the "reservation mark" of the sample.

[0049] Specifically, if a sample's "retention flag" is yes, the complete data of that sample (including questions, answers, etc.) is copied to a new buffer. The distance between its reward model score and the answer retention interval boundary is then determined. If the reward model score falls within the interval (distance to the boundary > δ, where δ is a preset small threshold), a filtering weight ω is assigned. i =1; if the reward model score falls near the boundary (distance from the boundary ≤ δ), then allocate 0 < ω. i =α<1 (α is a preset attenuation coefficient, such as 0.5), and store the sample information in association with the assigned weights.

[0050] Furthermore, if a sample's "retention label" is negative, the sample will be skipped, its data will not be copied to a new buffer, and the sample will not contribute to the loss function calculation of the policy model during that round of training.

[0051] Next, after iterating through all samples and completing the filtering, all the sample data marked as retained in the buffer are reorganized into a new, structured dataset. This dataset is the updated question-and-answer sample set. For example, the target question x for each sample is extracted. (k) Candidate answer y (k) j and the assigned filtering weights ω i According to the structured format of the original question-and-answer sample set (e.g., {x:x (k) ,y:y (k) j ,ω:ω i The system performs batch processing, automatically verifies the integrity of the sample format, removes samples with incorrect weight allocation, and finally generates an updated question-and-answer sample set.

[0052] Furthermore, the number of samples Nnew in the updated question-and-answer sample set and the number of samples Nold in the original question-and-answer sample set are read, and then the answer retention ratio η for this iteration is calculated by performing a floating-point division operation η=Nnew / Nold.

[0053] In this embodiment of the invention, by accurately screening high-confidence samples and calculating the retention ratio, the training signal is purified and the sample utilization rate is quantitatively fed back, providing support for strategy optimization and boundary adjustment, reducing reward noise interference and ensuring training efficiency.

[0054] In the fintech field, for the samples related to wealth management product recommendations, the samples retained after screening are all high-confidence samples that are "highly compliant and consistent with the reference scores". Updating the question-and-answer sample set can drive the model to optimize and generate recommendation schemes that meet regulatory requirements. The proportion of answers retained can reflect the utilization rate of compliant samples. If the proportion is too low, the range can be relaxed in the future to ensure sufficient training data.

[0055] In the healthcare field, for medication guidance samples, the samples retained after screening are high-confidence samples that are "scientific and non-misleading in their medication advice". The updated question and answer sample set helps the model to generate safe medication guidance after optimization. The proportion of answers retained can reflect the utilization rate of effective medical samples, providing a basis for dynamically adjusting the interval and balancing the filtering effect and sample sufficiency.

[0056] S5. Optimize the question-answering model according to the answer retention ratio, and use the optimized question-answering model to generate multiple candidate answers to user questions in the target scenario.

[0057] In this embodiment of the invention, the user question in the target scenario refers to a natural language text query input by a real end user through an interface in a real application environment, which needs to be understood and answered by a question-answering model. It is the direct object for providing services after the model has completed training.

[0058] In this embodiment of the invention, optimizing the question-answering model based on the answer retention ratio includes: calculating the loss function value of a preset loss function based on the answer weights; optimizing the preset weights and bias parameters in the question-answering model through backpropagation based on the loss function value to obtain optimized weight parameters and optimized bias parameters; adjusting the boundary parameters in the question-answering model according to the answer retention ratio to obtain optimized boundary parameters; and replacing the corresponding parameters in the question-answering model with the optimized weight parameters, the optimized bias parameters, and the optimized boundary parameters to generate an optimized question-answering model.

[0059] In detail, it iterates through and updates each sample in the question-and-answer sample set. For each sample, it reads the assigned answer weight, the probability ratio of the model generating that answer, and the calculated advantage function value. Then, it calculates the loss component for each sample according to the modified PPO loss function formula, as follows:

[0060] in, This is the total loss function value. As the weight of the answer, The probability ratio for the model to generate this answer. The dominant function value, For sample index.

[0061] Next, the loss components of all samples are summed or averaged to obtain the total loss function value for the current batch. By embedding the dynamically calculated answer weights directly into the core calculation of the policy gradient loss as multipliers, the contribution of each sample to the gradient is strictly proportional to the credibility of its reward signal, thus achieving precise filtering at the gradient level.

[0062] Specifically, starting with the total loss function value, backpropagation is performed along the computation graph to calculate the gradient of the loss function with respect to all trainable parameters in the question-answering model (such as the weight matrices and bias vectors of each neural network layer). θL; Subsequently, the system calls the optimizer (such as Adam), calculates the parameter update amount Δθ based on the gradient value and the optimizer's internal state (such as momentum), and performs the parameter update operation: θnew=θold+Δθ, thereby obtaining a set of optimized weight parameters and bias parameters.

[0063] Next, the calculated answer retention rate η is read and compared with a preset expected utilization threshold (such as ηtarget). Based on the comparison result, the system dynamically adjusts the boundary parameter β used to determine the answer retention interval according to predetermined rules (this parameter controls the width of the trust interval relative to the mean reward score). For example, if η is below the lower limit, β is increased by a certain step to widen the interval; if η is too high and the model performance improvement stagnates, β is decreased to tighten the interval. The adjusted new β value is the optimized boundary parameter. By using the sample utilization rate (η) as a closed-loop feedback signal to automatically adjust the tightness of the filtering mechanism, an adaptive balance between filtering strength and training efficiency is achieved.

[0064] Furthermore, the optimized weight parameters and optimized bias parameters obtained through backpropagation, along with the dynamically adjusted optimized boundary parameters, are used to replace the corresponding original parameters in the model in batches, according to the parameter storage structure of the question-answering model. This updates the model's policy distribution, completing one round of iterative optimization and generating an optimized question-answering model. In addition, multiple rounds of iterative optimization of the question-answering model can be performed.

[0065] In this embodiment of the invention, the multiple candidate answers refer to two or more different text answers independently generated by the optimized question-answering model during the inference phase, using nondeterministic decoding strategies such as random sampling, for the same user input question.

[0066] In this embodiment of the invention, as shown in Figure 4, the step of generating multiple candidate answers to a user question in a target scenario using an optimized question-answering model includes: S41, calling a multi-round generation function in the optimized question-answering model and setting the generation parameters of the multi-round generation function; S42, within a preset number of generation attempts, independently generating multiple rounds of user questions in the target scenario based on the multi-round generation function and the generation parameters to obtain multiple answer sequences; S43, standardizing the format of the answer sequences to obtain multiple candidate answers.

[0067] In detail, the multi-round generation function in the optimized question-answering model (obtained after training convergence) is called, and the generation parameters of the multi-round generation function are set, such as decoding temperature greater than 0 (e.g., 0.7, to avoid the generation result being single and fixed), sampling strategy using top-p (kernel sampling, e.g., p=0.9) or top-k (e.g., k=50) sampling, and the preset number of generation times is 2-6 (to control the generation delay within the acceptable range of business).

[0068] Next, within the preset number of generation iterations, the user question text of the target scenario is converted into an input format recognizable by the model and then input into the optimized question-answering model. The model independently analyzes and generates user questions based on probability distribution according to the multi-round generation function and generation parameters. Each generation outputs a differentiated answer sequence, ultimately resulting in 2-6 different answer sequences. The multiple answer sequences are then formatted to remove redundant markers that may exist during the generation process, unify the text encoding format (consistent with the question-answering sample format during the training phase), and verify the integrity of the answer sequences (removing sequences with abnormal length or semantic incompleteness), resulting in multiple standardized candidate answers.

[0069] In this embodiment of the invention, by collaboratively optimizing model parameters and boundary parameters, the model's generation capability and adaptability are improved, while generating diverse candidate answers to provide a basis for subsequent screening.

[0070] In the fintech field, an optimized robo-advisor dialogue model, when receiving a user's question "Should I invest in A-shares now?", will generate 3-5 responses with different focuses (such as analyzing the macroeconomic situation, pointing out short-term anomalies, and suggesting regular investment).

[0071] In the healthcare field, the optimized online triage model generates multiple responses with varying levels of caution when faced with the question "What should I do if I have a headache and fever?" (ranging from suggesting rest to explicitly listing warning symptoms and urging medical attention). This multi-candidate generation mechanism, combined with subsequent rapid reward assessment and trust interval filtering, ensures that the final answer presented to the user is the optimal solution after screening, greatly improving the reliability of AI services and user trust.

[0072] S6. Calculate the candidate reward score of the candidate answer, and retain the candidate answer corresponding to the candidate reward score in the answer retention interval to obtain the retained answer set.

[0073] In this embodiment of the invention, the candidate reward score refers to the scalar quality score calculated by the reward model during the online inference phase, after each candidate answer generated by the optimized question-answering model for the user's question is input along with the original question into the reward model. The reserved answer set refers to the candidate answers whose scores fall within the answer retention interval established (or dynamically maintained) during the training phase, after calculating the reward scores for all candidate answers during the inference phase.

[0074] In detail, the user's question and each candidate answer are concatenated according to a predefined format to form multiple temporary question-answer samples. These samples are then input into the same reward model as in the training phase, and forward inference is performed to obtain the candidate reward score rcand corresponding to each candidate answer. The currently maintained answer retention interval [Rlow, Rhigh] is read, and interval judgment is performed on each rcand: if Rlow≤rcand≤Rhigh, then the candidate answer and its score are added to a temporary list, and this list is encapsulated into a data structure, which is the retained answer set.

[0075] In this embodiment of the invention, lightweight quality verification and screening during the application stage ensure that the output answers meet high confidence standards, reduce the probability of extremely unreliable answers, and improve the security of the application scenario and user trust.

[0076] In the fintech field, when an optimized model generates marketing pitches for highly anomalous investment products, even if the model itself is aligned, input triggers or randomness may still produce overly aggressive or anomalous statements. Online reward filtering compares the reward scores of these statements with a conservative and compliant trust range. If a score is found to be too high (potentially indicating excessive appeal) or too low, it is filtered out directly, retaining only statements with scores falling within the "safe and compliant" range for subsequent selection, effectively avoiding regulatory anomalies.

[0077] In the healthcare field, online filtering can block candidate answers with abnormally high or low reward scores for health consultations, ensuring that the answers delivered to users are within a reasonable range of "professional and prudent" under the reward model evaluation system. This acts as an automatic gatekeeper to prevent AI medical advice from "going too far."

[0078] S7. Semantically fuse the answers in the reserved answer set to generate the final answer to the user's question.

[0079] In this embodiment of the invention, the final answer to the question refers to the text generated after semantic fusion, redundancy removal, and logical optimization of the high-confidence candidate answers in the retained answer set. It not only possesses the quality characteristics corresponding to the high-confidence reward, but also meets the business constraints of the target scenario (such as length, standardization, and diversity), and is the final response fed back to the user.

[0080] In this embodiment of the invention, the step of semantically fusing the answers in the reserved answer set to generate the final answer to the user's question includes: calculating the similarity of the questions and answers in the reserved answer set; filtering the questions and answers based on the similarity to obtain filtered answers; splitting the two questions and answers with the highest candidate reward scores in the filtered answers into language units; and performing grammatical coherence processing on the language units to obtain the final answer to the user's question.

[0081] In detail, the cosine similarity algorithm is used to calculate the semantic similarity between all pairs of candidate answers in the retained answer set (calculated based on sentence vectors of a pre-trained language model). A similarity threshold is set (e.g., 0.8). If the similarity between two answers is higher than the threshold, they are considered redundant. The answer with the higher candidate reward score is retained, and duplicate or highly similar answers are removed to obtain the filtered answer set.

[0082] Specifically, from the set of selected answers, the two answers with the highest candidate reward scores (referred to as answer A and answer B) are chosen. Natural language processing tools (such as word segmenters and dependency parsers) are then used to break down these two answer texts into a more granular sequence of language units. Language units can be phrases, clauses, or segments divided into semantic blocks. The goal is to obtain two semantically aligned or comparable sequence of units, such as UA=[uA1,uA2,...] and UB=[uB1,uB2,...].

[0083] Next, by calculating the semantic similarity or edit distance between units, alignment points that can be merged or replaced are found. According to the fusion rules, for example, the units of answer A with higher reward scores are retained as the backbone, and unique and high-quality units from answer B (such as more accurate data or more comprehensive supplementary explanations) are replaced or inserted into appropriate positions in the backbone. A lightweight language model or rule-based component is invoked to reorder the fused unit sequence, add conjunctions, adjust subject-verb agreement, and perform other grammatical coherence repairs, outputting a fluent, complete, and coherent text as the final answer to the question.

[0084] In this embodiment of the invention, multiple high-confidence candidate answers are integrated into a single high-quality final answer through semantic redundancy filtering, structured decomposition, and standardized fusion, which not only ensures the reliability and comprehensiveness of the answer but also meets the requirements of scenario-based expression.

[0085] In the fintech field, for users' questions about credit card bills, the high-scoring answers in the answer set may contain different interpretations of the bill. After semantic fusion, a logically coherent, comprehensive, and financially compliant final answer can be generated, clearly answering users' questions and highlighting relevant precautions.

[0086] In the healthcare field, for users seeking advice on chronic disease management, retaining high-scoring answers from the answer set may cover suggestions from different dimensions such as diet, exercise, and medication. After integration, a scientifically rigorous and logically clear final guidance plan can be generated to help users systematically manage their health status.

[0087] As can be seen, the above solution solves the problems of policy drift or overfitting caused by noise in the reward signal and susceptibility to low-confidence sample interference in traditional reinforcement learning alignment techniques. Furthermore, through dynamic confidence evaluation and adaptive interval adjustment, it achieves a dynamic balance between accurate purification of training signals and sample utilization, significantly improving the stability and convergence efficiency of model training. At the same time, in the application stage, the synergy of multi-candidate generation, high-confidence screening, and semantic fusion effectively avoids the information omission or unreliability problems that may exist in a single answer, ensuring that the final output answer not only conforms to human preferences and scenario business norms, but also has comprehensive information, logical coherence, and standardized expression, thus improving the accuracy of question answer generation.

[0088] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0089] In one embodiment, a question-answer generation device is provided, which corresponds one-to-one with the question-answer generation method in the above embodiments. As shown in FIG5, the question-answer generation device 100 includes a question-answer acquisition module 101, a question-answer sample set generation module 102, a confidence curve fitting module 103, an answer retention ratio calculation module 104, a question-answer model optimization module 105, a candidate reward score calculation module 106, and a question-answer fusion module 107. The functional modules are described in detail below: Question-Answer Acquisition Module 101, used to acquire multiple question-answer samples generated by a preset question-answering model for a target question within a preset question set, and to concatenate the target question with the corresponding multiple question-answer samples to form a question-answer sample; Question-Answer Sample Set Generation Module 102, used to aggregate the question-answer samples generated for each target question within the preset question set into a question-answer sample set, and to calculate the reward model score and reference model baseline score for each question-answer sample within the question-answer sample set; Confidence Curve Fitting Module 103, used to fit the reward model score and the reference model baseline score into a confidence curve, and to divide the answer retention interval according to the confidence curve; Answer Retention Ratio Calculation Module 104, The system is used to: filter the question and answer samples in the question and answer sample set according to the answer retention interval, form an updated question and answer sample set, and calculate the answer retention ratio in the updated question and answer sample set; the question and answer model optimization module 105 is used to optimize the question and answer model according to the answer retention ratio, and use the optimized question and answer model to generate multiple candidate answers to the user question in the target scenario; the candidate reward score calculation module 106 is used to calculate the candidate reward score of the candidate answer, and retain the candidate answer corresponding to the candidate reward score in the answer retention interval, to obtain a retained answer set; and the question and answer fusion module 107 is used to perform semantic fusion on the answers in the retained answer set to generate the final question and answer of the user question.

[0090] In one embodiment, the question-and-answer sample set generation module 102, when performing the calculation of the reward model score and the reference model baseline score for each question-and-answer sample in the question-and-answer sample set, is configured to: encode the question-and-answer sample into a first question-and-answer sample vector; perform a pooling operation on the question-and-answer sample vector to obtain a second question-and-answer sample vector; and map the second question-and-answer sample vector to the reward model score and the reference model baseline score using a preset reward model and a preset reference model, respectively.

[0091] In one embodiment, the confidence curve fitting module 103, when performing the process of fitting the reward model score and the reference model baseline score into a confidence curve, is configured to: for each question-answer sample, calculate the confidence coefficients of multiple question answers in the question-answer sample based on the reward model score and the reference model baseline score; divide the reward model score into score intervals according to the score distribution of the reward model score using a sliding window; calculate the average confidence coefficient corresponding to the confidence coefficient in each interval of the score interval; and plot a confidence curve with the score interval as the horizontal axis and the average confidence coefficient as the vertical axis.

[0092] In one embodiment, the answer retention ratio calculation module 104, when performing the filtering of question answers in the question-and-answer sample set according to the answer retention interval to form an updated question-and-answer sample set, is configured to: traverse the reward model scores corresponding to the question answers in the question-and-answer sample set, determine whether the reward model scores are within the answer retention interval; if they are within the answer retention interval, retain the question answers corresponding to the reward model scores and assign a preset answer weight to the question answers; if they are not within the answer retention interval, do not retain the question answers corresponding to the reward model scores; merge the retained question answers with the corresponding questions in the question-and-answer sample set to generate an updated question-and-answer sample set.

[0093] In one embodiment, the question-answering model optimization module 105, when optimizing the question-answering model according to the answer retention ratio, is configured to: calculate the loss function value of a preset loss function based on the answer weights; optimize the preset weights and bias parameters in the question-answering model through backpropagation based on the loss function value to obtain optimized weight parameters and optimized bias parameters; adjust the boundary parameters in the question-answering model according to the answer retention ratio to obtain optimized boundary parameters; and replace the corresponding parameters in the question-answering model with the optimized weight parameters, the optimized bias parameters, and the optimized boundary parameters to generate an optimized question-answering model.

[0094] In one embodiment, the question-answering model optimization module 105, when generating multiple candidate answers to a user question in a target scenario using the optimized question-answering model, is configured to: call a multi-round generation function in the optimized question-answering model and set the generation parameters of the multi-round generation function; within a preset number of generation attempts, independently generate multiple rounds of user questions in the target scenario based on the multi-round generation function and the generation parameters to obtain multiple answer sequences; and perform format standardization processing on the answer sequences to obtain multiple candidate answers.

[0095] In one embodiment, the question-answer fusion module 107, when performing semantic fusion of the answers in the reserved answer set to generate the final question answer for the user question, is configured to: calculate the similarity of the question answers in the reserved answer set; filter the question answers based on the similarity to obtain filtered answers; split the two question answers with the highest candidate reward scores in the filtered answers into language units; and perform grammatical coherence processing on the language units to obtain the final question answer for the user question.

[0096] This invention provides a question-answer generation device that solves the problems of policy drift or overfitting caused by noise in reward signals and susceptibility to low-confidence sample interference in traditional reinforcement learning alignment techniques. Furthermore, through dynamic confidence evaluation and adaptive interval adjustment, it achieves a dynamic balance between accurate purification of training signals and sample utilization, significantly improving the stability and convergence efficiency of model training. Simultaneously, in the application stage, the synergy of multi-candidate generation, high-confidence screening, and semantic fusion effectively avoids the information omissions or unreliability issues that may arise from a single answer, ensuring that the final output answer conforms to human preferences and scenario-specific business norms, while also possessing comprehensive information, logical coherence, and standardized expression, thus improving the accuracy of question-answer generation.

[0097] Specific limitations regarding the question-answer generation device can be found in the limitations of the question-answer generation method described above, and will not be repeated here. Each module in the aforementioned question-answer generation device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in the computer device in hardware form, or stored in the memory of the computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0098] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram is shown in Figure 6. The computer device includes a processor, memory, a network interface, and a database connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile and / or volatile storage media and internal memory. The non-volatile storage media stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The network interface of the computer device is used to communicate with external clients via a network connection. When the computer program is executed by the processor, it implements the functions or steps of a question-answer generation method on the server side.

[0099] In one embodiment, a computer device is provided, which can be a client, and its internal structure can be as shown in Figure 7. The computer device includes a processor, memory, network interface, display screen, and input device connected via a system bus. The processor of the computer device provides computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external server via a network connection. When the computer program is executed by the processor, it implements the functions or steps of a question-answer generation method on the client side.

[0100] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it performs the following steps: collecting multiple question answers generated by a preset question-answering model for a target question within a preset question set, and concatenating the target question with the corresponding multiple question answers to form a question-answer sample; aggregating the question-answer samples generated for each target question within the preset question set into a question-answer sample set, and calculating the reward model score and reference model baseline score for each question-answer sample in the question-answer sample set; fitting the reward model score and the reference model baseline score to a confidence level. The confidence curve is used to divide the answer retention interval; the question and answer samples in the question and answer sample set are filtered according to the answer retention interval to form an updated question and answer sample set, and the answer retention ratio in the updated question and answer sample set is calculated; the question and answer model is optimized according to the answer retention ratio, and multiple candidate answers to the user's question in the target scenario are generated using the optimized question and answer model; the candidate reward score of the candidate answer is calculated, and the candidate answer corresponding to the candidate reward score in the answer retention interval is retained to obtain the retained answer set; the answers in the retained answer set are semantically fused to generate the final question and answer of the user's question.

[0101] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When executed by a processor, the computer program performs the following steps: collecting multiple question-answering samples generated by a preset question-answering model for a target question within a preset question set, and concatenating the target question with the corresponding multiple question-answering samples to form a question-answering sample; aggregating the question-answering samples generated for each target question within the preset question set into a question-answering sample set, and calculating the reward model score and reference model baseline score for each question-answering sample within the question-answering sample set; fitting the reward model score and the reference model baseline score into a confidence curve, and dividing the answer retention interval according to the confidence curve; filtering the question answers in the question-answering sample set according to the answer retention interval to form an updated question-answering sample set, and calculating the answer retention ratio in the updated question-answering sample set; optimizing the question-answering model according to the answer retention ratio, and using the optimized question-answering model to generate multiple candidate answers to a user question in a target scenario; calculating the candidate reward scores of the candidate answers, and retaining the candidate answers corresponding to the candidate reward scores in the answer retention interval to obtain a retained answer set; and performing semantic fusion on the answers in the retained answer set to generate the final question answer for the user question.

[0102] It should be noted that the functions or steps that can be implemented by the computer-readable storage medium or computer device described above can be referred to the relevant descriptions on the server side and client side in the foregoing method embodiments. To avoid repetition, they will not be described one by one here.

[0103] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0104] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0105] It should be noted that if any software tools or components not belonging to our company appear in the embodiments of this application, they are merely for illustrative purposes and do not represent actual use.

[0106] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A method for generating answers to questions, characterized in that, include: Collect multiple question answers generated by a preset question-answering model for a target question within a preset question set, and concatenate the target question with the corresponding multiple question answers to form a question-answering sample; The question-and-answer samples generated for each target question in the preset question set are collected into a question-and-answer sample set. The reward model score and the reference model baseline score of each question-and-answer sample in the question-and-answer sample set are calculated one by one. The reward model score and the reference model baseline score are fitted into a confidence curve, and the answer retention interval is divided according to the confidence curve. Based on the answer retention interval, the question and answer samples in the question and answer sample set are filtered to form an updated question and answer sample set, and the answer retention ratio in the updated question and answer sample set is calculated; The question-answering model is optimized based on the answer retention ratio, and multiple candidate answers to user questions in the target scenario are generated using the optimized question-answering model; the candidate reward score of the candidate answer is calculated, and the candidate answer corresponding to the candidate reward score in the answer retention interval is retained to obtain the retained answer set; The answers in the reserved answer set are semantically fused to generate the final answer to the user's question.

2. The question-answer generation method as described in claim 1, characterized in that, The step of calculating the reward model score and reference model baseline score for each question and answer sample in the question and answer sample set includes: text encoding the question and answer sample to obtain a first question and answer sample vector; performing a pooling operation on the question and answer sample vector to obtain a second question and answer sample vector; and mapping the second question and answer sample vector to the reward model score and reference model baseline score using a preset reward model and a preset reference model, respectively.

3. The question-answer generation method as described in claim 1, characterized in that, The step of fitting the reward model score and the reference model baseline score into a confidence curve includes: for each question-answer sample, calculating the confidence coefficients of multiple question answers in the question-answer sample based on the reward model score and the reference model baseline score; dividing the reward model score into score intervals according to the score distribution of the reward model score using a sliding window; calculating the average confidence coefficient corresponding to the confidence coefficient in each interval of the score interval; and plotting a confidence curve with the score interval as the horizontal axis and the average confidence coefficient as the vertical axis.

4. The question-answer generation method as described in claim 1, characterized in that, The step of filtering the question-answer samples in the question-answer sample set according to the answer retention interval to form an updated question-answer sample set includes: traversing the reward model scores corresponding to the question-answers in the question-answer sample set, and determining whether the reward model scores are within the answer retention interval; if they are within the answer retention interval, retaining the question-answers corresponding to the reward model scores and assigning preset answer weights to the question-answers; if they are not within the answer retention interval, not retaining the question-answers corresponding to the reward model scores; and merging the retained question-answers with the corresponding questions in the question-answer sample set to generate an updated question-answer sample set.

5. The question-answer generation method as described in claim 1, characterized in that, The step of optimizing the question-answering model based on the answer retention ratio includes: calculating the loss function value of a preset loss function based on the answer weights; optimizing the preset weights and bias parameters in the question-answering model through backpropagation based on the loss function value to obtain optimized weight parameters and optimized bias parameters; adjusting the boundary parameters in the question-answering model according to the answer retention ratio to obtain optimized boundary parameters; and replacing the corresponding parameters in the question-answering model with the optimized weight parameters, the optimized bias parameters, and the optimized boundary parameters to generate an optimized question-answering model.

6. The question-answer generation method as described in claim 1, characterized in that, The step of generating multiple candidate answers to user questions in a target scenario using an optimized question-answering model includes: calling a multi-round generation function in the optimized question-answering model and setting the generation parameters of the multi-round generation function; generating multiple independent rounds of user questions in the target scenario based on the multi-round generation function and the generation parameters within a preset number of generation rounds to obtain multiple answer sequences; and performing format standardization processing on the answer sequences to obtain multiple candidate answers.

7. The question-answer generation method as described in claim 1, characterized in that, The step of semantically fusing the answers in the reserved answer set to generate the final answer to the user's question includes: calculating the similarity of the questions and answers in the reserved answer set; filtering the questions and answers based on the similarity to obtain filtered answers; splitting the two questions and answers with the highest candidate reward scores in the filtered answers into language units; and performing grammatical coherence processing on the language units to obtain the final answer to the user's question.

8. A question-answer generation device, characterized in that, include: The question-answer collection module is used to collect multiple question answers generated by a preset question-answering model for a target question within a preset question set, and to concatenate the target question with the corresponding multiple question answers to form a question-answer sample; The question-and-answer sample set generation module is used to collect the question-and-answer samples generated for each target question in the preset question set into a question-and-answer sample set, and calculate the reward model score and the reference model baseline score for each question-and-answer sample in the question-and-answer sample set; the confidence curve fitting module is used to fit the reward model score and the reference model baseline score into a confidence curve, and divide the answer retention interval according to the confidence curve; The answer retention ratio calculation module is used to filter the question and answer samples in the question and answer sample set according to the answer retention interval, form an updated question and answer sample set, and calculate the answer retention ratio in the updated question and answer sample set; The question-answering model optimization module is used to optimize the question-answering model according to the answer retention ratio, and to generate multiple candidate answers to user questions in the target scenario using the optimized question-answering model; The candidate reward score calculation module is used to calculate the candidate reward score of the candidate answer and retain the candidate answer corresponding to the candidate reward score in the answer retention interval to obtain the retained answer set; The question-answer fusion module is used to semantically fuse the answers in the reserved answer set to generate the final question answer for the user's question.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the question answer generation method as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the question answer generation method as described in any one of claims 1 to 7.

Images