Model training methods, question answering methods, devices, electronic devices and storage media

By optimizing the text filtering model through multiple rounds of training and answer evaluation, the problem of insufficient accuracy in answer generation in existing technologies is solved, and the accuracy of answers is improved in complex problem scenarios.

CN123115367APending Publication Date: 2026-11-17TENCENT TECHNOLOGY (SHENZHEN) CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202611631472.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-10-21
Publication Date
2026-11-17

AI Technical Summary

Technical Problem

Existing technologies have low accuracy in generating answers for complex problem scenarios. The optimization direction of text filtering models deviates from the needs of answer generation. They lack overall evaluation and adaptability across multiple documents and lack a direct feedback mechanism for answer quality.

Method used

The target text set is selected and answers are generated through multiple rounds of training. The training loss is determined based on the evaluation results of the answers, and the text selection model is optimized to generate more accurate answers in complex problem scenarios. Differentiated training methods are used to optimize text sets of different quality.

Benefits of technology

It improves the accuracy of answer generation in complex problem scenarios, and the text filtering model is more in line with the needs of practical applications, and can filter out reference texts that are more conducive to generating accurate answers.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123115367A_ABST
    Figure CN123115367A_ABST
Patent Text Reader

Abstract

This disclosure provides a model training method, a question-answering method, an apparatus, an electronic device, and a storage medium, relating to the field of artificial intelligence technology. The method includes: acquiring multiple training samples, each training sample including question text and multiple candidate texts; filtering a target text set from the candidate texts using a model to be trained; inputting the target text set and the question text into a question-answering model to generate an answer to the question text; evaluating the answer based on a preset evaluation metric, determining a training loss based on the evaluation result, and updating the model parameters of the model to be trained based on the training loss, until the training termination condition is met, thus obtaining a text filtering model. This disclosure optimizes the text filtering model based on the evaluation result of the answer, making the filtered text more conducive to generating accurate answers and improving the accuracy of answer generation.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This disclosure relates to the field of artificial intelligence technology, and more specifically, to a model training method, a question-answering method, an apparatus, an electronic device, and a storage medium. Background Technology

[0002] In applications such as intelligent question answering and knowledge retrieval, it is often necessary to retrieve multiple candidate texts related to the user's question from a large amount of text data, and further filter these candidate texts to generate corresponding answers based on the filtered texts.

[0003] However, when generating answers based on text selected using existing methods, the accuracy of the generated answers is still not ideal for some complex questions. Therefore, further research is needed to improve the selection of reference texts for answer generation in order to enhance the accuracy of the generated answers. Summary of the Invention

[0004] This disclosure provides a model training method, question answering method, apparatus, electronic device, computer-readable storage medium, and computer program product, which can solve the problem of low accuracy in generating answers to questions based on existing text filtering schemes in some complex question scenarios. The technical solution provided by this disclosure is as follows: According to one aspect of the present disclosure, a model training method is provided, the method comprising: Multiple training samples are obtained. Each training sample includes a question text and a candidate text set. The candidate text set includes multiple candidate texts, which are candidate reference texts for generating the answer to the question text. Using the multiple training samples, the model to be trained undergoes multiple rounds of first training operations until a preset first training termination condition is met. The model to be trained that meets the first training termination condition is then used as the text filtering model. Each round of first training operations includes: For each question text in a training sample, the model to be trained selects multiple target texts from the candidate text set based on the first relevance between each candidate text in the training sample and the question text, thus obtaining a target text set. For each training sample, the target text set and the question text are input into the question answering model, the question answering model generates the answer to the question text, the answer is evaluated based on a preset first evaluation metric, and the training loss of the training sample is determined based on the evaluation result of the answer. Based on the training loss of each training sample, the model parameters of the model to be trained are updated to obtain the model to be trained for the next round of training.

[0005] According to another aspect of the embodiments of this disclosure, a question-answering method is provided, the method comprising: Obtain the question text and a candidate text set, wherein the candidate text set includes multiple candidate texts, and the candidate texts are candidate reference texts for generating the answer to the question text; The question text and the candidate text set are input into the text filtering model. The text filtering model filters out multiple candidate texts from the candidate text set based on the relevance of each candidate text in the candidate text set to the question text, thereby obtaining the target text set. The text filtering model is trained using any of the model training methods described above. The target text set and the question text are input into the question-answering model, and the question-answering model generates the answer to the question text.

[0006] According to another aspect of the present disclosure, a model training apparatus is provided, the apparatus comprising: The acquisition module is used to acquire multiple training samples. Each training sample includes a question text and a candidate text set. The candidate text set includes multiple candidate texts, which are candidate reference texts for generating the answer to the question text. The training module is used to perform multiple rounds of first training operations on the model to be trained using the multiple training samples until a preset first training termination condition is met, and the model to be trained that meets the first training termination condition is used as the text filtering model. The training module includes: The filtering unit is used to filter multiple target texts from the candidate text set based on the first relevance between each candidate text in the training sample and the question text in each training sample, thereby obtaining a target text set. An evaluation unit is used to input the target text set and the question text into the question answering model for each training sample, generate an answer to the question text through the question answering model, evaluate the answer based on a preset first evaluation index, and determine the training loss of the training sample based on the evaluation result of the answer. The update unit is used to update the model parameters of the model to be trained based on the training loss of each training sample, so as to obtain the model to be trained for the next round of training.

[0007] As an optional embodiment, for each question text in the training sample, the target text set corresponding to the question text is obtained by the filtering unit in the following way: The first relevance between each candidate text in the candidate text set and the question text is determined using the model to be trained. Based on the first relevance between each candidate text and the question text, the sampling probability of each candidate text is obtained. Based on the preset number of single samplings and the sampling probability of each candidate text, the candidate text set is sampled N times, and the target text obtained from each sampling is taken as a target text set, where N is an integer greater than or equal to 1.

[0008] As an optional embodiment, N is an integer greater than or equal to 2; For each training sample, the answer to the question text, the evaluation result of the answer, and the training loss in the training sample are obtained by the evaluation unit in the following way: For each target text set corresponding to the training samples, the target text set and the question text are input into the question answering model. The question answering model generates an answer based on the question text of the target text set. The answer is evaluated based on a first evaluation index to obtain the evaluation result of the answer. The training loss of the training samples is determined based on the evaluation results of the answers to the question texts corresponding to each target text set.

[0009] As an optional embodiment, the training samples also include a standard text set of the question text, the standard text set including multiple standard texts, each of which is a standard reference text for generating the answer to the question text; For each training sample, when determining the training loss of the training sample based on the evaluation result of the answer, the evaluation unit is specifically used for: The first loss is determined based on the evaluation results of the answers corresponding to the training samples; The second loss is determined based on the difference between the target text set and the standard text set of the training samples; The training loss of the training samples is determined based on the first loss and the second loss.

[0010] As an optional embodiment, the number of target text sets corresponding to each training sample is multiple, and for each training sample, the evaluation unit is further used to: Based on a preset second evaluation index, the quality of each target text set corresponding to the training samples is evaluated to obtain a score value for each target text set. For each target text set, if the score of the target text set is less than a first threshold, the target text set is designated as the first text set; if the score of the target text set is greater than or equal to the first threshold, the target text set is designated as the second text set. For each of the second text sets, the advantage value of the second text set is determined based on the score value of the second text set and the benchmark score value, and the training loss of the second text set is determined based on the advantage value. For each training sample, when determining the second loss based on the difference between the target text set and the standard text set of the training sample, the evaluation unit is specifically used for: For each first text set corresponding to the training samples, the training loss corresponding to the first text set is determined based on the difference between the first text set and the standard text set. The second loss of the training sample is determined based on the training loss of each first text set and the training loss of each second text set corresponding to the training sample.

[0011] As an optional embodiment, when determining the second loss based on the difference between the target text set and the standard text set of the training samples, the evaluation unit is specifically used for: The question text, the candidate text set, and the standard text set are input into the model to be trained. The model to be trained then determines the second relevance between each candidate text in the candidate text set and the question text by referring to the standard text set. Determine the difference between the first relevance and the second relevance corresponding to each target text in the target text set, wherein the difference represents the difference between the target text set and the standard text set; The second loss of the training samples is determined based on the differences between the target text sets corresponding to the training samples.

[0012] As an optional embodiment, for each training sample, when the evaluation unit evaluates the quality of each target text set corresponding to the training sample based on a preset second evaluation index to obtain a score value for each target text set, it is specifically used for: Obtain the first prompt word, which is used to prompt the large language model to evaluate the target text set input to the large language model based on the second evaluation metric; The first prompt word, the question text, and the target text set are input into a large language model to obtain the evaluation result of the target text set output by the large language model; wherein, the second evaluation index includes at least one of the following: The extent to which all target texts in the target text set cover the information of the question text; The degree of information redundancy among the target texts in the target text set; The degree of matching between the relevance and contribution of each target text in the target text set, wherein the relevance is the relevance between the target text and the question text, and the contribution is the contribution of the target text set to the generation of the answer to the question text; The percentage of target texts in the target text set whose relevance to the problem text is lower than a preset value.

[0013] As an optional embodiment, the model to be trained is obtained in the following way: Obtain the teacher model trained using a supervised training method; Based on the multiple training samples and the teacher model, the student model undergoes multiple rounds of second training operations until a preset second training termination condition is met. The student model that meets the second training termination condition is then used as the model to be trained. Each round of second training operations includes: For each training sample, the third relevance between each candidate text in the candidate text set of the training sample and the question text is determined by the student model, and the fourth relevance between each candidate text in the candidate text set of the training sample and the question text is determined by the teacher model. For each training sample, the third loss of the training sample is determined based on the difference between the third and fourth relevance of each candidate text in the candidate text set of the training sample. Based on the third loss of each training sample, the model parameters of the student model are updated to obtain the student model for the next round of second training operation.

[0014] As an optional embodiment, the model training apparatus further includes a preprocessing module for: Obtain a preset second prompt word, which is used to prompt the large language model to output the answer to the question text and the text identifier of the candidate text referenced when generating the answer, based on the input question text and candidate text set of the large language model; The question text of the training samples and the candidate text set are input into the large language model. The large language model performs multiple answer generation operations to obtain multiple answers output by the large language model and the text identifier of the candidate text referenced in the generation of each answer. The text identifiers of the candidate texts referenced when generating each answer are statistically analyzed, and a preset number of candidate texts are selected from the referenced candidate texts in descending order of reference frequency. Based on the predetermined number of candidate texts, the standard text set is obtained.

[0015] As an optional embodiment, when the preprocessing module obtains the standard text set based on the determined preset number of candidate texts, it is specifically used for: The intent analysis of the question text is performed using a large language model, and a text evaluation strategy for the question text is generated based on the analysis results. Based on the preset number of candidate texts, multiple subsets of candidate texts are obtained; Based on the text evaluation strategy, each candidate text subset is evaluated, and the standard text set is determined from the multiple candidate text subsets according to the evaluation results.

[0016] According to another aspect of the present disclosure, a question-answering device is provided, the device comprising: The acquisition module is used to acquire the question text and the candidate text set, wherein the candidate text set includes multiple candidate texts, and the candidate texts are candidate reference texts for generating the answer to the question text; The filtering module is used to input the question text and the candidate text set into the text filtering model, and the text filtering model filters out multiple candidate texts from the candidate text set based on the relevance of each candidate text in the candidate text set to the question text, thereby obtaining the target text set; the text filtering model is trained by any of the model training methods described above; The generation module is used to input the target text set and the question text into the question-answering model, and generate the answer to the question text through the question-answering model.

[0017] According to another aspect of the present disclosure, an electronic device is provided, the electronic device including a memory, a processor and a computer program stored in the memory, the processor executing the computer program to implement the above-described model training method and question answering method.

[0018] According to another aspect of the present disclosure, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the above-described model training method and question-answering method.

[0019] According to another aspect of the present disclosure, a computer program product is provided, including a computer program that, when executed by a processor, implements the above-described model training method and question answering method.

[0020] The beneficial effects of the technical solutions provided in this disclosure are: In this embodiment of the disclosure, during the training of the text filtering model, the selected target text set is used to generate question answers, and the generated answers are evaluated based on preset evaluation metrics. The training loss is determined based on the evaluation results of the answers to iteratively train the text filtering model. This achieves optimization of the candidate text filtering process based on the final answer generation effect. Compared with determining the model optimization direction by evaluating the correlation between the text in the target text set and the question text, this method can match the optimization direction of the text filtering model with the final answer generation effect. In practical applications, the text filtering model trained by this embodiment can filter out reference texts that are more conducive to generating accurate answers from multiple candidate texts, which is more in line with the application needs of actual question-and-answer scenarios, thereby improving the accuracy of answer generation in complex question scenarios. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments of this disclosure, the accompanying drawings used in the description of the embodiments of this disclosure will be briefly introduced below.

[0022] Figure 1 A schematic diagram of the system architecture to which the question-answering method provided in the embodiments of this disclosure applies; Figure 2 This is a schematic diagram of a question-and-answer process provided in an embodiment of the present disclosure; Figure 3 A schematic flowchart of a model training method provided in an embodiment of this disclosure; Figure 4 A flowchart illustrating a first training operation provided in an embodiment of this disclosure; Figure 5 A flowchart illustrating the process of filtering text one by one, provided for an embodiment of this disclosure; Figure 6 This is a schematic diagram illustrating how multiple target text sets are obtained through multiple sampling, as provided in an embodiment of this disclosure. Figure 7 A schematic diagram of a process for determining the training loss of training samples provided in an embodiment of this disclosure; Figure 8 A schematic diagram illustrating a method for determining a first loss and a second loss for training samples, provided in an embodiment of this disclosure; Figure 9 A schematic diagram illustrating a method for determining a second loss for training samples, provided as an embodiment of this disclosure; Figure 10 A schematic diagram illustrating the determination of a second loss according to an embodiment of this disclosure; Figure 11 A schematic diagram of a process for training a student model provided in an embodiment of this disclosure; Figure 12A schematic flowchart of a model training method provided in an embodiment of this disclosure; Figure 13 A flowchart illustrating a question-and-answer method provided in an embodiment of this disclosure; Figure 14 This is a schematic diagram of the structure of a model training device provided in an embodiment of the present disclosure; Figure 15 This is a schematic diagram of the structure of a question-and-answer device provided in an embodiment of the present disclosure; Figure 16 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure. Detailed Implementation

[0023] The embodiments of this disclosure are described below with reference to the accompanying drawings. It should be understood that the embodiments described below with reference to the accompanying drawings are exemplary descriptions for explaining the technical solutions of the embodiments of this disclosure, and do not constitute a limitation on the technical solutions of the embodiments of this disclosure.

[0024] Those skilled in the art will understand that, unless otherwise stated, the singular forms “a,” “an,” “the,” and “the” used herein may also include the plural forms. It should be further understood that the terms “comprising” and “including” as used in embodiments of this disclosure mean that the corresponding feature can be implemented as the presented feature, information, data, step, operation, element, and / or component, but do not exclude implementation as other features, information, data, steps, operations, elements, components, and / or combinations thereof supported by the art.

[0025] It should be understood that when we say that a component is "connected" or "coupled" to another component, the component can be directly connected or coupled to the other component, or it can mean that the component and the other component establish a connection relationship through an intermediate component. Furthermore, the term "connected" or "coupled" as used herein can include wireless connections or wireless coupling. The term "and / or" as used herein indicates at least one of the items defined by the term; for example, "A and / or B" or "A, B" indicates implementation as "A", or implementation as "B", or implementation as "A and B".

[0026] In this disclosure, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0027] To make the objectives, technical solutions, and advantages of this disclosure clearer, the embodiments of this disclosure will be described in further detail below with reference to the accompanying drawings. Where there is no conflict, the features in the embodiments of this disclosure can be combined with each other.

[0028] Currently, in agent-based problem-solving scenarios, it is often necessary to retrieve and filter a subset of documents from a large number of documents as reference information for subsequent large language models to generate answers. Related technologies mainly include the following categories.

[0029] 1. Relevance-based document ranking method Traditional document retrieval and ranking methods include Best Matching 25 (BM25), Learning to Rank (L2R), and semantic retrieval. These methods typically score and rank candidate documents based on the relevance between the query text and the candidate documents, selecting the top-ranked documents. Related models can be trained on historical data such as user clicks and use metrics such as Normalized Discounted Cumulative Gain (NDCG) and Mean Reciprocal Rank (MRR) to evaluate ranking performance.

[0030] 2. Question Answering Method Based on Search Enhancement Retrieval-Augmented Generation (RAG) typically begins by retrieving documents based on the user's question. Then, it selects the top K documents from the search results based on metrics such as semantic relevance. These selected documents are then used as contextual information and input into a large language model, which combines the user's question and the selected documents to generate the answer. Thus, document retrieval and answer generation form a seamless, sequential processing flow.

[0031] 3. Model optimization methods based on reinforcement learning or preference learning.

[0032] Some reinforcement learning strategies can be used for policy optimization of large language models. They can optimize the output policy of the model based on information such as reward signals, preference data, or probability distributions provided by the teacher model.

[0033] 4. Document re-ranking method based on large language model evaluation This method uses a large language model to evaluate the retrieved candidate documents and re-ranks them based on the evaluation results. For example, it can evaluate the matching degree between each candidate document and the question, or compare the relative merits of different candidate documents for the current question, and then select the document for answer generation based on the evaluation results.

[0034] While the above solutions can achieve the retrieval, filtering, and answer generation of candidate documents, the following problems still exist in document filtering scenarios facing complex questions.

[0035] First, there is a certain difference between the optimization objectives of document screening and the objectives of answer generation. Existing retrieval or ranking methods mainly focus on the relevance between candidate documents and questions, but documents that are relevant to the question do not necessarily have the same effective impact on the generation of the final answer. Therefore, determining the document screening results solely based on relevance may cause the optimization direction of the screening model to deviate from the actual needs of answer generation.

[0036] Second, there is a lack of evaluation of the overall effectiveness of multiple selected documents. Some solutions mainly evaluate individual candidate documents independently or compare candidate documents pairwise, while the actual input to the large language model is a text set composed of multiple documents. Different documents may have complementary or overlapping information, and filtering based solely on the evaluation results of a single document is insufficient to fully reflect the overall effect of multiple documents working together to generate the answer.

[0037] Third, applying a uniform optimization method to screening results of different quality levels has limited adaptability. The required model optimization methods may differ between high-quality and low-quality document screening results. If the same reinforcement learning or distillation method is used uniformly, it may result in insufficient correction of low-quality results or limitations on further exploration of high-quality results.

[0038] Fourth, there is a lack of direct feedback mechanism between document screening results and the quality of the final answer. Existing document screening models typically derive their training signals from document relevance, ranking results, or evaluations of the documents themselves, without fully utilizing the quality of the answers generated after the screened documents actually participate in the answer generation process. Therefore, while the model may be able to select documents that are relatively relevant to the question, these documents may not effectively support the generation of accurate answers by the large language model, potentially affecting the accuracy of the generated answers in complex problem scenarios.

[0039] The model training method, question answering method, apparatus, electronic device, computer-readable storage medium, and computer program product disclosed herein aim to solve at least one of the above-mentioned technical problems in the prior art. In the training process of the text selection model, this disclosure uses the selected target text set to generate answers to question texts, and determines the training loss based on the evaluation results of the answers, thus linking the optimization direction of the text selection model with the actual answer generation effect. This enables the trained text selection model to select reference texts that are more conducive to generating accurate answers. By comprehensively evaluating the selected target text set and classifying the target text set according to the evaluation results, different training methods are adopted for target text sets of different qualities. For target text sets with higher evaluation results, a strategy optimization method based on the dominance value is adopted, enabling the model to further explore better text combinations based on existing selection results. For target text sets with lower evaluation results, the model undergoes strategy distillation based on the teacher signal corresponding to the standard text set to perform fine-grained correction of the text selection process. Therefore, appropriate optimization methods can be adopted according to the text selection results of different qualities, improving the text selection model's ability to select candidate text combinations, thereby improving the accuracy of answer generation in complex question scenarios.

[0040] The following description of several exemplary embodiments illustrates the technical solutions of this disclosure and the technical effects produced by these solutions. It should be noted that the following embodiments can be referenced, learned from, or combined with each other. Identical terms, similar features, and similar implementation steps in different embodiments will not be repeated.

[0041] The following describes a system architecture applicable to the model training method and question answering method provided in the embodiments of this disclosure. Figure 1 This is a schematic diagram of the system architecture for implementing the model training scheme and question answering method provided in the embodiments of this disclosure. The system includes a first server 10, a second server 20, a terminal 30, and a database 40. The first server 10 can communicate with the database 40 and the second server 20, and the second server 20 can communicate with the terminal 30 through a network.

[0042] Database 40 can be used to store training samples for the text filtering model. The training samples can include the question text and a set of candidate texts corresponding to the question text. The candidate text set includes multiple candidate texts, which can serve as reference texts for generating answers to the question text. For example, a question text can be pre-defined, and multiple text fragments related to the question can be retrieved from data sources such as databases and web pages to form a candidate text set. A training sample is then constructed based on the question text and the candidate text set. The reference text can be text containing at least one of the following: conceptual explanations, factual information, data, or analytical viewpoints related to the question text.

[0043] The first server 10 can serve as a model training server, used to train the model to be trained, resulting in a text filtering model. The first server 10 can obtain training samples from the database 40 and train the model based on these samples. During training, the model can filter multiple candidate texts from the candidate text set corresponding to the training samples to obtain a target text set. The training loss is determined based on the evaluation results of the answers generated from the target text set, and the model parameters of the model to be trained are updated according to the training loss. After the training termination condition is met, the trained text filtering model is obtained. The first server 10 can send the trained text filtering model to the second server 20, allowing the second server 20 to deploy the text filtering model. Sending the text filtering model can involve sending a model parameter file corresponding to the text filtering model. The second server 20 can load this model parameter file to obtain the trained model parameters and deploy the text filtering model based on these parameters.

[0044] The second server 20 can be an application server, which can be used to provide question-and-answer services for the target application. The target application can be, for example, a question-and-answer service application, such as a medical consultation application, an intelligent customer service application, or an educational Q&A application. The terminal 30 can have the client of the target application installed, or can access the target application through a browser, mini-program, or other means.

[0045] Users can enter their questions in the question-and-answer interface displayed on terminal 30 and trigger the corresponding question submission operation. Terminal 30 responds to the user's question submission operation, obtains the user-inputted question text, and sends a question-and-answer request carrying that question text to the second server 20 via the network. Upon receiving the question-and-answer request from terminal 30, the second server 20 performs text retrieval based on the question text, obtains multiple candidate texts related to the question text, and filters these candidate texts using a deployed text filtering model to obtain a target text set for generating the answer. The second server 20 can use the target text set as reference information for the question text, generate the corresponding answer text using the question-and-answer model, and return the generated answer text to terminal 30. After receiving the answer text, terminal 30 can display the answer text in the question-and-answer interface of the target application.

[0046] It should be noted that the system architecture shown in the figure is only one example of an embodiment of this disclosure. In other embodiments, the training process and question-answering process of the text filtering model may also be executed by the same server, and the database 40 may be set in the first server 10, the second server 20, or in other data storage devices that can communicate with the corresponding servers. This disclosure does not limit this.

[0047] In some scenarios, the text filtering model provided in this disclosure can be applied to the agent's question processing process. For example... Figure 2 As shown, Figure 2This is a schematic diagram of a question-and-answer process provided in an embodiment of this disclosure. An Agent can be deployed in the second server 20. This Agent can perform operations such as question decomposition, information retrieval, candidate text selection, and answer generation based on the user's question. Specifically, it can include: S1, User Questioning. The user can initiate a query to the Agent through the terminal 30, inputting the text of the question to be processed. S2, Question Decomposition. After receiving the user's question, the Agent can analyze the question and decompose it into one or more sub-queries according to the processing requirements. S3, Search Engine Retrieval. For one or more sub-queries, the Agent can call a search engine to retrieve information from sources such as the network, knowledge base, and document library, obtaining multiple candidate texts related to each sub-query. S4, Aggregation Selection. The Agent can aggregate the retrieval results corresponding to different sub-queries and select a portion of the candidate texts from the multiple candidate texts using the text filtering model trained according to this disclosure, obtaining a target text set. Since different candidate texts may provide different reference values ​​for answering the user's question, aggregating and selecting candidate texts can provide corresponding reference information for subsequent answer generation. S5, Answer Generation. The Agent can use the target text set as context information and, combined with the user's question, generate the corresponding answer text through a question-and-answer model. S6: The answer is returned to the user. The Agent can return the generated answer text to terminal 30 so that the corresponding answer can be displayed to the user through terminal 30. In practical applications, according to this embodiment, the Agent can utilize a trained text filtering model to select reference text from candidate texts that is more conducive to completing the question-and-answer task based on the current question. This adapts the Agent's information selection process to the subsequent answer generation process, improving the accuracy of the Agent in generating answers to complex questions.

[0048] The first server 10 and the second server 20 can be independent physical servers, or they can be server clusters, distributed systems, or cloud servers, etc., which are not limited in this disclosure. The first server 10 and the second server 20 can communicate through wired networks or wireless networks.

[0049] Terminal 30 can be an electronic device with data processing and communication functions, such as a smartphone, tablet, laptop, desktop computer, or smart wearable device. Terminal 30 can communicate with the second server 20 via mobile communication networks, wireless LANs, the Internet, etc.

[0050] Database 40 can be a relational database, a non-relational database, a vector database, a document database, etc., or it can be a distributed database or database cluster composed of multiple data storage devices. Database 40 can be deployed in an independent data storage device, or it can be deployed in the first server 10, the second server 20, or other servers; this disclosure does not limit it in this way.

[0051] The model training method provided in this embodiment can be executed by the first server 10, and the question-answering method provided in this embodiment can be executed by the second server 20.

[0052] The following description of several exemplary embodiments illustrates the technical solutions of this disclosure and the technical effects produced by these solutions. It should be noted that the following embodiments can be referenced, learned from, or combined with each other. Identical terms, similar features, and similar implementation steps in different embodiments will not be repeated.

[0053] This disclosure provides a method for model training, such as... Figure 3 As shown, the method includes: S201. Obtain multiple training samples.

[0054] Each training sample includes a question text and a set of candidate texts, which contain multiple candidate texts that serve as reference texts for generating answers to the question text.

[0055] In this embodiment of the disclosure, a training dataset can be obtained from a database, which includes multiple pre-generated samples. For each training sample, when generating the training sample, a question text can be obtained first, and multiple candidate texts can be searched from various data sources using a search engine to form a candidate text set. Each candidate text is a reference text related to the question text. To distinguish it from the standard reference text of the question text later, the reference text obtained by the search engine is called a candidate reference text.

[0056] Optionally, based on the text generation capabilities of the large language model, multiple question texts can be generated by prompting the large language model with preset prompt words, and a training sample can be constructed based on each question text. Optionally, questions asked by users in the past can also be obtained, and question texts can be obtained based on the questions asked in the past. Where a question in a past question contains multiple questions, the question in the past question can be split, and each of the split questions can be used as a separate question text.

[0057] Optionally, a search engine can be invoked, treating the question text as the query target. Based on each question text, searches can be performed from multiple data sources to obtain multiple candidate texts, which serve as the candidate text set in the training samples. Optionally, the search engine may include a retrieval unit, which retrieves text related to the question text from the data sources and can return multiple candidate texts based on the degree of relevance.

[0058] The data source can be set according to the actual application scenario and business needs. This disclosure does not limit the specific type or method of determining the data source. For example, the data source can include one or more of the following: internet data source, domain knowledge base, and business database. An internet data source can be a data source that can be accessed and obtain text information via the internet, such as web pages and public documents; a domain knowledge base can be a data collection built for a specific domain and storing knowledge in that domain, such as a medical knowledge base; a business database can be a database built for a specific business and storing business-related data, such as a database storing product information, user service information, and business documents. In practical applications, one or more data sources can be selected for candidate text retrieval based on the application scenario and business needs corresponding to the question text. This disclosure does not limit this selection.

[0059] Optionally, relevant question text can be generated based on the actual application scenario. For example, in application scenarios such as medical Q&A, educational Q&A, and intelligent customer service, question text related to the corresponding application scenario can be generated, so that the question text in the training samples matches the actual application scenario of the text filtering model, thereby making the trained text filtering model more suitable for the corresponding application scenario. Optionally, it can also be applied to a full-domain question answering scenario, that is, question text involving multiple different domains can be generated to improve the applicability of the trained text filtering model to questions in different domains.

[0060] S202. Using the above multiple training samples, perform multiple rounds of first training operations on the model to be trained until the preset first training termination condition is met. The model to be trained that meets the first training termination condition is used as the text filtering model.

[0061] In this embodiment of the disclosure, the model to be trained can be a neural network model with natural language understanding and text processing capabilities, such as a Large Language Model (LLM) or a language model based on the Transformer architecture.

[0062] The model to be trained serves as the initial model for training the text selection model. Through multiple rounds of training, the text selection model is obtained. This text selection model, also known as a set selection model, is used to determine the relevance between the question text and multiple candidate texts, and then select multiple candidate texts based on these relevances. These selected candidate texts are then used to generate the answers to the question text.

[0063] To distinguish it from other subsequent training operations, the training operations performed on the model to be trained are referred to as the first training operation, and the training termination condition of the model to be trained is referred to as the first training termination condition.

[0064] Optionally, the first training termination condition can be set according to actual training needs. For example, the first training termination condition may include at least one of the following: the number of rounds of the first training operation reaches a preset number of rounds, the total training loss converges, or the total training loss is less than a preset loss threshold. If the first training termination condition is met, the training of the model to be trained can be terminated, and the current model to be trained can be used as the trained text filtering model.

[0065] Optionally, batch training can be used when training the model to be trained. For example, multiple training samples can be divided into multiple batches. In the first training operation of each round, the corresponding training loss is determined based on the training samples in each batch, and the model parameters of the model to be trained are updated according to the training loss.

[0066] like Figure 4 As shown, the first training operation in each round includes: S301. For each question text in a training sample, the model to be trained selects multiple candidate texts from the candidate text set based on the first relevance between each candidate text in the training sample and the question text, thus obtaining the target text set.

[0067] In this embodiment, the question text and a set of candidate texts can be input into the model to be trained. Preset prompts are used to guide the model to filter multiple candidate texts from the candidate text set. The model can determine the first relevance of each candidate text to the question text, and based on the first relevance of each candidate text, filter multiple candidate texts from the candidate text set to obtain the target text set.

[0068] The model to be trained can predict the probability of each candidate text being selected based on the question text and each candidate text, obtain the probability distribution corresponding to each candidate text, and determine the first relevance corresponding to each candidate text based on the probability distribution.

[0069] Optionally, the relevance in this embodiment of the disclosure can be the prediction score (logit) of the model output for the candidate text, or it can be the probability value obtained after performing Softmax processing on the prediction score.

[0070] Optionally, a step-by-step selection method can be used to filter multiple candidate texts from the candidate text set. Specifically, the question text is taken as the initial state. During the first selection, the initial relevance of each candidate text in the candidate text set to the initial state can be determined by reasoning based on the initial state. Then, a candidate text is selected from the candidate text set based on the initial relevance of each candidate text to the initial state. The initial state is updated based on the selected candidate text to obtain the initial state for the next selection.

[0071] In subsequent selections, reasoning is performed on the remaining candidate texts based on the initial state to determine the first relevance of each candidate text to the initial state corresponding to the current selection. Then, a candidate text is selected from the remaining candidate texts based on the first relevance of each candidate text to the initial state corresponding to the current selection. This process is repeated until a preset number of candidate texts are selected.

[0072] Optionally, for each selection, when reasoning about the remaining candidate texts based on the initial state, it can be determined whether the selected candidate texts are sufficient to answer the question text. If it is determined that the selected candidate texts are sufficient to answer the question text, the selection can end. If it is determined that the selected candidate texts are insufficient to answer the question, candidate texts can be selected from the remaining candidate texts based on the above selection process. The first relevance between the candidate texts and the question text can include the first relevance between the candidate texts and each initial state.

[0073] In this embodiment of the disclosure, during the step-by-step selection process, the selection of subsequent candidate texts can take into account the previously selected candidate texts, so that different candidate texts in the target text set can jointly provide a reference for generating the answer to the question text.

[0074] Considering that existing solutions using the Top-K approach typically prioritize candidate texts with high relevance, resulting in a relatively fixed target text set for the same question text, it is difficult to fully grasp the overall optimization direction of different candidate text combinations. Therefore, this disclosure also provides a text filtering method based on probability sampling, which will be described in detail below.

[0075] As an optional implementation, for each question text in a training sample, the target text set corresponding to that question text is obtained in the following way: Using the model to be trained, determine the first relevance between each candidate text in the candidate text set and the question text; Based on the first relevance of each candidate text to the question text, the sampling probability of each candidate text is obtained. Based on the preset number of single samplings and the sampling probability of each candidate text, the candidate text set is sampled N times. The target text obtained from each sampling is used as a target text set, where N is an integer greater than or equal to 1.

[0076] In this embodiment, the first relevance between each candidate text and the question text can characterize the probability that each candidate text predicted by the model to be trained will be selected. A corresponding sampling probability can be determined based on the first relevance of each candidate text, and sampling can be performed according to the sampling probability of each candidate text to select a preset number of candidate texts from the candidate text set, thus obtaining the target text set. Here, the sampling probability can also be called the sampling weight. Optionally, the first relevance of each candidate text can be normalized, and the result of the normalization process can be used as the sampling probability of each candidate text.

[0077] For example, the candidate text set includes candidate texts A, B, C, D, and E, with first relevance scores of 0.8, 0.9, 0.6, 0.4, and 0.3 respectively to the question text. In one implementation, weighted sampling can be performed directly based on the first relevance scores of each candidate text. That is, 0.8, 0.9, 0.6, 0.4, and 0.3 are used as sampling probabilities for candidate texts A, B, C, D, and E, respectively. Multiple different candidate texts are selected from the five candidates at once according to their sampling probabilities. It can be seen that the probability of selecting candidate text A is greater than the probability of selecting candidate text E. The number of samples can be a preset number or determined by the model based on whether the selected candidate texts are sufficient to answer the question text. Since candidate texts with higher first relevance scores have higher sampling weights and are more likely to be selected into the target text set, this disclosure does not select the three candidate texts with the highest first relevance scores after sorting them by their first relevance scores, but rather performs weighted sampling based on the first relevance scores of each candidate text. In another implementation, the first relevance of each candidate text can be normalized first, and the normalization result can be used as the sampling probability of the corresponding candidate text. Then, according to the sampling probability of each candidate text, a sample is taken from the five candidate texts to obtain candidate texts A, B, and D. Candidate texts A, B, and D constitute the target text set. Optionally, combined with the aforementioned method of sequential screening, candidate texts can be selected from the candidate text set according to their sampling probabilities. Specifically, as shown... Figure 5 As shown, Figure 5 This is a schematic diagram of a process for filtering text one by one, provided as an embodiment of the present disclosure.

[0078] During the initial selection, the question text can be used as the initial state. The initial state and a set of candidate texts are input into the model to be trained. The model can then determine the first relevance of candidate text 1, candidate text 2, candidate text 3, and candidate text 4 to the initial state, based on the question text and the candidate text set. Based on the first relevance of each candidate text, a corresponding sampling probability is determined. A candidate text is then sampled from candidate text 1 to candidate text 4 according to this sampling probability. Taking candidate text 1 as an example, it can be considered the selected text.

[0079] In the next selection, the question text and the already selected candidate text 1 can be used as the initial state for the current selection. The initial state and the remaining candidate texts 2 to 4 in the candidate text set are input into the model to be trained. The first relevance corresponding to each remaining candidate text is determined, namely the first relevance of candidate text 2 to the initial state of the current selection, the first relevance of candidate text 3 to the initial state of the current selection, and the first relevance of candidate text 4 to the initial state of the current selection. Based on each first relevance, the corresponding sampling probability is determined, and sampling is performed again to select a candidate text from the remaining candidate texts. The selection can be repeated in the above manner until the selected candidate text meets the termination condition mentioned above, and the selected candidate texts are combined into a target text set.

[0080] In this embodiment of the disclosure, after selecting a candidate text each time, the context information (i.e., the initial state of the preceding text) on which the next selection is based is updated according to the selected candidate text. The model to be trained can then select from the remaining candidate texts based on the updated context information. By training the model to be trained based on this text selection method, the text filtering model can assign a lower first relevance score to other candidate texts with a high degree of repetition with the selected candidate text, even when the selected candidate text already contains reference information of a certain aspect, and can assign a higher first relevance score to candidate texts that can provide other relevant reference information.

[0081] It should be noted that the above-mentioned method of selecting candidate texts one by one can be regarded as an autoregressive generation process of a large language model. That is, the candidate texts selected each time can be used as conditions for subsequent selections, so that the selection results of subsequent candidate texts depend on the question text and the previously selected candidate texts.

[0082] In this embodiment of the disclosure, each sampling can correspond to multiple text selections. For the same training sample, the above-described sampling operation based on sampling probability can be performed multiple times. For example, N samplings can be performed, with each sampling selecting multiple candidate texts from the candidate text set to obtain a target text set, thereby obtaining N target text sets through N samplings.

[0083] like Figure 6 As shown, Figure 6 This is a schematic diagram illustrating multiple sampling to obtain multiple target text sets according to an embodiment of this disclosure. For the same candidate text set, a first sampling, a second sampling, ..., an Nth sampling can be performed. Each sampling can select multiple candidate texts from the candidate text set based on the sampling probability corresponding to each candidate text, forming the target text set corresponding to that sampling. Since a probability-based sampling method is used, the candidate texts selected in different samplings can be different. Therefore, N samplings can yield N different or partially identical target text sets. It should be noted that text selection in this embodiment of the disclosure refers to selecting multiple candidate texts in a single sampling process to form a target text set; N samplings mean performing the above selection process N times. Each sampling starts the selection process from the candidate text set again. Therefore, the same candidate text can be selected again in different samplings. For example, if the first sampling yields {A, C, D}, the second sampling may still yield {A, C, D}, or it may yield {A, B, D}. Therefore, the N target text sets obtained from N samplings may be the same, partially the same, or different.

[0084] N can be an integer greater than or equal to 1. When N is greater than 1, multiple combinations of candidate texts can be obtained by sampling the same training sample multiple times. Based on multiple target text sets, the processing effect of the model to be trained on the training sample can be determined, which is beneficial to provide more diverse training information for the training of the model to be trained.

[0085] Optionally, when determining the sampling probability of each candidate text based on its initial relevance, the initial relevance of each candidate text can be normalized using a Softmax function with a sampling temperature parameter. The sampling temperature can be used to adjust the probability distribution of each candidate text output by the model under training. The randomness of each sampling result can be controlled by presetting the sampling temperature value.

[0086] The sampling probability of candidate text i can be expressed as:

[0087] Where, p i s represents the sampling probability corresponding to the i-th candidate text; i represents the first relevance of the i-th candidate text; T represents the sampling temperature; M represents the number of candidate texts currently participating in the sampling.

[0088] It can be seen that the higher the sampling temperature, the smaller the difference in sampling probabilities between candidate texts, making candidate texts with relatively low selection probabilities more likely to be selected, thereby increasing the diversity of sampling results; the lower the sampling temperature, the more the sampling probability is concentrated on candidate texts with higher selection probabilities, making the sampling results more certain.

[0089] In this embodiment, a sampling temperature greater than a preset threshold can be set, such as 0.7, with the specific value determined according to actual application requirements. By increasing the differences in text composition among different target text sets under the same question text, the diversity of the sampled target text sets can be improved, resulting in differences in the answers generated subsequently based on different target text sets. By evaluating different answers, the impact of different target text sets on the quality of answer generation can be reflected based on the evaluation results, thereby avoiding the problem that target text sets obtained from multiple samplings are too consistent, making it difficult to form effective training information based on the differences between different target text sets, and improving the effectiveness of the training information used to update the text filtering model.

[0090] It should be noted that increasing the diversity of different target text sets can increase the differences in evaluation results between different target text sets and their corresponding answers. This is conducive to forming high-quality and low-quality target text sets based on the evaluation results, and then targeted training can be carried out based on the quality of the target text sets. The specific process will be explained later.

[0091] S302. For each training sample, the target text set and the question text are input into the question answering model. The question answering model generates the answer to the question text. The answer is evaluated based on the preset first evaluation index. Based on the evaluation result of the answer, the training loss of the training sample is determined.

[0092] Existing solutions typically evaluate the selected text, for example, based on the degree of matching between the selected text and the question text, and then train the text selection model based on this evaluation result. However, a high correlation between the selected text and the question text does not necessarily guarantee the generation of a high-quality answer based on that selected text. For instance, multiple selected texts may each have a high correlation with the question text, but they may contain a lot of duplicate information or lack some key information needed to generate an answer. This could result in answers generated from these texts that are still incomplete or inaccurate. Therefore, the training signal obtained by evaluating only the selected text itself may differ from the performance of the text selection model in real-world question-answering scenarios.

[0093] Based on this, the embodiments of this disclosure determine the corresponding training loss according to the quality of the generated answers, thereby indirectly evaluating the accuracy of the selection results of the target text set by utilizing the accuracy of answer generation. This aligns the optimization direction of the model with practical application preferences, where alignment with practical application preferences can be understood as ensuring that the evaluation objective of the target text set during training is consistent with the ultimate goal of selecting the target text set in practical application. In the embodiments of this disclosure, the ultimate goal of text selection is to select a target text set that helps generate high-quality answers. Therefore, the training loss can be determined based on the quality of the answers generated based on the target text set, and the model parameters of the text selection model can be updated based on this training loss. This allows the text selection model to learn which combinations of candidate texts are more conducive to generating high-quality answers, thereby improving the accuracy of text selection. This enables the trained text selection model to select the target text set used to generate high-quality answers in practical application.

[0094] Optionally, the question-answering model can be a large language model. The question text and its corresponding target text set can be input into the question-answering model, allowing the model to use multiple candidate texts from the target text set as reference information when generating the answer, and to generate the answer corresponding to the question text based on the reference information.

[0095] Optionally, the answers generated by the question-answering model can be evaluated using an Answer Reward Model (Answer RM) to obtain an evaluation result corresponding to the answer. Answer RM is a comprehensive scoring system composed of at least one reward model. In this embodiment, the quality of the answers generated by the question-answering model can be evaluated based on Answer RM. Optionally, Answer RM may include one or more evaluation models such as a Discriminant Reward Model (BTRM), a Criterion-Based Reward Model (RubricRM), or a Generative Reward Model (GRM), and can evaluate the generated answers from one or more evaluation dimensions such as authenticity, completeness, relevance, and fluency to obtain an evaluation result corresponding to the answer. This evaluation result can be a score.

[0096] Optionally, the above reward model can be implemented using a large language model. For example, a reward model based on evaluation criteria can input the question text, the answer generated by the question-answering model, the standard answer, and the preset evaluation criteria into the large language model. The large language model then uses the standard answer as an evaluation reference and evaluates the generated answer according to the preset evaluation criteria. For example, it can determine the authenticity score based on the degree of consistency between the generated answer and the standard answer in terms of factual information, the completeness score based on the degree of coverage of relevant information in the standard answer by the generated answer, the relevance score based on the degree of semantic matching between the generated answer and the question text, and the fluency score based on the language expression quality of the generated answer.

[0097] Discriminative reward models can input the question text, the generated answer, and the standard answer into a large language model. The large language model then judges the generated answer based on the standard answer, such as determining the degree of consistency between the generated answer and the standard answer, its quality level, or the probability of meeting preset quality requirements, and obtains the corresponding evaluation result. Optionally, it can also combine the target text set to determine whether the generated answer has corresponding textual basis.

[0098] Generative reward models can input question text, generated answers, and standard answers into a large language model. The large language model uses the standard answer as an evaluation reference to analyze the generated answers and generate evaluation information, such as information differences and missing information between the generated answers and the standard answers, and determines the corresponding evaluation results based on the evaluation information.

[0099] In this embodiment of the disclosure, when the Answer RM includes multiple reward models, the evaluation results obtained by each reward model for the same generated answer can be obtained separately, and the evaluation results are weighted to obtain the comprehensive evaluation result of the Answer RM for the generated answer. The preset first evaluation indicators may include one or more of the following: authenticity indicator, completeness indicator, relevance indicator, and fluency indicator. The Answer RM can determine the score of the answer under one or more first evaluation indicators, and determine the comprehensive evaluation score of the answer based on the evaluation scores corresponding to each first evaluation indicator, using this comprehensive evaluation score as the evaluation result of the answer.

[0100] Since the reference text used to generate the answers is the target text set selected by the model to be trained, the evaluation results of the answers can reflect the actual support effect of the target text set currently selected by the model for answer generation. When the target text set can support the question-answering model to generate high-quality answers, a corresponding positive training signal can be given; when the target text set leads to low-quality generated answers, a corresponding negative training signal can be given, and the training loss of the training sample can be determined based on the corresponding training signal to update the model parameters of the model to be trained.

[0101] Optionally, during the training of the model to be trained, the question-answering model and the Answer RM can be used as external models or external services, and their model parameters can remain unchanged, i.e., the model parameters of the question-answering model and the Answer RM are not updated based on the training loss mentioned above. The training loss mentioned above can be used to update the model parameters of the model to be trained, so that the model to be trained gradually learns a candidate text selection method that can support the question-answering model in generating high-quality answers.

[0102] The following section, in conjunction with the aforementioned alternative scheme of sampling the same training sample multiple times to obtain multiple target text sets, specifically explains the process of generating answers to question texts based on multiple target text sets, determining the evaluation results of each answer, and determining the training loss based on each evaluation result.

[0103] As an optional embodiment, N is an integer greater than or equal to 2; For each training sample, the answer to the question text, the evaluation result of the answer, and the training loss are obtained in the following way: For each target text set corresponding to the training sample, the target text set and the question text are input into the question answering model. The question answering model generates an answer to the question text based on the target text set. The answer is evaluated based on the first evaluation metric to obtain the evaluation result of the answer. The training loss of the training sample is determined based on the evaluation results of the answers to the questions corresponding to each target text set.

[0104] In this embodiment of the disclosure, for each training sample, multiple target text sets can be obtained through the aforementioned multiple sampling method, and corresponding answers can be generated based on each target text set, so as to determine the training loss of the training sample according to the evaluation results of multiple answers.

[0105] like Figure 7 As shown, Figure 7 This is a flowchart illustrating a process for determining the training loss of a training sample, provided by an embodiment of this disclosure. For a training sample, N sampling operations can be performed based on the question text and candidate text set in the training sample to obtain N target text sets. Each sampling operation corresponds to one target text set. For each of the N target text sets, the target text set and the question text of the training sample can be input into a question-answering model. The question-answering model uses the target text set as reference information when generating an answer to generate an answer corresponding to the question text. Thus, for the N target text sets, a first answer, a second answer, ..., an Nth answer can be generated respectively, each answer corresponding to a target text set obtained from one sampling operation.

[0106] The N answers can be evaluated separately based on the first evaluation index to obtain the evaluation result for each answer. Specifically, each answer can be evaluated separately in the same way as described above to determine the evaluation result of each answer under at least one evaluation dimension such as authenticity, completeness, relevance, and fluency, and the evaluation result of the answer can be determined according to the evaluation result corresponding to each evaluation dimension.

[0107] In this embodiment of the disclosure, the evaluation result of the answer can be a score value, which is used to characterize the accuracy of the corresponding answer. After obtaining the score values ​​corresponding to N answers, the training feedback corresponding to each sample can be determined according to each score value, and the corresponding training loss can be determined based on the training feedback corresponding to each sample. Further, the training losses corresponding to N samples can be summed or averaged, and the training loss of the training sample can be determined according to the result of the summation or averaging.

[0108] In this embodiment of the disclosure, by sampling the same training sample multiple times and generating and evaluating answers based on the target text sets obtained from each sampling, the evaluation results can indirectly reflect the support capability of the target text sets selected by the current model to be trained for generating correct answers. The determined training loss can more accurately characterize the text selection capability of the current model to be trained, thereby providing a reliable training basis for the parameter optimization of the model to be trained.

[0109] Optionally, the training loss of the training samples can also be determined by considering the selection of the target text set itself. A high-quality standard text set can be pre-determined for the question text, and the corresponding loss can be determined based on the difference between the target text set selected by the model to be trained and the standard text set. This ensures that the training process of the model to be trained not only considers the quality of the answer after the target text set is actually used to generate the answer, but also the difference between the target text set and the standard text set. This will be explained in detail below.

[0110] As an optional embodiment, the training samples also include a standard text set for the question text, which includes multiple standard texts, each of which is a standard reference text for generating the answer to the question text; For each training sample, the training loss for that training sample is determined based on the evaluation result of the answer, including: The first loss is determined based on the evaluation results of the answers corresponding to the training samples; The second loss is determined based on the difference between the target text set and the standard text set of the training samples; Based on the first loss and the second loss, the training loss of the training sample is determined.

[0111] In this embodiment of the disclosure, the training samples may further include a standard text set corresponding to the question text. The standard text set includes multiple standard texts, each of which can be a standard reference text used when generating the answer to the question text. The standard text set can be determined in advance based on the question text. For example, multiple texts that can provide effective references for generating the answer to the question text can be determined from the candidate text set through manual annotation, and the determined multiple texts are combined into a standard text set.

[0112] For a training sample, after obtaining the target text set corresponding to the training sample based on the model to be trained, the target text set and the question text can be input into the question answering model in the manner described above to generate the answer corresponding to the question text, and the training loss determined based on the evaluation result of the answer is used as the first loss of the training sample.

[0113] In this embodiment of the disclosure, in addition to determining the training loss of the training sample based on the evaluation result of the answer, the target text set obtained by the model to be trained can be compared with the standard text set corresponding to the training sample to determine the difference between the target text set and the standard text set, and the second loss of the training sample can be determined according to the difference. Based on the first loss and the second loss of the training sample, the training loss of the training sample is determined.

[0114] For example, the differences between the target text set and the standard text set can be determined based on one or more pieces of information, such as identical texts, different texts, and the order in which the texts were selected. The smaller the difference between the target text set and the standard text set, the smaller the second loss; the larger the difference between the target text set and the standard text set, the larger the second loss.

[0115] Optionally, the training loss of the training sample can be determined based on the first loss and the second loss. For example, the first loss and the second loss can be weighted and summed to obtain the training loss of the training sample. The weights corresponding to the first loss and the second loss can be set according to actual training needs, and this disclosure does not limit this.

[0116] In this embodiment, the first loss reflects the ability of the target text set currently selected by the model under test to support the generation of correct answers from the perspective of answer generation results. The second loss reflects the difference between the selection results of the model under test and the standard text set from the perspective of text selection results. By combining the first and second losses to train the model under test, the model parameters can be optimized simultaneously using the answer generation effect and the reference information provided by the standard text set.

[0117] Optionally, in the aforementioned implementation method of sampling the same training sample multiple times, one training sample can correspond to multiple target text sets. The following section combines... Figure 8This needs to be explained. For example... Figure 8 As shown, Figure 8 This diagram illustrates a method for determining a first loss and a second loss for a training sample, as provided in an embodiment of this disclosure. For a training sample, multiple samplings can be performed based on the question text and candidate text set of the training sample to obtain multiple target text sets.

[0118] For each target text set across multiple target text sets, the target text set and the question text can be input into the question-answering model to generate the corresponding answer. The generated answers are then evaluated to obtain the corresponding evaluation result. Based on the evaluation results of the answers corresponding to each of the multiple target text sets, the first sub-loss for each target text set can be determined, and the first loss for the training sample can be determined based on each first sub-loss. For example, the first loss for the training sample can be obtained by summing or averaging multiple first sub-losses.

[0119] Optionally, each target text set can be compared with the standard text set corresponding to the training sample. Based on the difference between each target text set and the standard text set, the second sub-loss corresponding to that target text set is determined, and the second loss of the training sample is determined based on the second sub-losses corresponding to multiple target text sets. For example, the second loss of the training sample can be obtained by summing or averaging multiple second sub-losses.

[0120] In this embodiment of the disclosure, by determining a first loss based on the quality of the answers corresponding to the target text set and a second loss based on the difference between the target text set and the standard text set, the model to be trained can be optimized from two aspects: the answer generation effect and the text selection result. This allows the text selected by the model to effectively support the generation of high-quality answers and match the high-quality text selection results represented by the standard text set, thereby improving the model's text selection ability.

[0121] In this embodiment of the disclosure, the above-described scheme of sampling the same training sample multiple times to obtain multiple target text sets can be used to obtain multiple text selection results of the model to be trained for the same question text. The training loss is determined by combining the answer quality corresponding to multiple text selection results and the differences between them and the standard text set, thereby more comprehensively reflecting the current text selection ability of the model to be trained and providing richer training information for model parameter optimization.

[0122] The following is an alternative method for determining the standard text in the training samples.

[0123] As an optional implementation, the standard text set in each training sample is obtained in the following way: Obtain a preset second prompt word, which is used to prompt the large language model to output the answer to the question text and the text identifier of the candidate text referenced when generating the answer, based on the input question text and candidate text set of the large language model; The question text of the training sample and the candidate text set are input into the large language model. The large language model performs multiple answer generation operations to obtain multiple answers output by the large language model, as well as the text identifiers of the candidate texts referenced in the generation of each answer. The text identifiers of the candidate texts referenced when generating each answer are statistically analyzed, and a preset number of candidate texts are selected from the referenced candidate texts in descending order of reference frequency. Based on the predetermined number of candidate texts, the standard text set is obtained.

[0124] In this embodiment of the disclosure, corresponding text identifiers can be pre-set for each candidate text in the candidate text set. The second prompt word can prompt the large language model to annotate the candidate text referenced in the answer when generating the answer to the question text. For example, the large language model can annotate the corresponding candidate text in the answer by referencing superscripts or text numbers, thereby establishing a correspondence between the answer content and the referenced candidate text.

[0125] For the same training sample, a large language model can perform multiple answer generation operations, each of which can be called a Rollout. During each Rollout, the large language model can generate the corresponding answer based on the question text and the candidate text set, and output the text identifiers of one or more candidate texts that were actually referenced when generating the answer.

[0126] Furthermore, the text identifiers corresponding to multiple rollouts can be statistically analyzed. For example, for each candidate text, the number of times it is referenced in multiple answer generation processes can be counted, and the reference frequency corresponding to the candidate text can be determined based on the number of references. According to the reference frequency from high to low, a preset number of candidate texts can be determined from the referenced candidate texts, such as 4 to 7 candidate texts, and a standard text set can be obtained based on the determined candidate texts.

[0127] In cases where the large language model generates answers independently multiple times, the candidate texts referenced in each generation may differ. The higher the frequency with which a candidate text is referenced across multiple generation processes, the more stable its use by the large language model in generating answers to the given question. Therefore, by statistically analyzing the candidate texts actually referenced in multiple generation processes, we can utilize the citation information generated by the large language model during answer generation to determine the reference value of candidate texts. This reduces the impact of the randomness of individual answer generation results on the determination of the standard text set, providing stable candidate texts for the subsequent construction of the standard text set.

[0128] As an optional embodiment, the standard text set obtained based on the determined preset number of candidate texts includes: The intent analysis of the question text is performed using a large language model, and a text evaluation strategy for the question text is generated based on the analysis results. Based on this preset number of candidate texts, multiple subsets of candidate texts are obtained; Based on this text evaluation strategy, each candidate text subset is evaluated, and the standard text set is determined from these multiple candidate text subsets according to the evaluation results.

[0129] In this embodiment, by analyzing the reference frequency of candidate texts during multiple answer generation processes, candidate texts that are frequently referenced by the large language model when generating answers to the question text can be obtained. However, the text identifiers output by the large language model during answer generation may contain noise. For example, the identified candidate texts may have low relevance to the actual answer requirements of the question text, or there may be significant overlap in information between different candidate texts. Therefore, the aforementioned candidate texts can be further evaluated and selected to obtain a high-quality standard text set.

[0130] Optionally, after obtaining candidate texts for reference through multiple answer generation operations, the candidate texts can be further processed through a preset offline text selection process. This offline text selection process can generate a text evaluation strategy that matches the question text based on the analysis results of the question text, and clean and optimize the candidate texts based on the text evaluation strategy to obtain the standard text set corresponding to the question text.

[0131] Specifically, intent analysis can be performed on the question text using a large language model to determine the information needed to answer the question, and a text evaluation strategy can be generated based on the intent analysis results. This text evaluation strategy can include one or more evaluation dimensions and corresponding evaluation criteria for each dimension. For example, evaluation dimensions may include one or more of the following: relevance, authority, timeliness, comprehensiveness, and content conflict. For different question texts, different text evaluation strategies can be generated based on the intent of the corresponding question text, ensuring that the text evaluation strategy matches the actual answer requirements of the question.

[0132] Optionally, multiple candidate text subsets can be constructed based on the aforementioned predetermined number of candidate texts. Different candidate text subsets may include different combinations of candidate texts. For each candidate text subset, an overall evaluation can be performed according to the text evaluation strategy. For example, the evaluation can assess whether the candidate text subset contains the core information needed to answer the question, whether the information provided by different candidate texts is complementary, whether there is excessive repetition, whether the provided information has sufficient authority and timeliness, and whether there are content conflicts between different candidate texts, and obtain the evaluation result corresponding to the candidate text subset. Optionally, the second evaluation index for the target text set described below can be used to perform an overall evaluation of the candidate text subsets. The specific evaluation scheme for the second evaluation index will be explained in detail below.

[0133] Based on the evaluation results corresponding to multiple candidate text subsets, a candidate text subset whose evaluation results meet the preset conditions can be selected from the multiple candidate text subsets, and the selected candidate text subset is determined as the standard text set corresponding to the question text.

[0134] In this embodiment of the disclosure, based on the preliminary candidate results obtained by utilizing the candidate text reference information generated during multiple answer generation processes, a corresponding text evaluation strategy can be further generated according to the specific intent of the question text. The different candidate text combinations are then evaluated as a whole according to the text evaluation strategy, thereby filtering out low-quality texts in the reference results. Furthermore, by comprehensively considering the complementarity, repetition, and content conflicts among multiple texts, a standard text set that is more suitable for supporting the generation of answers to the question text is obtained, thus improving the quality of the standard text set.

[0135] Optionally, in the alternative approach of obtaining multiple target text sets by sampling the same training sample multiple times, the quality of different target text sets varies. If the same training method is used for target text sets of different qualities, it may be difficult to fully utilize the training information provided by different target text sets. Based on this, the embodiments of this disclosure can first evaluate the quality of multiple target text sets, and divide the multiple target text sets into text sets of different qualities according to the evaluation results. Different methods are used to determine the training loss for text sets of different qualities, which will be described in detail below.

[0136] As an optional embodiment, the number of target text sets corresponding to each of the above training samples is multiple, and for each training sample, the method further includes: Based on a preset second evaluation index, the quality of each target text set corresponding to the training samples is evaluated to obtain a score value for each target text set. For each target text set, if the score of the target text set is less than a first threshold, the target text set is designated as the first text set; if the score of the target text set is greater than or equal to the first threshold, the target text set is designated as the second text set. For each of the second text sets, the advantage value of the second text set is determined based on the score value of the second text set and the benchmark score value, and the training loss of the second text set is determined based on the advantage value. For each training sample, determining the second loss based on the difference between the target text set and the standard text set of the training sample includes: For each first text set corresponding to the training samples, the training loss corresponding to the first text set is determined based on the difference between the first text set and the standard text set. The second loss of the training sample is determined based on the training loss of each first text set and the training loss of each second text set corresponding to the training sample.

[0137] like Figure 9 As shown, Figure 9 This diagram illustrates a method for determining a second loss for training samples, as provided in an embodiment of this disclosure. In this embodiment, each training sample may correspond to multiple target text sets, which can be obtained through the aforementioned method of sampling candidate text sets multiple times. For each training sample, the quality of each target text set corresponding to that training sample can be evaluated based on a preset second evaluation metric to obtain a score value for each target text set.

[0138] The second evaluation metric can be used to evaluate the quality of the target text set from the perspective of the target text set as a whole. Optionally, the target text set can be evaluated using a pre-defined text evaluation model. For example, an evaluation strategy-based model (Rubric Judge RM) can be used to evaluate the target text set as a whole according to the text evaluation strategy corresponding to the question text.

[0139] In this embodiment of the disclosure, the evaluation object is the entire target text set. Compared with evaluating each candidate text in the target text set independently, it can comprehensively consider the relationship between each candidate text and the overall information quality of the combination of multiple candidate texts. This avoids inaccurate evaluation of the target text set due to high quality of a single candidate text but content duplication, missing information or content conflict among multiple candidate texts, thereby improving the accuracy of the evaluation of the quality of the text combination.

[0140] For example, if multiple candidate texts in a target text set are highly relevant to the question text, but there is significant overlap in the information provided by these candidate texts, the overall score of the target text set may still be low. Conversely, if the multiple candidate texts can provide different aspects of the information needed to answer the question, and there is good information complementarity between the different candidate texts, the target text set can achieve a higher score. Therefore, the score can more accurately reflect the overall support capability of the target text set as a combination of texts for answering the question.

[0141] Optionally, in conjunction with the aforementioned scheme of setting the sampling temperature value to be greater than a preset threshold, by increasing the difference between the target text sets obtained from multiple samplings, multiple target text sets with significant quality differences can be obtained during the training process. This is beneficial for identifying the target text sets with higher quality and those with lower quality, thus providing a foundation for subsequent training on target text sets of different quality.

[0142] Optionally, after obtaining the score values ​​corresponding to each target text set, the multiple target text sets can be classified by quality based on a preset first threshold. For each target text set, if the score value of the target text set is less than the first threshold, the target text set can be identified as the first text set; if the score value of the target text set is greater than or equal to the first threshold, the target text set can be identified as the second text set. The first text set can correspond to target text sets of relatively lower quality, and the second text set can correspond to target text sets of relatively higher quality. Thus, appropriate training signals can be provided for target text sets of different qualities based on their current quality.

[0143] Optionally, the first threshold can be a pre-set fixed threshold, or it can be dynamically determined based on the score values ​​of multiple target text sets corresponding to the current training sample. For example, the score values ​​of multiple target text sets corresponding to the same training sample can be averaged, and the resulting average score value can be determined as the first threshold. Thus, based on the overall quality level of multiple target text sets under the same training sample, a relative quality classification can be made for each target text set.

[0144] For a second text set whose score is greater than or equal to the first threshold, the advantage value corresponding to the second text set can be determined based on the score value of the second text set and the benchmark score value, and the training loss corresponding to the second text set can be determined based on the advantage value.

[0145] The benchmark score can be determined based on the scores of multiple target text sets corresponding to the same training sample. Optionally, the average score of the N target text sets corresponding to the training sample can be used as the benchmark score.

[0146] Optionally, the score of the second text set can be standardized based on the average and standard deviation of the scores of the N target text sets to obtain the advantage value corresponding to the second text set. Wherein, the advantage value of the i-th second text set... It can be represented as follows:

[0147] in, Let be the score value of the i-th second text set. Let be the average score of the N target text sets. Let be the standard deviation of the score values ​​for the N target text sets.

[0148] If the score of the second text set is higher than the benchmark score, the advantage value of the second text set is positive, thus forming a positive training signal, which increases the probability of the model being trained selecting the second text set or a text combination with similar selection features to the second text set. If the score of the second text set is lower than the benchmark score, the advantage value of the second text set can be negative, thus forming a negative training signal, which decreases the probability of the model being trained selecting the corresponding text combination.

[0149] Optionally, for the second text set, Group Relative Policy Optimization (GRPO) can be used to determine the corresponding training loss based on the aforementioned advantage value. By determining the advantage value using the relative scoring relationship between multiple target text sets corresponding to the same training sample, multiple sampling results within the group can be directly used to form a relative training signal, achieving the effect of exploring diverse high-scoring text sets.

[0150] Among them, the training loss corresponding to each second text set It can be represented as follows:

[0151] in, This refers to the set of second text sets whose score values ​​are greater than or equal to the first threshold across multiple target text sets corresponding to the same training sample; Indicates the first A second text set; Indicates the number of text sets in the second text set; Indicates the first Normalized dominance values ​​of each second text set pair; Indicates that the model to be trained is for the first... The strategy probability ratio corresponding to each second text set; This indicates the preset clipping parameters; This indicates a clipping operation, used to limit the probability ratio within a preset range. E represents the expectation, which is the expected value of the quantity in parentheses according to the corresponding data.

[0152] It should be noted that the second text set can be trained primarily based on its quality advantage over other target text sets under the same training samples, without requiring the second text set to be consistent with the standard text set. Since the second text set already represents a high-scoring text selection result, the model to be trained can continue to explore other high-quality text combinations based on the already superior text selection results, thereby learning a text selection method that differs from the standard text set but can still effectively support question answering.

[0153] Optionally, for a first text set whose score is less than a first threshold, the training loss corresponding to the first text set can be determined based on the difference between the first text set and the standard text set corresponding to the training sample.

[0154] The low score of the first text set indicates that the quality of the text selection results obtained by the model to be trained is relatively low. For this type of text selection result, the aforementioned offline-determined standard text set can be used to provide clear guidance, enabling the text selection results of the model to be trained to move closer to the better text selection results represented by the standard text set. The scheme for determining the difference between the first text set and the standard text set can refer to the scheme for determining the difference between the target text set and the standard text set mentioned earlier.

[0155] Optionally, since the second text set is a high-quality text set, for each first text set, the training loss corresponding to the first text set can also be determined based on the difference between the first text set and at least one second text set, so that the model to be trained can learn the text selection features corresponding to the second text set and improve the ability of the model to be trained to select high-quality text.

[0156] In this embodiment of the disclosure, for each training sample, the training loss corresponding to each first text set and the training loss corresponding to each second text set can be obtained respectively, and the second loss of the training sample can be determined based on the training loss corresponding to each first text set and the training loss corresponding to each second text set. For example, the training losses corresponding to each first text set and each second text set can be summed, averaged, or weighted to obtain the second loss of the training sample.

[0157] In this embodiment, multiple target text sets are stratified according to their quality, and different optimization methods are adopted for target text sets of different qualities. For the second text set with higher quality, the relative merits of multiple sampling results can be used for strategy optimization, enabling the model to further explore other high-quality text combinations based on the already superior text selection results. For the first text set with lower quality, a standard text set can be used to provide clearer reference information to correct the current text selection results. Compared to using the same training method for target text sets of different qualities, training signals adapted to the quality of each target text set can be provided, thereby making fuller use of training data of different qualities obtained from multiple samplings and improving the training effect of the model to be trained.

[0158] Optionally, by combining the first loss determined from the evaluation results of the answers generated based on the target text set, the model to be trained can be jointly optimized at different levels. The first loss reflects the ability of the target text set selected by the model to support the generation of high-quality answers from the perspective of the final answer generation effect; the second loss strengthens the relative advantage of high-quality text selection results from the perspective of the quality of the target text set itself, and corrects lower-quality text selection results using a standard text set. Thus, the training signals at the answer generation level and the quality-layered training signals at the text selection level can be used together to optimize the parameters of the model to be trained, enabling the model to gradually learn text selection methods that are more conducive to generating high-quality answers.

[0159] Specifically, for each training sample, the training loss for that training sample... It can be expressed as follows:

[0160] The training loss in this embodiment of the disclosure can be regarded as an optimization objective. The training loss is determined based on all the second text sets. The training loss is determined based on all the first text sets. The second loss that constitutes the training samples of this disclosure, The training loss is determined based on all target text sets, which is also the first loss for the training samples. , and It can be a preset weight value.

[0161] The following is an alternative scheme for determining the differences between the target text set and the standard text set. This alternative scheme is also applicable to determining the differences between the first text set and the standard text set.

[0162] As an optional embodiment, the above-described determination of the second loss based on the difference between the target text set and the standard text set of the training samples includes: Input the question text, candidate text set, and standard text set into the model to be trained. The model to be trained then determines the second relevance between each candidate text in the candidate text set and the question text by referring to the standard text set. Determine the difference between the first and second relevance scores for each target text in the target text set, where the difference represents the difference between the target text set and the standard text set; The second loss of the training samples is determined based on the differences between the target text sets corresponding to the training samples.

[0163] In this embodiment of the disclosure, a second loss can be determined based on the difference between the target text set and the standard text set, so as to guide the text selection process of the model to be trained using the standard text set. The question text, candidate text set and standard text set can be input into the model to be trained, and the model to be trained can refer to the standard text set to determine the second relevance between each candidate text in the candidate text set and the question text.

[0164] The second relevance can be used to characterize the probability that each candidate text will be selected by the model to be trained, given that a standard text set provides reference information. The second relevance can be represented by the probability, probability distribution, score, or other information that can characterize the probability of a candidate text being selected.

[0165] The aforementioned first relevance can be the relevance of each candidate text determined based on the question text and the candidate text set when the model to be trained does not refer to the standard text set; the second relevance can be the relevance of each candidate text determined when the model to be trained further refers to the standard text set. Therefore, the first relevance can reflect the selection tendency of the model to be trained when selecting candidate texts according to the current text selection strategy, and the second relevance can reflect the reference selection tendency when selecting candidate texts with the reference of the standard text set.

[0166] For each target text in the target text set, the difference between the first relevance and the second relevance can be determined, and based on the differences between each target text, the difference for the entire target text set can be determined. The larger the difference between the first and second relevance, the greater the deviation between the selection tendency of the model being trained and the selection tendency obtained from the reference standard text set; the smaller the difference, the closer the selection tendencies are. Therefore, the difference between the first and second relevance of each target text can be used to characterize the difference between the target text set and the standard text set.

[0167] In cases where the same training sample corresponds to multiple target text sets, the aforementioned differences for each target text set can be determined separately, and the second loss for that training sample can be determined based on these differences. For example, the differences for each target text set can be summed, averaged, or weighted to obtain the second loss.

[0168] Therefore, compared to determining the difference between the target text set and the standard text set based on whether the texts contained in the target text set are the same, the embodiments of this disclosure can further determine the difference between the current text selection result and the text selection result obtained from the reference standard text set based on the selection tendency of the model to be trained on candidate texts, thereby providing a more granular training signal for the model to be trained.

[0169] Combining the previous method of gradually filtering text, such as Figure 10 As shown, Figure 10 This is a schematic diagram illustrating the determination of a second loss according to an embodiment of this disclosure. In this embodiment, a target text set can be generated by progressively selecting candidate texts, and a first relevance and a second relevance can be determined at each text selection position. The second loss is then determined based on the relevance differences corresponding to each text selection position. The following example illustrates a candidate text set including candidate texts A, B, C, and D, with the standard text set corresponding to the text selection order being "A→B→C".

[0170] Without referencing a standard text set, the question text and candidate text set can be input into the model to be trained. The model then determines the first relevance of each candidate text based on the question text and candidate text set, and progressively selects candidate texts based on this first relevance. For example, after selecting candidate text C at the first text selection position, the selected candidate text C can be used as a condition for subsequent text selection to redetermine the first relevance of the remaining candidate texts; after further selecting candidate text B, the selected candidate texts C and B can be used as conditions for subsequent text selection to again determine the first relevance of the remaining candidate texts. This process can progressively generate the corresponding target text set. On the other hand, the question text, candidate text set, and standard text set can be input into the model to be trained, allowing the model to further refer to the standard text set and determine the second relevance of each candidate text at the corresponding text selection position. The second relevance can serve as reference selection information for the corresponding text selection position, representing the probability that each candidate text will be selected at that position given the reference provided by the standard text set.

[0171] For each selected text location, the first relevance determined by the model being trained can be compared with the second relevance determined by the reference standard text set to determine the relevance difference corresponding to that selected text location. For example, in Figure 10 The first text selection position shown can determine the difference between the first relevance and the second relevance corresponding to candidate text C; given that candidate text C has been selected, the difference between the first relevance and the second relevance corresponding to candidate text B can be determined at the second text selection position; furthermore, given that candidate texts C and B have been selected, the difference between the first relevance and the second relevance corresponding to candidate text D can be determined at the third text selection position.

[0172] Optionally, when relevance is represented by a probability distribution, the difference corresponding to the selected text position can be determined by the distance between two probability distributions. For example, methods such as KL divergence can be used to determine the difference between the first probability distribution output by the model to be trained at the corresponding text selection position and the second probability distribution obtained from the reference standard text set. Furthermore, the differences corresponding to multiple text selection positions in the target text set can be summed, averaged, or weighted to obtain the training loss corresponding to the target text set, and the second loss of the training sample can be determined based on the training loss corresponding to each target text set.

[0173] In this embodiment, the difference between the current text selection tendency of the model to be trained and the text selection tendency obtained from a reference standard text set can be determined at different text selection positions in the target text set. When the difference between the first relevance and the second relevance corresponding to a certain text selection position is large, the text selection position can generate a relatively large training signal, thereby making a stronger adjustment to the model to be trained at that text selection position; when the difference between the two is small, the corresponding training signal is also small, thereby reducing the adjustment of the model to be trained at that text selection position. Thus, correction signals of different intensities can be provided according to the actual deviation of the model to be trained at different text selection positions, improving the accuracy of correction in the text selection process.

[0174] Optionally, the method of determining the training loss based on the difference between the first and second relevance can be applied to the first text set whose score is less than the first threshold. Since the first text set has been identified as having relatively low-quality text selection results, a standard text set determined offline can be used to provide reference selection information. Online policy distillation (OPD) is then performed on the model under test by comparing the current text selection probability distribution of the model under test with the reference probability distribution obtained from the standard text set. Compared to continuing to explore based on low-quality text selection results, this method can utilize the superior text selection results provided by the standard text set to specifically adjust the text selection positions where the model under test has significant biases. This allows the text selection strategy of the model under test to be adjusted towards the superior text selection region represented by the standard text set, thereby improving the efficiency of correcting low-quality text selection results.

[0175] In this embodiment of the disclosure, when determining the second loss based on the difference between the target text set and the standard text set, the difference between the current selection probability distribution of the model to be trained and the reference probability distribution determined by the reference standard text set can be determined at each text selection position along the text selection trajectory of the target text set generated by the model to be trained, and the second loss can be determined based on the difference corresponding to each text selection position. For example, the second loss can be expressed as:

[0176] in, This represents the training loss determined based on the difference between the target text set and the standard text set; This represents the target text set obtained by the model being trained itself. Represents the target text set According to the current text selection strategy of the model to be trained Obtained by sampling; Represents the question text; Indicates a set of candidate texts; This represents the probability distribution of candidate text selection by the model to be trained based on the question text and the candidate text set. This represents the reference probability distribution of candidate texts determined based on a standard text set. This represents the KL divergence, used to characterize the difference between the two probability distributions mentioned above.

[0177] Furthermore, for each text selection position during the process of generating the target text set by the model to be trained, the correction signal corresponding to that text selection position can be determined based on the difference between the reference probability determined based on the standard text set and the selection probability determined by the model to be trained. For example, the correction signal corresponding to the t-th text selection position can be expressed as:

[0178] in, This represents the correction signal corresponding to the t-th text selection position; This represents the candidate text selected by the model to be trained at the t-th text selection position; This represents the candidate texts selected up to the t-th text selection position; Represents the question text; This indicates a reference selection strategy determined based on a standard text set. This indicates that, given the question text and previously selected candidate texts, candidate texts are selected based on the selection strategy. The probability of; This indicates the current text selection strategy of the model to be trained. This indicates that the model to be trained selects candidate texts under the same conditions. The probability of.

[0179] Alternatively, the question text, candidate text set, and standard text set can be input into the teacher model. The teacher model can then refer to the standard text set to determine the second relevance between each candidate text in the candidate text set and the question text, so that the model to be trained can learn the text relevance distribution obtained by the teacher model under the guidance of the standard text set.

[0180] Optionally, a preset third prompt word can be used to prompt the model to be trained to select the target text set from the candidate text set based on the input question text and candidate text set, and output the probability (i.e., the first relevance) of each candidate text in the target text set; alternatively, a preset fourth prompt word can be used to prompt the teacher model to refer to the standard text set and determine the probability (i.e., the second relevance) of each candidate text in the target text set based on the question text and candidate text set.

[0181] S303. Based on the training loss of each training sample, update the model parameters of the model to be trained to obtain the model for the next round of training.

[0182] In this embodiment of the disclosure, the total training loss for the current training round can be determined based on the training loss corresponding to each training sample used in the current training round. For example, the training losses corresponding to each training sample can be averaged, and the resulting average training loss can be determined as the total training loss. Optionally, the total training loss can also be determined based on the training losses corresponding to each training sample by summation, weighted summation, or weighted averaging, etc., and this disclosure does not limit this method.

[0183] After determining the total training loss, the gradients corresponding to the model parameters of the model to be trained can be determined based on the total training loss. The model parameters of the model to be trained can be updated based on the gradients, and the model to be trained with updated model parameters can be used as the model for the next round of training.

[0184] By repeatedly performing the above training process, the model under test can gradually learn to select candidate texts from the candidate text set that are conducive to generating high-quality answers. Training can be terminated when the first training termination condition is met, and the current model under test is identified as the trained text selection model.

[0185] In this embodiment of the disclosure, during the training of the text filtering model, the selected target text set is used to generate question answers, and the generated answers are evaluated based on preset evaluation metrics. The training loss is determined based on the evaluation results of the answers to iteratively train the text filtering model. This achieves optimization of the candidate text filtering process based on the final answer generation effect. Compared with determining the model optimization direction by evaluating the correlation between the text in the target text set and the question text, this method can match the optimization direction of the text filtering model with the final answer generation effect. In practical applications, the text filtering model trained by this embodiment can filter out reference texts that are more conducive to generating accurate answers from multiple candidate texts, which is more in line with the application needs of actual question-and-answer scenarios, thereby improving the accuracy of answer generation in complex question scenarios.

[0186] The second evaluation index in the embodiments of this disclosure will be described below.

[0187] As an optional embodiment, for each training sample, the quality of each target text set corresponding to that training sample is evaluated based on a preset second evaluation metric to obtain the evaluation result for each target text set, including: Obtain the first prompt word, which is used to prompt the large language model to evaluate the target text set input to the large language model based on the second evaluation metric; The first prompt word, the second evaluation metric, the question text, and the target text set are input into a large language model to obtain the evaluation result of the target text set output by the large language model; wherein, the second evaluation metric includes at least one of the following: The degree of information coverage of the question text by all candidate texts in the target text set; The degree of information redundancy among candidate texts in the target text set; The degree of matching between the relevance and contribution of each candidate text in the target text set. The relevance is the relevance between the candidate text and the question text, and the contribution is the contribution of the target text set to the obtained answer. The percentage of candidate texts in the target text set whose relevance to the question text is lower than a preset value.

[0188] In this embodiment of the disclosure, for each training sample, the large language model can evaluate each target text set corresponding to the training sample based on a preset second evaluation index to obtain the evaluation result corresponding to each target text set.

[0189] Specifically, a first prompt word can be obtained, which is used to prompt the large language model to evaluate the input target text set according to the second evaluation metric. The first prompt word, the second evaluation metric, the question text, and the target text set to be evaluated can be input into the large language model. The large language model combines the question text with the overall analysis of multiple candidate texts in the target text set, and outputs the evaluation result corresponding to the target text set according to the second evaluation metric.

[0190] Optionally, the above evaluation process can be implemented using a large language model evaluation service based on an evaluation strategy, such as the aforementioned Rubric Judge RM. Rubric Judge RM can be word-driven, eliminating the need for additional model training for the evaluation task of the target text set. The first prompt word can be used to define the evaluation object, evaluation dimensions, evaluation rules for each evaluation dimension, and the output format of the evaluation results for the large language model, enabling the large language model to perform structured evaluation of the target text set according to preset evaluation rules. Furthermore, when it is necessary to adjust the second evaluation metric, the content related to the second evaluation metric in the first prompt word can be modified, allowing the large language model to evaluate according to the modified second evaluation metric. Compared to schemes that achieve different evaluation objectives by training a reward model, this embodiment can change the evaluation criteria by adjusting the first prompt word, eliminating the need to retrain the reward model for changes in the evaluation criteria, thereby reducing the training cost required for adjusting the evaluation criteria.

[0191] The second evaluation indicator may include at least one of the following: First, information coverage. The degree to which all candidate texts in the target text set cover the information of the question text can be used to characterize how well the information provided by the target text set covers the various information needs involved in answering the question text. For example, one or more information needs that need to be answered can be identified based on the question text, and it can be determined how well the information provided by multiple candidate texts in the target text set can cover these information needs. The more comprehensive the coverage of information needs, the higher the corresponding degree of information coverage.

[0192] Second, the degree of information redundancy. The degree of information redundancy among candidate texts in the target text set can be used to characterize the degree of overlap in the information provided by different candidate texts. For example, if multiple candidate texts contain a large amount of the same or similar information, the degree of information redundancy can be high; if different candidate texts provide different aspects of information and there is good information complementarity between them, the degree of information redundancy can be low. Therefore, it is possible to avoid evaluating a highly repetitive candidate text set as a high-quality target text set simply because multiple candidate texts have high relevance individually.

[0193] Third, the degree of matching between relevance and contribution. The degree of matching between the relevance and contribution of each candidate text in the target text set can be used to characterize whether the order of the candidate texts in the target text set matches their importance to the question answer. Relevance characterizes the degree of correlation between the candidate text and the question text; contribution characterizes the degree of contribution of the candidate text to generating the corresponding answer to the question text. For example, if a candidate text that is highly relevant to the question text and contributes significantly to the generated answer has a high degree of matching in the target text set, then the target text set has a high degree of matching. Therefore, the evaluation results can further reflect the rationality of the order of the candidate texts in the target text set.

[0194] Fourth, the proportion of low-relevance candidate texts. The proportion of candidate texts in the target text set whose relevance to the question text is lower than a preset value can be used to characterize the noise level in the target text set. For example, the relevance between each candidate text in the target text set and the question text can be determined separately, and candidate texts with relevance lower than the preset value can be identified as low-relevance candidate texts. The proportion is determined based on the ratio between the number of low-relevance candidate texts and the total number of candidate texts in the target text set. The higher this proportion, the more information in the target text set is irrelevant or only marginally relevant to the question answer, and the lower the overall quality of the target text set.

[0195] Furthermore, multiple sub-scores corresponding to the target text set can be obtained based on the aforementioned evaluation indicators, and these sub-scores can be comprehensively processed to obtain the overall score of the target text set. For example, the sub-scores can be weighted and summed according to the preset weights corresponding to each second evaluation indicator.

[0196] in, This represents the score of the target text set. Indicates the target text set in the 1st... Sub-scores under the second evaluation indicator Indicates the first The weights corresponding to the second evaluation indicators This indicates the number of second evaluation indicators. The weights for each evaluation indicator can be preset or determined based on the type of question text, the information requirements corresponding to the question text, etc.

[0197] It should be noted that the second evaluation metric is used to evaluate the overall quality of the target text set composed of multiple candidate texts. For example, even if each candidate text in the target text set is highly relevant to the question text, if there is a large amount of information duplication among the candidate texts, the information requirements of the question are not fully covered, or the order of the candidate texts is unreasonable, the target text set can still receive a low score. Conversely, if the multiple candidate texts can jointly cover the information requirements corresponding to the question text, the information complementarity between different candidate texts is high, and there is little noisy text, a higher score can be obtained.

[0198] Therefore, the embodiments of this disclosure use a first prompt word to prompt a large language model to evaluate the target text set as a whole according to multiple second evaluation indicators. This can comprehensively consider the information coverage of the target text set, the information duplication between texts, the rationality of the arrangement of candidate texts, and the situation of noisy texts. This allows the evaluation results to more comprehensively characterize the ability of multiple candidate texts as a text combination to support the answer to the question, and provides an evaluation basis for the aforementioned quality stratification of multiple target text sets and determination of the second loss.

[0199] As an optional implementation, the above-mentioned model to be trained is obtained in the following way: Obtain the teacher model trained using a supervised training method; Based on the multiple training samples and the teacher model, the student model undergoes multiple rounds of second training operations until a preset second training termination condition is met. The student model that meets the second training termination condition is then used as the model to be trained. Each round of second training operations includes: For each training sample, the third relevance between each candidate text in the candidate text set of the training sample and the question text is determined by the student model, and the fourth relevance between each candidate text in the candidate text set of the training sample and the question text is determined by the teacher model. For each training sample, the third loss of the training sample is determined based on the difference between the third and fourth relevance of each candidate text in the candidate text set of the training sample. Based on the third loss of each training sample, the model parameters of the student model are updated to obtain the student model for the next round of second training operation.

[0200] In this embodiment of the disclosure, before performing the aforementioned first training operation, the student model can be pre-trained, and the trained student model can be used as the model to be trained. Specifically, distillation training can be used to train the student model. Figure 11 As shown, Figure 11 This is a schematic diagram illustrating a process for training a student model, provided as an embodiment of this disclosure. Figure 11 As shown, training the student model can include: S1, preparing training samples; S2, supervising and fine-tuning the teacher model based on the training samples; S3, performing distillation training on the student model based on the teacher model and the training samples. The details are explained below.

[0201] In this embodiment, a teacher model trained using a supervised training method can be obtained. During supervised training of the teacher model, a standard text set corresponding to the training samples can be used as a supervision label, enabling the teacher model to learn to select candidate texts included in the standard text set from the candidate text set based on the question text. For example, for candidate texts in the standard text set, the higher the probability that the candidate text will be selected, the lower the corresponding training loss; conversely, the lower the probability that the candidate text will be selected, the higher the corresponding training loss. Through this supervised training, the teacher model can learn the correlation between the question text and the candidate texts, and can determine the relevance between each candidate text in the candidate text set and the question text based on the question text and the candidate text set. Optionally, the teacher model can be a large language model with a large parameter scale and strong text understanding capabilities.

[0202] Optionally, the training samples used for supervised training of the teacher model can be the same as or different from the training samples used for the first training operation on the model to be trained; this disclosure does not limit this. Even when the training samples are different, the training samples used for supervised training of the teacher model can still include the question text, the candidate text set, and the corresponding standard text set.

[0203] It should be noted that the standard text set in this embodiment can be used in different training stages and play different roles. On the one hand, in the supervised training stage of the teacher model, the standard text set can be used as a supervision label, enabling the teacher model to learn to select candidate texts given a question text and a candidate text set, and to learn the text selection method represented by the standard text set; furthermore, the student model can be trained by distillation using the candidate text relevance output by the teacher model, enabling the student model to acquire the corresponding initial text filtering ability.

[0204] On the other hand, during the first training operation of the model to be trained, the standard text set is also used to determine the difference between the target text set and the standard text set, and to determine the second loss based on this difference. For example, for a target text set of lower quality, corresponding reference selection information can be determined based on the standard text set, and the text selection process of the model to be trained can be corrected according to the difference between the current text selection result of the model to be trained and the reference selection information.

[0205] After obtaining the teacher model, multiple rounds of second training operations can be performed on the student model based on multiple training samples and the teacher model until the preset second training termination condition is met. The student model that meets the second training termination condition is then used as the aforementioned model to be trained. The student model can be a model with a smaller parameter size than the teacher model, so that it can learn the teacher model's ability to judge the relevance of candidate texts through the training information provided by the teacher model.

[0206] In each round of the second training operation, for each training sample, the question text and the candidate text set can be input into the student model and the teacher model, respectively. The student model can determine the third relevance between each candidate text and the question text based on the question text and the candidate text set; the teacher model can determine the fourth relevance between each candidate text and the question text based on the same question text and candidate text set. The third relevance characterizes the student model's current judgment on the relevance between each candidate text and the question text, while the fourth relevance characterizes the teacher model's judgment on the relevance between the corresponding candidate text and the question text.

[0207] For each training sample, the third and fourth relevance scores of each candidate text in the candidate text set can be compared separately, and the third loss for that training sample can be determined based on the difference between the two. When the third and fourth relevance scores are represented by probability distributions, the third loss can be determined based on the difference between the probability distributions output by the student model and the teacher model. For example, the difference between the two probability distributions can be measured using methods such as KL divergence. The greater the difference between the student model and the teacher model's relevance judgments of the candidate texts, the larger the corresponding third loss can be; the closer their judgments are, the smaller the corresponding third loss can be.

[0208] Optionally, the total loss can be determined based on the third loss corresponding to each training sample in the current round of the second training operation, and the model parameters of the student model can be updated according to the total loss. The student model with updated model parameters can then be used as the student model for the next round of the second training operation. By repeatedly executing the above second training operation, the third relevance of each candidate text output by the student model can gradually approach the fourth relevance output by the teacher model, thereby transferring the candidate text relevance judgment ability of the teacher model to the student model.

[0209] The second training termination condition can be set according to actual training needs. For example, the second training termination condition may include one or more of the following: the number of rounds of the second training operation reaches a preset number of rounds, the third loss converges, or the third loss is less than a preset loss threshold. If the second training termination condition is met, the second training operation can be terminated, and the current student model can be used as the model to be trained, so as to further train the model to be trained according to the aforementioned first training operation.

[0210] In this embodiment of the disclosure, the training loss of the teacher model can be expressed as follows:

[0211] in, This represents the training loss of the teacher model; Represents the question text; Indicates a set of candidate texts; The first in the standard text set The text identifier corresponding to each text; Indicates the first The text identifier of a previously defined standard text; Indicates the number of texts included in the standard text set; This represents the probability of candidate text selection output by the teacher model. Using the above loss, a standard text set can be used as a supervisory label to supervise the training of the teacher model, enabling it to learn the text selection methods represented by the standard text set.

[0212] The training loss of the student model can be expressed as follows:

[0213] in, This represents the training loss corresponding to the student model, i.e., the aforementioned third loss; This means that the probability distribution of each candidate text determined by the student model based on the question text and the candidate text set can correspond to the aforementioned third relevance. This indicates that the probability distribution of each candidate text determined by the teacher model based on the same question text and candidate text set can correspond to the aforementioned fourth relevance; This represents the KL divergence, used to characterize the difference between the probability distributions output by the student model and the teacher model.

[0214] By minimizing the aforementioned third loss, the candidate text selection probability distribution output by the student model can gradually approach the candidate text selection probability distribution output by the teacher model. This allows the text selection ability learned by the teacher model based on the standard text set to be transferred to the student model, enabling the student model to have basic text selection ability before the subsequent first training operation, and providing reasonable model parameter initialization for subsequent training.

[0215] Optionally, when both the student model and the teacher model select multiple candidate texts from the candidate text set using a stepwise selection method, the probability distribution of the outputs of the student model and the teacher model for each unselected candidate text can be determined at each text selection position. The training loss of the student model can then be determined based on the difference between the probability distributions of the two models at each text selection position. Specifically, the training loss of the student model can be expressed as:

[0216] in, Indicates the number of candidate texts selected from the target text set; Represents the question text; Indicates a set of candidate texts; Indicates the first Candidate texts that have been selected before the selected text position; The student model is represented in the first... The probability distribution of each candidate text determined by the selected text position; This represents the probability distribution of each candidate text determined by the teacher model under the same selection conditions.

[0217] Optionally, the relevance mentioned above in the embodiments of this disclosure can be the prediction score (logit) of the model output for the candidate text, or it can be the probability value obtained after performing Softmax processing on the prediction score.

[0218] In this embodiment of the disclosure, by calculating the difference between the probability distributions output by the student model and the teacher model at each text selection position, the stepwise text selection process of the student model can be finely distilled, enabling the student model to not only learn which candidate texts should be selected in the end, but also to learn the candidate text selection preferences of the teacher model at each text selection position, thereby improving the basic text screening ability of the student model.

[0219] To better understand and illustrate the model training scheme provided in the embodiments of this disclosure and its beneficial effects, the following is combined with... Figure 12 The solutions provided in the embodiments of this disclosure will be described. Figure 12 This is a schematic flowchart of a model training method provided in an embodiment of the present disclosure.

[0220] The model training method provided in this disclosure may include stages such as offline data generation, supervised training cold start, policy sampling and quality evaluation, and joint reinforcement of generated preferences. Specifically, it may include the following steps.

[0221] S1, Offline data generation.

[0222] It can generate corresponding candidate text sets and standard text sets for the question text used for model training, so as to construct the training data required for subsequent model training.

[0223] For each question text query, one or more search queries can be constructed based on the question text, and a search engine can be invoked to retrieve multiple candidate texts related to the question text from the corresponding data source. For example, 15-30 candidate texts can be retrieved for a single question text to form a candidate text set corresponding to that question text.

[0224] After obtaining the candidate text set, a large language model can be used to perform multiple answer generation operations on the same question text and candidate text set, i.e., perform multiple rollouts. During each answer generation process, the large language model can be prompted to output the text identifiers of the candidate texts referenced when generating the answer. By parsing the answers generated by multiple rollouts, the reference frequency of each candidate text can be statistically analyzed, and a preliminary selection of candidate texts can be obtained based on information such as the reference frequency of the candidate texts.

[0225] The candidate texts can be further processed through a pre-defined offline text selection process. For example, intent analysis can be performed on the question text, and a text evaluation strategy corresponding to the question text can be generated based on the intent analysis results. Based on this text evaluation strategy, the initially screened candidate texts can be cleaned and optimized. For example, evaluations can be made based on relevance, authority, timeliness, information completeness, and content conflict, to filter out candidate texts that are irrelevant to the question text, incorrectly cited, or of low quality. From the remaining candidate texts, the text combination that can better support the answer to the question can be determined, resulting in the standard text set GoldenSet corresponding to the question text.

[0226] Optionally, a standard text set may include 4-7 candidate texts. Thus, corresponding candidate text sets and standard text sets can be generated for multiple question texts, and training samples for subsequent model training can be constructed based on the aforementioned data.

[0227] It should be noted that the aforementioned standard text set can be generated offline before model reinforcement training and can play different roles in different training stages. On the one hand, it can serve as a supervisory label during teacher-supervised model training; on the other hand, it can serve as reference information for low-quality text selection results during subsequent reinforcement training, correcting the text selection strategy of the model to be trained.

[0228] S2. Conduct supervised training cold start After obtaining the above training data, the teacher model can be supervised and trained using a standard text set, and the student model can be further trained by distillation using the teacher model, so that the student model can acquire basic text filtering capabilities in advance.

[0229] Specifically, the question text, candidate text set, and corresponding standard text set can be used for supervised training of the teacher model. The standard text set can serve as a supervisory label, enabling the teacher model to learn, given the question text and candidate text set, to progressively select candidate texts from the standard text set.

[0230] For example, in a step-by-step selection of candidate texts, at the first text selection position, the teacher model can determine the selection probability of each candidate text based on the question text and the candidate text set, and supervise the teacher model based on the first standard text in the standard text set. At subsequent text selection positions, the previously selected standard text can be used as selection criteria to further determine the selection probability of the remaining candidate texts, and supervision can be performed based on the standard text corresponding to each position. Through this supervised training, the teacher model can learn the text selection method represented by the standard text set.

[0231] After obtaining the teacher model, the student model can be trained using distillation. For the same question text and candidate text set, the relevance or selection probability of each candidate text can be determined using both the teacher model and the student model, and the distillation loss can be determined based on the difference between the probability distributions output by the teacher model and the student model.

[0232] Optionally, when progressively filtering candidate texts, the probability distributions of the teacher and student models' outputs can be determined at each selected text position, and the differences between them can be determined using methods such as KL divergence. The training loss of the student model is determined based on the differences corresponding to multiple selected text positions, and the model parameters of the student model are updated based on this training loss. The above training process is repeated until a preset training termination condition is met.

[0233] Therefore, the text selection ability learned by the teacher model based on the standard text set can be transferred to the student model, so that the student already has the basic ability to select candidate texts before entering the subsequent reinforcement training stage. This provides reasonable model parameter initialization for subsequent reinforcement training and avoids the student model starting from a random text selection strategy.

[0234] S3. Perform strategy sampling and target text set quality evaluation. After completing the above cold start training, the trained Student can be used as the model to be trained, and the model to be trained can be used to sample multiple times for each training sample to obtain multiple target text sets.

[0235] For each training sample, the question text and the candidate text set can be input into the model to be trained. The model to be trained will gradually determine the first relevance of each candidate text and select multiple candidate texts from the candidate text set based on the first relevance to form a target text set.

[0236] To obtain multiple target text sets with different text compositions, multiple rollouts can be performed on the same training sample. For example, a higher sampling temperature can be set, and multiple samples can be taken based on the candidate text probability distribution output by the model to be trained, so that candidate texts with relatively low selection probabilities also have a certain possibility of being selected. In this way, the same training sample can yield multiple target text sets composed of different texts, thus forming multiple candidate text selection schemes.

[0237] After obtaining multiple target text sets, the Rubric Judge RM can be used to evaluate the overall quality of each target text set separately. The Rubric Judge RM can provide evaluation services for large language models driven by prompt words, and can perform structured evaluation of the entire target text set according to a preset second evaluation metric, rather than evaluating only individual candidate texts.

[0238] For example, the second evaluation metric could include the degree to which the target text set covers the information requirements of the question text, the degree of information duplication among different candidate texts in the target text set, the degree of matching between the relevance of the candidate texts and their contribution to answer generation, and the proportion of low-relevance candidate texts in the target text set. Sub-scores corresponding to each evaluation dimension can be determined separately, and a weighted average can be used to obtain the comprehensive score for the target text set.

[0239] Therefore, multiple target text sets and their corresponding scores can be obtained for the same training sample, providing a basis for implementing different training methods based on the quality of the target text sets.

[0240] S4. Perform joint reinforcement training on generative preferences.

[0241] After obtaining multiple target text sets and their corresponding scores, the quality of the multiple target text sets can be divided according to a preset first threshold, and different training methods can be used for target text sets of different quality.

[0242] Optionally, the first threshold can be a pre-set fixed threshold, or it can be dynamically determined based on the score values ​​of multiple target text sets corresponding to the same training sample. For example, the average, median, or corresponding quantile of the score values ​​of multiple target text sets can be determined as the first threshold.

[0243] For target text sets with score values ​​greater than or equal to the first threshold, they can be identified as the second text set, i.e. Figure 12 The high-scoring set is shown; for target text sets with scores less than the first threshold, they can be identified as the first text set, i.e. Figure 12 The low-segregate shown.

[0244] For high-scoring sets, Group Relative Policy Optimization (GRPO) can be used for training. Specifically, a baseline score can be determined based on the scores of multiple target text sets corresponding to the same training sample, and the advantage value of each second text set in the high-scoring set can be determined based on the difference between the score of each second text set and the baseline score. For example, the score of each second text set can be standardized based on the average and standard deviation of the scores of multiple target text sets to obtain a normalized advantage value.

[0245] The GRPO training loss can be determined based on the normalized advantage value and the policy probability when the model generates the corresponding second text set. For text selection results that have a high advantage relative to the benchmark score, a corresponding positive training signal can be generated; for text selection results that have a relatively low advantage, a corresponding negative training signal can be generated.

[0246] Since the high-resolution set already represents a relatively high-quality text combination, it is not necessary for this part of the training data to be completely consistent with the standard text set. Instead, the strategy can be optimized by utilizing the relative merits of multiple sampling results, so that the model to be trained can further explore other text combinations that can better support the answer to the question based on the already better text selection results.

[0247] For low-resolution datasets, online policy distillation (OPD) can be performed using a standard text set. Since this portion of the target text set has already been evaluated as a relatively low-quality text selection result, the standard text set can be used to provide more explicit correction information for the model to be trained.

[0248] Specifically, during the process of the model to be trained gradually generating the first text set, for each text selection position, the first probability distribution output by the model to be trained without referring to the standard text set, and the second probability distribution obtained with referring to the standard text set, can be determined, and the difference between the two probability distributions can be determined. For example, KL divergence can be used to determine the difference corresponding to the corresponding text selection position, and the training loss corresponding to the low set can be determined based on the differences corresponding to multiple text selection positions.

[0249] Therefore, when the current selection tendency of the model under test at a certain text selection position deviates significantly from the selection tendency obtained from the reference standard text set, the training signal corresponding to that position can be relatively large, thus providing strong correction for that position; where the two are more consistent, the corresponding training signal can be smaller. Through this text-by-text selection position correction method, the model under test can focus on adjusting positions with large deviations in the text selection process, causing the low-quality text selection strategy to adjust towards the better text selection region represented by the standard text set.

[0250] Furthermore, end-to-end training signals for answer generation can be introduced for each target text set. Specifically, for each target text set sampled by the model to be trained, the target text set and the corresponding question text can be input into the Agent or question-answering model to generate the corresponding question answer; furthermore, the generated answer can be evaluated by a preset first evaluation metric, and the answer loss can be determined based on the evaluation result.

[0251] This answer loss can reflect the actual support capability of the target text set for answering questions from the perspective of the final answer generation effect. In other words, even if a target text set has a high score in the text set quality evaluation, if the quality of the answers generated based on the target text set is low, the corresponding text selection strategy can be adjusted through the training signal at the answer generation level.

[0252] Furthermore, the different training signals mentioned above can be used together to optimize model parameters. For example, a joint optimization objective can be determined based on the training loss of the answer generation level, the GRPO training loss corresponding to the high set, and the OPD training loss corresponding to the low set, and the model parameters of the model to be trained can be updated based on this joint optimization objective.

[0253] Therefore, in the same training process, three mutually complementary training signals can be formed: the answer generation level training signal is used to evaluate the text selection results from the final answer effect; the GRPO training signal is used to strengthen the better text selection method by utilizing the relative advantages and disadvantages among high-quality sampling results; and the OPD training signal is used to perform fine-grained correction of low-quality text selection results using a standard text set.

[0254] By repeatedly executing steps S3 and S4, the text selection strategy of the model to be trained can be continuously updated. After the preset training termination conditions are met, the current model to be trained can be used as the final text selection model.

[0255] In this embodiment of the disclosure, through the model training process shown in S1-S4 above, a high-quality text selection reference can first be established using an offline-generated standard text set, and the student model can acquire basic text selection capabilities through supervised training and teacher-student distillation. On this basis, text combinations with different qualities are obtained through multiple strategy samplings, and the sampling results are stratified according to the quality of the set-level evaluation results. For high-quality sampling results, the relative advantages within the group are used to continue exploring better text combinations, and for low-quality sampling results, the standard text set is used for targeted correction. At the same time, the quality of the actual answers generated by each text combination is combined to form an end-to-end training signal.

[0256] Therefore, it avoids limiting the model's text selection method to predetermined standard answers by relying solely on standard text sets, and also avoids relying entirely on reinforcement learning for unconstrained exploration when the model's text selection ability is weak. This allows the model to obtain appropriate training signals based on sampling results of different qualities, building upon its already superior text selection ability, and ensuring that the optimization direction of the text selection strategy matches the final answer generation effect. This improves the text selection ability of the trained text filtering model for complex questions and the accuracy of the generated answers.

[0257] The above approach uses the answer generation performance of the target text set as a training signal, enabling the text selection model to learn and select candidate texts that are more conducive to generating high-quality answers. Simultaneously, tiered training is performed based on the quality of the target text set, reinforcing the relative advantages of high-quality text sets and correcting low-quality text sets using a standard text set, thus balancing the exploration of better text combinations with the correction of low-quality results. Furthermore, the standard text set can be used for both supervised training and subsequent correction, improving the efficiency of labeled data utilization. A holistic evaluation of the target text set can comprehensively consider set attributes such as information complementarity and content repetition among texts. Structured evaluation using a prompt-word-driven large language model allows for flexible adjustment of evaluation indicators and rules according to application needs, improving the applicability of the evaluation method.

[0258] like Figure 13 As shown in the embodiments of this disclosure, a question-and-answer method is also provided, which includes S401-S403.

[0259] S401. Obtain the question text and a set of candidate texts, which include multiple candidate texts. These candidate texts are candidate reference texts for generating the answer to the question text.

[0260] In this embodiment of the disclosure, the question text input by the user through a terminal can be obtained, and a search engine can be invoked based on the question text to retrieve multiple candidate texts related to the question text from data sources such as the Internet, knowledge bases, and business databases, thereby forming a candidate text set. Each candidate text can serve as a candidate reference text for generating the corresponding answer to the question text.

[0261] S402. Input the question text and the candidate text set into the text filtering model. The text filtering model filters out multiple candidate texts from the candidate text set based on the relevance of each candidate text in the candidate text set to the question text, and obtains the target text set. The text filtering model is trained by any of the above model training methods.

[0262] The question text and a set of candidate texts can be input into a text filtering model. The text filtering model determines the relevance between each candidate text in the candidate text set and the question text, and filters out multiple candidate texts from the candidate text set based on the relevance to obtain the target text set. This text filtering model can be trained using the model training method provided in any of the foregoing embodiments.

[0263] Because the text filtering model takes into account the support effect of the filtered text on answer generation during the training process, the target text set obtained by the text filtering model can provide relevant reference information for subsequent answer generation.

[0264] S403. Input the target text set and the question text into the question-answering model, and generate the answer to the question text through the question-answering model.

[0265] In this embodiment of the disclosure, a target text set and a question text can be input into a question-answering model. The target text set is used as reference information when generating the answer. The question-answering model generates the answer corresponding to the question text and returns the generated answer to the user terminal.

[0266] In this embodiment of the disclosure, the text filtering model trained based on the aforementioned model training scheme can filter out candidate texts that are more suitable for supporting answer generation from multiple retrieved candidate texts, and generate answers based on the filtered target text set, thereby improving the accuracy of answer generation in question-and-answer scenarios.

[0267] The virtual device in the embodiments of this disclosure will now be described.

[0268] This disclosure provides a model training apparatus, such as... Figure 14 As shown, the model training device 60 may include: an acquisition module 601 and a training module 602, wherein, The acquisition module 601 is used to acquire multiple training samples, each training sample including a question text and a candidate text set, wherein the candidate text set includes multiple candidate texts, and the candidate texts are candidate reference texts for generating the answer to the question text; Training module 602 is used to perform multiple rounds of first training operations on the model to be trained using the multiple training samples until a preset first training termination condition is met, and to use the model to be trained that meets the first training termination condition as a text filtering model. The training module 602 includes: The filtering unit is used to filter multiple target texts from the candidate text set based on the first relevance between each candidate text in the training sample and the question text in each training sample, thereby obtaining a target text set. An evaluation unit is used to input the target text set and the question text into the question answering model for each training sample, generate an answer to the question text through the question answering model, evaluate the answer based on a preset first evaluation index, and determine the training loss of the training sample based on the evaluation result of the answer. The update unit is used to update the model parameters of the model to be trained based on the training loss of each training sample, so as to obtain the model to be trained for the next round of training.

[0269] As an optional embodiment, for each question text in the training sample, the target text set corresponding to the question text is obtained by the filtering unit in the following way: The first relevance between each candidate text in the candidate text set and the question text is determined using the model to be trained. Based on the first relevance between each candidate text and the question text, the sampling probability of each candidate text is obtained. Based on the preset number of single samplings and the sampling probability of each candidate text, the candidate text set is sampled N times, and the target text obtained from each sampling is taken as a target text set, where N is an integer greater than or equal to 1.

[0270] As an optional embodiment, N is an integer greater than or equal to 2; For each training sample, the answer to the question text, the evaluation result of the answer, and the training loss in the training sample are obtained by the evaluation unit in the following way: For each target text set corresponding to the training samples, the target text set and the question text are input into the question answering model. The question answering model generates an answer based on the question text of the target text set. The answer is evaluated based on a first evaluation index to obtain the evaluation result of the answer. The training loss of the training samples is determined based on the evaluation results of the answers to the question texts corresponding to each target text set.

[0271] As an optional embodiment, the training samples also include a standard text set of the question text, the standard text set including multiple standard texts, each of which is a standard reference text for generating the answer to the question text; For each training sample, when determining the training loss of the training sample based on the evaluation result of the answer, the evaluation unit is specifically used for: The first loss is determined based on the evaluation results of the answers corresponding to the training samples; The second loss is determined based on the difference between the target text set and the standard text set of the training samples; The training loss of the training samples is determined based on the first loss and the second loss.

[0272] As an optional embodiment, the number of target text sets corresponding to each training sample is multiple, and for each training sample, the evaluation unit is further used to: Based on a preset second evaluation index, the quality of each target text set corresponding to the training samples is evaluated to obtain a score value for each target text set. For each target text set, if the score of the target text set is less than a first threshold, the target text set is designated as the first text set; if the score of the target text set is greater than or equal to the first threshold, the target text set is designated as the second text set. For each of the second text sets, the advantage value of the second text set is determined based on the score value of the second text set and the benchmark score value, and the training loss of the second text set is determined based on the advantage value. For each training sample, the evaluation unit, when determining the second loss based on the difference between the target text set and the standard text set of the training sample, specifically uses: For each first text set corresponding to the training samples, the training loss corresponding to the first text set is determined based on the difference between the first text set and the standard text set. The second loss of the training sample is determined based on the training loss of each first text set and the training loss of each second text set corresponding to the training sample.

[0273] As an optional embodiment, when determining the second loss based on the difference between the target text set and the standard text set of the training samples, the evaluation unit is specifically used for: The question text, the candidate text set, and the standard text set are input into the model to be trained. The model to be trained then determines the second relevance between each candidate text in the candidate text set and the question text by referring to the standard text set. Determine the difference between the first relevance and the second relevance corresponding to each target text in the target text set, wherein the difference represents the difference between the target text set and the standard text set; The second loss of the training samples is determined based on the differences between the target text sets corresponding to the training samples.

[0274] As an optional embodiment, for each training sample, when the evaluation unit evaluates the quality of each target text set corresponding to the training sample based on a preset second evaluation index to obtain a score value for each target text set, it is specifically used for: Obtain the first prompt word, which is used to prompt the large language model to evaluate the target text set input to the large language model based on the second evaluation metric; The first prompt word, the question text, and the target text set are input into a large language model to obtain the evaluation result of the target text set output by the large language model; wherein, the second evaluation index includes at least one of the following: The extent to which all target texts in the target text set cover the information of the question text; The degree of information redundancy among the target texts in the target text set; The degree of matching between the relevance and contribution of each target text in the target text set, wherein the relevance is the relevance between the target text and the question text, and the contribution is the contribution of the target text set to the generation of the answer to the question text; The percentage of target texts in the target text set whose relevance to the problem text is lower than a preset value.

[0275] As an optional embodiment, the model to be trained is obtained in the following way: Obtain the teacher model trained using a supervised training method; Based on the multiple training samples and the teacher model, the student model undergoes multiple rounds of second training operations until a preset second training termination condition is met. The student model that meets the second training termination condition is then used as the model to be trained. Each round of second training operations includes: For each training sample, the third relevance between each candidate text in the candidate text set of the training sample and the question text is determined by the student model, and the fourth relevance between each candidate text in the candidate text set of the training sample and the question text is determined by the teacher model. For each training sample, the third loss of the training sample is determined based on the difference between the third and fourth relevance of each candidate text in the candidate text set of the training sample. Based on the third loss of each training sample, the model parameters of the student model are updated to obtain the student model for the next round of second training operation.

[0276] As an optional embodiment, the model training apparatus further includes a preprocessing module for: Obtain a preset second prompt word, which is used to prompt the large language model to output the answer to the question text and the text identifier of the candidate text referenced when generating the answer, based on the input question text and candidate text set of the large language model; The question text of the training samples and the candidate text set are input into the large language model. The large language model performs multiple answer generation operations to obtain multiple answers output by the large language model and the text identifier of the candidate text referenced in the generation of each answer. The text identifiers of the candidate texts referenced when generating each answer are statistically analyzed, and a preset number of candidate texts are selected from the referenced candidate texts in descending order of reference frequency. Based on the predetermined number of candidate texts, the standard text set is obtained.

[0277] As an optional embodiment, when the preprocessing module obtains the standard text set based on the determined preset number of candidate texts, it is specifically used for: The intent analysis of the question text is performed using a large language model, and a text evaluation strategy for the question text is generated based on the analysis results. Based on the preset number of candidate texts, multiple subsets of candidate texts are obtained; Based on the text evaluation strategy, each candidate text subset is evaluated, and the standard text set is determined from the multiple candidate text subsets according to the evaluation results.

[0278] This disclosure also provides a question-and-answer device, such as... Figure 15 As shown, the question-and-answer device 70 may include: an acquisition module 701, a filtering module 702, and a generation module 703, wherein, The acquisition module 701 is used to acquire the question text and the candidate text set, wherein the candidate text set includes multiple candidate texts, and the candidate texts are candidate reference texts for generating the answer to the question text; The filtering module 702 is used to input the question text and the candidate text set into the text filtering model, and the text filtering model filters out multiple candidate texts from the candidate text set based on the relevance of each candidate text in the candidate text set to the question text, thereby obtaining the target text set; the text filtering model is trained by any of the model training methods described above. The generation module 703 is used to input the target text set and the question text into the question-answering model, and generate the answer to the question text through the question-answering model.

[0279] The apparatus of this disclosure embodiment can execute the method provided in this disclosure embodiment, and its implementation principle is similar, and it has corresponding technical effects. The actions performed by each module in the apparatus of each embodiment of this disclosure correspond to the steps in the method of each embodiment of this disclosure. For a detailed functional description of each module of the apparatus, please refer to the description in the corresponding method shown above, and it will not be repeated here.

[0280] This disclosure provides an electronic device, including a memory, a processor, and a computer program stored in the memory. The processor executes the computer program to implement the steps of the method provided in any optional embodiment of this disclosure. Compared with the prior art, it can achieve the following: In the process of training a text filtering model, by using the filtered target text set for generating question answers and evaluating the generated answers based on preset evaluation metrics, the training loss is determined according to the evaluation results of the answers to iteratively train the text filtering model. This achieves optimization of the candidate text filtering process based on the final answer generation effect. Compared with determining the model optimization direction by evaluating the correlation between the text in the target text set and the question text, this allows the optimization direction of the text filtering model to match the final answer generation effect. In practical applications, the text filtering model trained by this disclosure can filter out reference texts that are more conducive to generating accurate answers from multiple candidate texts, which is more in line with the application needs of actual question-and-answer scenarios, thereby improving the accuracy of answer generation in complex question scenarios.

[0281] In one alternative embodiment, an electronic device is provided, such as Figure 16 As shown, Figure 16 The illustrated electronic device 4000 includes a processor 4001 and a memory 4003. The processor 4001 and the memory 4003 are connected, for example, via a bus 4002. Optionally, the electronic device 4000 may further include a transceiver 4004, which can be used for data interaction between the electronic device and other electronic devices, such as sending and / or receiving data. It should be noted that in practical applications, the transceiver 4004 is not limited to one type, and the structure of the electronic device 4000 does not constitute a limitation on the embodiments of this disclosure.

[0282] Processor 4001 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It may implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with this disclosure. Processor 4001 may also be a combination that implements computational functions, such as including one or more microprocessor combinations, a combination of a DSP and a microprocessor, etc.

[0283] Bus 4002 may include a pathway for transmitting information between the aforementioned components. Bus 4002 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. Bus 4002 can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 16 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0284] The memory 4003 may be ROM (Read Only Memory) or other types of static storage devices capable of storing static information and instructions, RAM (Random Access Memory) or other types of dynamic storage devices capable of storing information and instructions, or EEPROM (Electrically Erasable Programmable Read Only Memory), CD-ROM (Compact Disc Read Only Memory) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media, other magnetic storage devices, or any other medium capable of carrying or storing computer programs and capable of being read by a computer, without limitation herein.

[0285] The memory 4003 is used to store computer programs that execute embodiments of the present disclosure, and is controlled by the processor 4001 to execute them. The processor 4001 is used to execute the computer programs stored in the memory 4003 to implement the steps shown in the foregoing method embodiments.

[0286] This disclosure provides a computer-readable storage medium storing a computer program, which, when executed by a processor, can implement the steps and corresponding content of the aforementioned method embodiments.

[0287] This disclosure also provides a computer program product, including a computer program that, when executed by a processor, can implement the steps and corresponding content of the aforementioned method embodiments.

[0288] It should be understood that although arrows indicate various operation steps in the flowcharts of the embodiments of this disclosure, the order in which these steps are implemented is not limited to the order indicated by the arrows. Unless explicitly stated herein, in some implementation scenarios of the embodiments of this disclosure, the implementation steps in each flowchart can be executed in other orders as required. Furthermore, some or all of the steps in each flowchart may include multiple sub-steps or multiple stages based on the actual implementation scenario. Some or all of these sub-steps or stages can be executed at the same time, and each sub-step or stage can also be executed at different times. In scenarios where execution times differ, the execution order of these sub-steps or stages can be flexibly configured as required, and the embodiments of this disclosure do not limit this.

[0289] The above description is only an optional implementation method for some implementation scenarios of this disclosure. It should be noted that for those skilled in the art, other similar implementation methods based on the technical concept of this disclosure without departing from the technical concept of this disclosure also fall within the protection scope of the embodiments of this disclosure.

Claims

1. A model training method, characterized in that, The method includes: Multiple training samples are obtained. Each training sample includes a question text and a candidate text set. The candidate text set includes multiple candidate texts, which are candidate reference texts for generating the answer to the question text. Using the multiple training samples, the model to be trained undergoes multiple rounds of first training operations until a preset first training termination condition is met. The model to be trained that meets the first training termination condition is then used as the text filtering model. Each round of first training operations includes: For each question text in a training sample, the model to be trained selects multiple target texts from the candidate text set based on the first relevance between each candidate text in the training sample and the question text, thus obtaining a target text set. For each training sample, the target text set and the question text are input into the question answering model, the question answering model generates the answer to the question text, the answer is evaluated based on a preset first evaluation metric, and the training loss of the training sample is determined based on the evaluation result of the answer. Based on the training loss of each training sample, the model parameters of the model to be trained are updated to obtain the model to be trained for the next round of training.

2. The method according to claim 1, characterized in that, For each training sample's question text, the corresponding target text set is obtained in the following way: The first relevance between each candidate text in the candidate text set and the question text is determined using the model to be trained. Based on the first relevance between each candidate text and the question text, the sampling probability of each candidate text is obtained. Based on the preset number of single samplings and the sampling probability of each candidate text, the candidate text set is sampled N times, and the target text obtained from each sampling is taken as a target text set, where N is an integer greater than or equal to 1.

3. The method according to claim 2, characterized in that, N is an integer greater than or equal to 2; For each training sample, the answer to the question text, the evaluation result of the answer, and the training loss are obtained in the following way: For each target text set corresponding to the training samples, the target text set and the question text are input into the question answering model. The question answering model generates an answer based on the question text of the target text set. The answer is evaluated based on a first evaluation index to obtain the evaluation result of the answer. The training loss of the training samples is determined based on the evaluation results of the answers to the question texts corresponding to each target text set.

4. The method according to claim 1, characterized in that, The training samples also include a standard text set for the question text, which includes multiple standard texts, each of which is a standard reference text for generating the answer to the question text. For each training sample, determining the training loss of the training sample based on the evaluation result of the answer includes: The first loss is determined based on the evaluation results of the answers corresponding to the training samples; The second loss is determined based on the difference between the target text set and the standard text set of the training samples; The training loss of the training samples is determined based on the first loss and the second loss.

5. The method according to claim 4, characterized in that, The number of target text sets corresponding to each training sample is multiple, and for each training sample, the method further includes: Based on a preset second evaluation index, the quality of each target text set corresponding to the training samples is evaluated to obtain a score value for each target text set. For each target text set, if the score of the target text set is less than a first threshold, the target text set is designated as the first text set; if the score of the target text set is greater than or equal to the first threshold, the target text set is designated as the second text set. For each of the second text sets, the advantage value of the second text set is determined based on the score value of the second text set and the benchmark score value, and the training loss of the second text set is determined based on the advantage value. For each training sample, determining the second loss based on the difference between the target text set and the standard text set of the training sample includes: For each first text set corresponding to the training samples, the training loss corresponding to the first text set is determined based on the difference between the first text set and the standard text set. The second loss of the training sample is determined based on the training loss of each first text set and the training loss of each second text set corresponding to the training sample.

6. The method according to claim 4 or 5, characterized in that, The step of determining the second loss based on the difference between the target text set and the standard text set of the training samples includes: The question text, the candidate text set, and the standard text set are input into the model to be trained. The model to be trained then determines the second relevance between each candidate text in the candidate text set and the question text by referring to the standard text set. Determine the difference between the first relevance and the second relevance corresponding to each target text in the target text set, wherein the difference represents the difference between the target text set and the standard text set; The second loss of the training samples is determined based on the differences between the target text sets corresponding to the training samples.

7. The method according to claim 5, characterized in that, For each training sample, the quality of each target text set corresponding to the training sample is evaluated based on a preset second evaluation metric to obtain a score value for each target text set, including: Obtain the first prompt word, which is used to prompt the large language model to evaluate the target text set input to the large language model based on the second evaluation metric; The first prompt word, the question text, and the target text set are input into a large language model to obtain the evaluation result of the target text set output by the large language model; wherein, the second evaluation index includes at least one of the following: The extent to which all target texts in the target text set cover the information of the question text; The degree of information redundancy among the target texts in the target text set; The degree of matching between the relevance and contribution of each target text in the target text set, wherein the relevance is the relevance between the target text and the question text, and the contribution is the contribution of the target text set to the generation of the answer to the question text; The percentage of target texts in the target text set whose relevance to the problem text is lower than a preset value.

8. The method according to claim 1, characterized in that, The model to be trained was obtained in the following way: Obtain the teacher model trained using a supervised training method; Based on the multiple training samples and the teacher model, the student model undergoes multiple rounds of second training operations until the preset second training termination condition is met. The student model that meets the second training termination condition is then used as the model to be trained. The second training session in each round includes: For each training sample, the third relevance between each candidate text in the candidate text set of the training sample and the question text is determined by the student model, and the fourth relevance between each candidate text in the candidate text set of the training sample and the question text is determined by the teacher model. For each training sample, the third loss of the training sample is determined based on the difference between the third and fourth relevance of each candidate text in the candidate text set of the training sample. Based on the third loss of each training sample, the model parameters of the student model are updated to obtain the student model for the next round of second training operation.

9. The method according to claim 4, characterized in that, The standard text set for each training sample is obtained in the following way: Obtain a preset second prompt word, which is used to prompt the large language model to output the answer to the question text and the text identifier of the candidate text referenced when generating the answer, based on the input question text and candidate text set of the large language model; The question text of the training samples and the candidate text set are input into the large language model. The large language model performs multiple answer generation operations to obtain multiple answers output by the large language model and the text identifier of the candidate text referenced in the generation of each answer. The text identifiers of the candidate texts referenced when generating each answer are statistically analyzed, and a preset number of candidate texts are selected from the referenced candidate texts in descending order of reference frequency. Based on the predetermined number of candidate texts, the standard text set is obtained.

10. The method according to claim 9, characterized in that, The standard text set is obtained based on the predetermined number of candidate texts, including: The intent analysis of the question text is performed using a large language model, and a text evaluation strategy for the question text is generated based on the analysis results. Based on the preset number of candidate texts, multiple subsets of candidate texts are obtained; Based on the text evaluation strategy, each candidate text subset is evaluated, and the standard text set is determined from the multiple candidate text subsets according to the evaluation results.

11. A question-and-answer method, characterized in that, The method includes: Obtain the question text and a candidate text set, wherein the candidate text set includes multiple candidate texts, and the candidate texts are candidate reference texts for generating the answer to the question text; The question text and the candidate text set are input into a text filtering model. The text filtering model filters out multiple candidate texts from the candidate text set based on the relevance of each candidate text in the candidate text set to the question text, thereby obtaining the target text set. The text filtering model is trained by the method described in any one of claims 1-10. The target text set and the question text are input into the question-answering model, and the question-answering model generates the answer to the question text.

12. A model training device, characterized in that, The device includes: The acquisition module is used to acquire multiple training samples. Each training sample includes a question text and a candidate text set. The candidate text set includes multiple candidate texts, which are candidate reference texts for generating the answer to the question text. The training module is used to perform multiple rounds of first training operations on the model to be trained using the multiple training samples until a preset first training termination condition is met, and the model to be trained that meets the first training termination condition is used as the text filtering model. The training module includes: The filtering unit is used to filter multiple target texts from the candidate text set based on the first relevance between each candidate text in the training sample and the question text in each training sample, thereby obtaining a target text set. An evaluation unit is used to input the target text set and the question text into the question answering model for each training sample, generate an answer to the question text through the question answering model, evaluate the answer based on a preset first evaluation index, and determine the training loss of the training sample based on the evaluation result of the answer. The update unit is used to update the model parameters of the model to be trained based on the training loss of each training sample, so as to obtain the model to be trained for the next round of training.

13. A question-and-answer device, characterized in that, The device includes: The acquisition module is used to acquire the question text and the candidate text set, wherein the candidate text set includes multiple candidate texts, and the candidate texts are candidate reference texts for generating the answer to the question text; A filtering module is used to input the question text and the candidate text set into a text filtering model, and the text filtering model filters out multiple candidate texts from the candidate text set based on the relevance of each candidate text in the candidate text set to the question text, thereby obtaining a target text set; the text filtering model is trained by the method described in any one of claims 1-10; The generation module is used to input the target text set and the question text into the question-answering model, and generate the answer to the question text through the question-answering model.

14. An electronic device comprising a memory, a processor, and a computer program stored in the memory, characterized in that, The processor executes the computer program to implement the method according to any one of claims 1-11.

15. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method described in any one of claims 1-11.

16. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method described in any one of claims 1-11.