Method for obtaining training data of answer generation model and training method of answer generation model

Abstract

The application discloses an answer generation model training data acquisition method and an answer generation model training method, and relates to the technical field of computers. The answer generation model training data acquisition method comprises the following steps: acquiring a plurality of sample questions; acquiring an answer corresponding to a first sample question by using a first inference model, wherein the first sample question is one of the plurality of sample questions; determining the complexity identification of the first sample question based on the answer corresponding to the first sample question and a preset reference answer; determining a sample inference chain and a sample answer corresponding to the first sample question based on the complexity identification of the first sample question; and determining first sample data based on the first sample question, the complexity identification of the first sample question, the sample inference chain and the sample answer, and stopping the operation until sample data is acquired based on all sample questions. According to the application, training data with different inference chain lengths is determined based on the complexity of the questions.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a method for obtaining training data for an answer generation model and a method for training the answer generation model. Background Technology

[0002] With the rapid development of artificial intelligence technology, inference models have demonstrated outstanding performance in complex reasoning tasks such as natural language understanding, text generation, mathematical logic, and code generation. Inference models decompose complex problems through structured text sequences (also known as inference chains) using step-by-step reasoning and reflective verification, thus fulfilling the reasoning requirements of complex tasks. However, inference models rely on long sequence processing, and their autoregressive decoding characteristics cause computational costs to increase quadratically with the length of the inference chain, leading to a bottleneck caused by long inference chain dependency. On the one hand, long inference chain dependency can lead to overthinking of problems; the inference model lacks adaptive complexity judgment, generating redundant inference chains even for simple problems, wasting computational resources and affecting inference efficiency. On the other hand, the high computing power and storage requirements brought about by long inference chain dependency result in excessively high deployment costs, making it difficult to apply in resource-constrained scenarios such as edge devices. Therefore, how to save computational resources, improve inference efficiency, and reduce deployment costs while ensuring the inference performance of inference models is a current focus. Summary of the Invention

[0003] This application provides a method for obtaining training data for an answer generation model and a method for training the answer generation model, in order to at least solve problems such as waste of computing resources, low inference efficiency, and high deployment costs.

[0004] This application provides a method for obtaining training data for an answer generation model, the method comprising: The problem of obtaining multiple samples; Using the first inference model, obtain the answer corresponding to the first sample question, where the first sample question is one of multiple sample questions; Based on the answer corresponding to the first sample question and the preset reference answer, the complexity identifier of the first sample question is determined, wherein the complexity identifier is used to characterize the complexity of the first sample question; Based on the complexity identifier of the first sample problem, determine the sample inference chain and sample answer corresponding to the first sample problem; Based on the first sample question, its complexity identifier, the sample inference chain, and the sample answer, the first sample data is determined. The operation continues until sample data is obtained for each of the sample questions. All sample data is used to train the pre-built answer generation model to obtain the final answer generation model.

[0005] This application provides a method for training an answer generation model, the method comprising: Obtain training and validation sets, wherein the sample data in the training and validation sets are obtained based on the method for obtaining training data for the first-party answer generation model; In the current training round, the pre-built answer generation model is trained based on the training set to obtain the trained answer generation model; Based on the validation set, determine the accuracy and first average inference chain length of the trained answer generation model; Based on the validation set, determine the second average inference chain length of the answer generation model; If the preset iteration stopping condition is not met based on accuracy, first average inference chain length, and second average inference chain length, then proceed to the next training round until the preset iteration stopping condition is met based on accuracy, first average inference chain length, and second average inference chain length, at which point the final answer generation model is obtained.

[0006] This application also provides a device for acquiring training data for an answer generation model, including: The first acquisition module is used to acquire multiple sample questions; The second acquisition module is used to obtain the answer corresponding to the first sample question using the first reasoning model, wherein the first sample question is one of multiple sample questions; The first determining module is used to determine the complexity identifier of the first sample question based on the answer corresponding to the first sample question and the preset reference answer, wherein the complexity identifier is used to characterize the complexity of the first sample question; The second determination module is used to determine the sample reasoning chain and sample answer corresponding to the first sample question based on the complexity identifier of the first sample question. The third determination module is used to determine the first sample data based on the first sample question, the complexity identifier of the first sample question, the sample inference chain, and the sample answer. The operation stops after obtaining sample data based on all sample questions. All sample data are used to train the pre-built answer generation model to obtain the final answer generation model.

[0007] This application also provides a training device for an answer generation model, comprising: The third acquisition module is used to acquire the training set and the validation set. The sample data in the training set and the validation set are obtained based on the acquisition method of the answer generation model training data in the first aspect. The training module is used to train the pre-built answer generation model based on the training set in the current training round to obtain the trained answer generation model; The validation module is used to determine the accuracy and first average inference chain length of the trained answer generation model based on the validation set. The fourth determination module is used to determine the second average inference chain length of the answer generation model based on the validation set. The fifth determination module is used to proceed to the next training round if the preset iteration stopping condition is not met based on the accuracy, the first average inference chain length, and the second average inference chain length, until the preset iteration stopping condition is met based on the accuracy, the first average inference chain length, and the second average inference chain length, and then obtain the final answer generation model.

[0008] This application also provides an electronic device, comprising: a memory for storing a computer program; and a processor for executing the computer program to implement the steps of acquiring training data for any of the above-described answer generation models, or to implement the steps of training any of the above-described answer generation models.

[0009] This application also provides a computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor, it implements the steps of the method for obtaining training data for any of the above-described answer generation models, or the steps of the method for training any of the above-described answer generation models.

[0010] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of acquiring training data for any of the above-described answer generation models, or the steps of training a method for any of the above-described answer generation models.

[0011] This application achieves accurate identification of the complexity of sample questions by comparing the answers output by the first inference model with preset reference answers. Furthermore, it trains an answer generation model using sample data containing sample questions, complexity indicators, sample inference chains, and sample answers. This results in a final answer generation model that possesses both the strong inference capabilities of the inference model and solves the problem of long inference chain dependence. Specifically, it can dynamically adjust the length of the inference chain according to the complexity of the question. While ensuring accurate inference capabilities, it avoids overthinking of simple questions, reduces redundant calculations, lowers memory usage and latency during the inference process, improves inference efficiency, and enables lightweight deployment of the final answer generation model in resource-constrained scenarios. Attached Figure Description

[0012] To more clearly illustrate the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0013] Figure 1A flowchart illustrating a method for obtaining training data for an answer generation model, provided in an embodiment of this application; Figure 2 A flowchart illustrating a training method for an answer generation model provided in an embodiment of this application; Figure 3 This is a schematic diagram of the structure of a device for acquiring training data for an answer generation model, provided in an embodiment of this application. Figure 4 A schematic diagram of the structure of a training device for an answer generation model provided in an embodiment of this application; Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0014] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the protection scope of this application.

[0015] It should be noted that, in the description of this application, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. The terms "first," "second," etc., in this application are used to distinguish similar objects and are not used to describe a specific order or sequence.

[0016] To enable those skilled in the art to better understand the present application, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0017] First, the application scenarios of the embodiments of this application will be introduced by way of example.

[0018] With the rapid development of artificial intelligence technology, inference models, such as Large Language Models (LLMs), have achieved breakthroughs in many fields, including natural language processing and computer vision, demonstrating outstanding performance, especially in complex reasoning tasks requiring deep logical analysis. Inference models decompose complex problems through structured text sequences (also known as reasoning chains or thought chains) using step-by-step reasoning and multi-solution path switching verification, efficiently solving tasks that traditional models struggle with, such as mathematical logic calculations, code generation and debugging, and complex decision analysis.

[0019] However, the reasoning ability of inference models is highly dependent on the generation and processing of long text sequences. Their inherent autoregressive decoding characteristics lead to significant performance bottlenecks, namely, the computational cost of the inference process increases quadratically with the length of the inference chain. When an inference model generates long inference chains, it not only consumes more memory to store intermediate computation results but also significantly increases inference latency. This dependence on long inference chains presents two key challenges in the actual deployment of inference models.

[0020] On the one hand, inference models in related technologies generally suffer from overthinking. For example, for simple calculation problems like "1+1=?" that do not require complex logical decomposition, the inference model still generates a multi-step inference chain similar to "analyzing the problem type - determining the core operation rules - performing basic calculations - verifying the rationality of the result," resulting in a large amount of invalid information in the output, wasting computing resources, and reducing inference efficiency. On the other hand, the high computational overhead brought by long inference chains makes the deployment cost of inference models too high. In resource-constrained scenarios such as edge computing devices and mobile terminals, limited computing power and storage resources cannot support the generation and processing of long inference chains, making it difficult to achieve lightweight deployment of large language models with strong inference capabilities. Therefore, how to solve the problems of overthinking and high deployment costs caused by reliance on long inference chains while ensuring the inference performance of large language models is a current focus.

[0021] In view of this, embodiments of this application provide a method for obtaining training data for an answer generation model and a method for training the answer generation model, in order to solve the problems of wasted computing resources, low inference efficiency, and high deployment costs in the application of the above-mentioned reasoning model.

[0022] It should be noted that the methods for acquiring training data and training the answer generation model provided in the embodiments of the present invention can be executed by devices for acquiring training data and devices for training the answer generation model. These devices can be implemented as part or all of an electronic device through software, hardware, or a combination of both. The electronic device can be a server or a terminal. In this embodiment, the server can be a single server or a server cluster composed of multiple servers. The terminal in this embodiment can be a smartphone, personal computer, tablet computer, wearable device, or other intelligent hardware device such as an intelligent robot. The following method embodiments will use an electronic device as an example for explanation.

[0023] According to an embodiment of the present invention, an embodiment of a method for obtaining training data for an answer generation model is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0024] Figure 1 This is a flowchart illustrating a method for obtaining training data for an answer generation model according to an embodiment of the present invention. Figure 1 As shown, the process includes: S101, the problem of obtaining multiple samples.

[0025] For example, multiple sample questions can be generated through manual annotation or automated tools to serve as training data for building an answer generation model. It is understood that the multiple sample questions need to be as broad as possible to cover different levels of complexity. The data type of the sample questions can be plain text, or it can be multimodal, such as images or videos; this application embodiment does not limit this. Taking plain text as an example, sample questions can include social science, security, and dialogue-based questions. Taking multimodal questions as an example, sample questions can include image descriptions, visual question answering, and chart parsing. Taking server maintenance and repair issues in the server domain as an example, a plain text sample question could be "What are the basic troubleshooting steps for a server that is unresponsive upon startup?" For example, a multimodal sample question could be "Based on the server hardware panel diagram, identify three hardware interfaces in the diagram that can be used to troubleshoot a server that is unresponsive upon startup." Correspondingly, the inference chain corresponding to the sample questions can be used to train the answer generation model to learn the problem analysis logic in server maintenance scenarios. The answers to the sample questions are used to train the answer generation model to output standardized and accurate operation and maintenance solutions, so that the answers generated by the final answer generation model can be used to accurately and reliably guide operation and maintenance work in actual use.

[0026] S102, using the first reasoning model, obtain the answer corresponding to the first sample question.

[0027] The first sample problem is one of the multiple sample problems.

[0028] Specifically, the first reasoning model is the basic model used to initially generate answers to sample questions, providing a basis for determining the complexity of sample questions.

[0029] It is understood that the first inference model is an inference model with the ability to form short inference chains. For example, the temperature sampling parameter of the first inference model is set to 0.6, and the top-p sampling parameter is set to 0.95.

[0030] S103, Based on the answer corresponding to the first sample question and the preset reference answer, determine the complexity identifier of the first sample question.

[0031] The complexity identifier is used to characterize the complexity of the first sample problem.

[0032] Specifically, the preset reference answer is a standard answer pre-set for each sample question, used to verify the accuracy of the answer output by the first inference model.

[0033] The complexity identifier for the first sample problem is a label used to characterize the complexity of the sample problem. For example, the complexity identifier can be a label such as simple or complex, or it can be a numerical value that quantifies the complexity.

[0034] S104. Based on the complexity identifier of the first sample problem, determine the sample reasoning chain and sample answer corresponding to the first sample problem.

[0035] Specifically, the sample reasoning chain for the first sample problem is a reasoning process optimized by combining problem complexity, which can avoid the problem of overthinking mentioned above.

[0036] For example, if the first sample problem is a simple problem, the steps of the sample reasoning chain are simplified; if the first sample problem is a complex problem, the sample reasoning chain contains complete complex reasoning logic.

[0037] S105. Based on the first sample question, as well as the complexity identifier of the first sample question, the sample inference chain, and the sample answer, determine the first sample data. Continue until the sample data is obtained based on all sample questions, and then stop the operation. All sample data are used to train the pre-built answer generation model to obtain the final answer generation model.

[0038] Specifically, the first sample data is a complete training data unit for the first sample question, which includes the question, complexity label, sample inference chain, and sample answer.

[0039] For example, the pre-built answer generation model can be a lightweight large language model to be trained. The final answer generation model is a model obtained after training on sample data, which has the ability to generate complete reasoning chains for complex questions and simplify reasoning steps for simple questions.

[0040] In this embodiment, the complexity of the sample problem is accurately identified by comparing the answer output by the first inference model with the preset reference answer. Furthermore, the answer generation model is trained using sample data containing sample problems, complexity indicators, sample inference chains, and sample answers. This enables the final answer generation model to not only have the strong inference capabilities of the inference model but also solve the problem of long inference chain dependence. That is, it can dynamically adjust the length of the inference chain according to the complexity of the problem. While ensuring accurate inference capabilities, it avoids overthinking of simple problems, reduces redundant calculations, reduces memory consumption and latency in the inference process, improves inference efficiency, and realizes lightweight deployment of the final answer generation model in resource-constrained scenarios.

[0041] In some embodiments, based on the foregoing embodiments, the answers corresponding to the first sample question include multiple answers. This is because, considering that a single answer may be accidental, using the first reasoning model to obtain multiple answers to the first sample question allows multiple answers to more comprehensively and accurately reflect the complexity of the first sample question, thereby improving the accuracy of complexity identification.

[0042] For example, the multiple answers corresponding to the first sample question are obtained by inputting the first sample question into the first inference model in each of the preset second loops, with each loop corresponding to an independent answer.

[0043] In one possible scenario, in the above 103, the complexity identifier of the first sample problem is determined as follows: When at least one of the answers matches the preset reference answer, the complexity identifier of the first sample problem indicates that the first sample problem is a simple problem.

[0044] Alternatively, when all answers do not match the preset reference answer, the complexity identifier of the first sample problem is determined to indicate that the first sample problem is a complex problem.

[0045] Specifically, simple problems can be solved using short reasoning chains with simple reasoning steps. Therefore, the complexity of the first sample problem can be determined by comparing the answer output by a first reasoning model with short reasoning chain capabilities with a preset reference answer. If the first reasoning model can produce an answer that matches the preset reference answer, the first sample problem is determined to be a simple problem. Complex problems, however, have more reasoning steps and stronger logical connections, and the first reasoning model with short reasoning chain capabilities may produce a biased answer due to missing steps or logical errors. If multiple generation attempts fail to produce an answer that matches the preset reference answer, it indicates that the problem requires a more complex and complete long reasoning chain, thus determining the problem to be complex.

[0046] For example, for each answer, the semantic similarity between the answer and a preset reference answer is used to determine whether the answer matches the preset reference answer. For instance, if the semantic similarity is greater than a preset semantic similarity threshold, the answer is determined to match the preset reference answer. If the semantic similarity is less than or equal to the preset semantic similarity threshold, the answer is determined not to match the preset reference answer.

[0047] In another possible implementation, in step 103 above, the complexity identifier of the first sample problem is determined based on the answer corresponding to the first sample problem and the preset reference answer, specifically including the following: First, determine the number of answers that match the preset reference answer among the multiple answers corresponding to the first sample question.

[0048] Then, based on the quantity and the pre-defined mapping relationship between the quantity and the complexity identifier, the complexity identifier of the first sample problem is determined.

[0049] For example, the first sample problem with a number greater than a preset threshold is defined as a simple problem. The first sample problem with a number less than or equal to the preset threshold is defined as a complex problem.

[0050] Considering the randomness of matching results for a single answer, the complexity of the problem can be reflected by statistically analyzing the number of matches across multiple answers; simpler problems have more matches, while complex problems have fewer. This quantitative approach reduces the impact of random factors on complexity assessment and improves accuracy.

[0051] In this embodiment, the complexity of the problem is determined by matching multiple answers with a preset reference answer, simplifying the logic for judging the complexity of the problem. At the same time, multiple answers reduce the random error of a single result, ensuring accurate differentiation between simple and complex problems, improving the accuracy of training data, and enabling the subsequently trained answer generation model to accurately output a simplified reasoning chain for simple problems and a complete reasoning chain for complex problems. This avoids problems of overthinking or insufficient reasoning, thereby further reducing the computational cost and latency of model reasoning and improving the flexibility and practicality of the answer generation model in problem scenarios of different complexity.

[0052] In some embodiments, based on any of the foregoing embodiments, the sample reasoning chain and sample answer corresponding to the first sample problem are determined based on the complexity identifier of the first sample problem, specifically including the following steps: a1. Based on the complexity identifier of the first sample problem and the pre-defined mapping relationship between the complexity identifier and the inference model, determine the target inference model corresponding to the first sample problem.

[0053] Specifically, the mapping relationship between the preset complexity identifier and the inference model can be understood as the correspondence rule between the complexity identifier and the inference model. It is used to automatically match the appropriate target inference model according to the complexity of the problem in order to generate the corresponding sample inference chain and sample answer.

[0054] For example, for simple problems, a lightweight inference model (Mini-LLM) with short inference chains is used as the target inference model, while for complex problems, a full inference model (such as GPT-4) with long inference chains is used as the target inference model.

[0055] Understandably, for simple problems, the sample inference chain determined based on the target inference model is a short inference chain, while for complex problems, the sample inference chain determined based on the target inference model is a long inference chain.

[0056] In one possible implementation, in a1 above, the target inference model corresponding to the first sample problem is determined in the following way: If the complexity indicator of the first sample problem indicates that the first sample problem is a complex problem, then the second inference model is determined as the target inference model.

[0057] For the same input problem, the inference chain length generated by the second inference model is greater than that generated by the first inference model.

[0058] Alternatively, if the complexity indicator of the first sample problem indicates that the first sample problem is a simple problem, then the first inference model is determined as the target inference model.

[0059] Specifically, the reasoning chain length is the number of independent logical steps contained in the reasoning chain, used to quantify the level of detail in the reasoning process. For example, the sample reasoning chain length is 1 for simple problems and 5 for complex problems.

[0060] The second inference model is designed to handle complex problems. For the same input problem, the second inference model generates a longer inference chain than the first inference model, possessing a more complete ability to decompose logic and refine steps. For example, the temperature sampling parameter of the second inference model is set to 0.6, and the top-p sampling parameter is set to 0.95.

[0061] In the embodiments of this application, the first reasoning model is used both to initially generate answers to determine the complexity of the first sample question and as a target reasoning model for simple questions to generate concise reasoning chains.

[0062] Because complex problems require multiple layers of logical decomposition, derivation, or result verification, the short reasoning chain of the first reasoning model is insufficient to cover the complete reasoning logic, easily leading to omissions of steps and incorrect answers. The long reasoning chain of the second reasoning model can refine each reasoning detail, such as explanations of formula principles and verification of intermediate results, ensuring the rigor of the reasoning process and meeting the reasoning requirements of complex problems. Therefore, for complex problems, the second reasoning model is used as the target reasoning model.

[0063] For simple problems, there's no need for complex logical decomposition; the short reasoning chain of the first reasoning model can yield the correct answer. In this case, using the first reasoning model as the target reasoning model avoids the waste of computing power and reasoning delays caused by calling the second reasoning model. Furthermore, the first reasoning model has already generated the corresponding reasoning chain during the process of generating the answer to the first sample problem. Therefore, the answer and reasoning chain generated by the first reasoning model can be directly used as the sample answer and sample reasoning chain.

[0064] a2. Using the target reasoning model, determine the sample reasoning chain and sample answer corresponding to the first sample question.

[0065] For example, the first sample question is input into the target reasoning model to obtain the sample reasoning chain and sample answer corresponding to the first sample question.

[0066] In one possible implementation, a target reasoning model is used to determine the sample reasoning chain and sample answer corresponding to the first sample question, including: b1. According to the preset number of first loops, input the first sample question into the target reasoning model in each loop, and obtain the candidate reasoning chain and candidate answer corresponding to the first sample question in each loop.

[0067] Specifically, the second iteration count refers to the number of times the same first sample question is input into the target inference model, used to obtain multiple candidate results to improve accuracy. For example, setting the first iteration count to 8 means that the first sample question is input into the target inference model 8 times, and each time a candidate result is obtained (each candidate result contains a candidate inference chain and a corresponding candidate answer), ultimately resulting in 8 candidate results.

[0068] In this way, multiple independent candidate results are generated through repeated iterations, and incorrect answers are subsequently filtered out, reducing the error of the sample data, facilitating the selection of the optimal sample reasoning chain, and improving the quality of the sample data.

[0069] b2. Based on the preset reference answer corresponding to the first sample question, all candidate answers are filtered to obtain the filtered candidate answers.

[0070] For example, the preset reference answer is compared with each candidate answer individually, and all candidate answers are filtered based on the comparison results. For instance, the semantic similarity between the preset reference answer and the candidate answers is calculated, and candidate answers with a semantic similarity greater than a preset semantic similarity threshold are selected as the filtered candidate answers. In this way, the filtered candidate answers are consistent with the preset reference answer, ensuring the accuracy of the answer.

[0071] b3. Based on all the filtered candidate answers and the candidate reasoning chain corresponding to each filtered candidate answer, determine the sample reasoning chain and sample answer corresponding to the first sample question.

[0072] Optionally, based on all the filtered candidate answers and the candidate inference chain corresponding to each filtered candidate answer, determine the sample inference chain and sample answer corresponding to the first sample question, specifically including the following steps: First, based on the inference chain length of the candidate inference chains corresponding to all the filtered candidate answers, a sample inference chain corresponding to the first sample question is selected from the candidate inference chains corresponding to all the filtered candidate answers.

[0073] For example, the candidate inference chain with the shortest inference chain among all the filtered candidate answers is taken as the sample inference chain.

[0074] In this way, while ensuring the accuracy of sample answers, redundant steps and invalid expressions in the reasoning process are minimized, avoiding the problem of overthinking from the source. This allows the answer generation model to achieve the correct result with the simplest logic, improving the reasoning efficiency and response speed of the answer generation model, and making the answer generation model more suitable for resource-constrained application scenarios.

[0075] Then, the candidate answers corresponding to the sample reasoning chain of the first sample question are determined as the sample answers corresponding to the first sample question.

[0076] Optionally, based on all the filtered candidate answers and the candidate inference chain corresponding to each filtered candidate answer, determine the sample inference chain and sample answer corresponding to the first sample question, specifically including the following steps: First, cluster all the filtered candidate answers to obtain at least one set of candidate answers.

[0077] Here, the purpose of clustering is to group the filtered candidate answers according to the consistency of results, similarity of expression, etc., so that the core of the answers in the same candidate answer set is consistent, and only the expression differs in detail.

[0078] Secondly, based on the number of answers in each candidate answer set, the target answer set is determined from at least one candidate answer set.

[0079] For example, the set of candidate answers with the largest number of answers can be selected as the target answer set. This prioritizes core results that repeatedly appear in multiple model inferences, as these results are more reliable and effectively reduce the impact of random errors from single inference results on the sample data. Furthermore, the set of candidate answers with the largest number of answers indicates that the answers in this set are more consistent with the model's regular reasoning logic for the question, and the target set ensures that the selection of candidate sample answers and inference chains is more representative.

[0080] Then, determine the semantic similarity between the first candidate answer and the candidate reasoning chain corresponding to the first candidate answer.

[0081] The first candidate answer is one of the answers in the target answer set.

[0082] Specifically, the semantic similarity between the first candidate answer and the corresponding candidate reasoning chain indicates the degree of semantic matching between the candidate answer and the reasoning chain, and is used to measure whether the answer and the reasoning chain are logically consistent.

[0083] Finally, after determining the semantic similarity of each candidate answer in the target answer set, the sample answer and sample inference chain corresponding to the first sample question are determined based on the semantic similarity of each candidate answer in the target answer set.

[0084] In this way, firstly, by clustering the selected candidate answers, the interference of differences in the expression of candidate answers is eliminated, ensuring that answers with consistent core results are grouped into one category. Next, the selection of the target answer set based on the number of answers ensures the universality and reliability of the sample answers. Furthermore, the semantic similarity between the reasoning chain and the candidate answers is calculated, quantifying whether the reasoning logic is consistent, thus ensuring the accuracy and standardization of the sample answers.

[0085] In this embodiment, by establishing a pre-defined mapping relationship between complexity identifiers and inference models, the target inference model is adaptively adjusted. This allows simple problems to be adapted to lightweight inference models to generate short inference chains, while complex problems are adapted to full models to generate complete inference chains. This avoids the waste of computing power and inference delays caused by using full models for simple problems, while ensuring the accuracy of inference for complex problems. It also improves the quality of sample data and provides more accurate and adaptable sample inference chains and sample answers for the training of subsequent answer generation models. This results in a balance between inference efficiency, computing power cost, and inference accuracy in the final answer generation model, enhancing the model's practicality in different scenarios.

[0086] In some embodiments, based on any of the foregoing embodiments, when the target inference model is a first inference model, i.e., the first sample problem is a simple problem, the first sample data is determined based on the first sample problem, the complexity identifier of the first sample problem, the sample inference chain, and the sample answer. Specifically, this includes the following steps: First, the reasoning chain that satisfies the preset conditions obtained by inputting the first sample question into the first reasoning model is determined as the supplementary reasoning chain, and the answer corresponding to the supplementary reasoning chain is determined as the supplementary answer.

[0087] Specifically, the preset conditions are pre-defined criteria used to filter and supplement the reasoning chain. For example, the preset conditions could be the answer that matches the preset reference answer after the first sample question is input into the first reasoning model, or the answer that matches the preset reference answer and has the shortest reasoning chain.

[0088] Then, the first sample question, the complexity identifier of the first sample question, the sample reasoning chain, the sample answer, as well as the supplementary reasoning chain and the supplementary answer are determined as the first sample data.

[0089] At this point, the first sample data obtained is a complete data unit containing the first sample question, complexity identifier, sample reasoning chain, sample answer, supplementary reasoning chain, and supplementary answer, which is used to enrich the training sample dimensions of simple questions.

[0090] Considering that the second reasoning model has the ability to form long reasoning chains and its reasoning process is more rigorous, in this embodiment of the application, supplementary reasoning chains and supplementary answers are added to the first sample data to supplement the complete reasoning logic of short questions. This allows the answer generation model to not only learn fast and concise reasoning ability, but also understand the complete logical derivation process, avoiding the lack of reasoning logic caused by only learning short reasoning chains, and improving the accuracy of the answer generation model in reasoning for simple questions.

[0091] Figure 2 This is a flowchart illustrating a training method for an answer generation model according to an embodiment of the present invention. Figure 2 As shown, the process includes: S201, Obtain the training set and validation set.

[0092] The sample data in the training and validation sets were obtained using the method described above for obtaining training data for the answer generation model.

[0093] Specifically, the training set and validation set are partitioned from multiple sample data sets. The training set is used to train the model parameters, and the validation set is used to evaluate the model's training performance. For example, 90% of the multiple sample data sets can be used as the training set, and 10% can be used as the validation set.

[0094] It is understandable that both the training set and the validation set contain complexity labels, sample reasoning chains, and sample answers corresponding to simple and complex problems, respectively.

[0095] S202, in the current training round, the pre-built answer generation model is trained based on the training set to obtain the trained answer generation model.

[0096] Specifically, the answer generation model can be a large language model that lacks reasoning capabilities, such as the Transformer model. The trained answer generation model is the model whose parameters have been updated after the current training epoch. Compared to the initial answer generation model or the model trained in the previous training epoch (i.e., the model before this training epoch), it has a better fit on the training set.

[0097] In one possible implementation, in S202 above, the answer generation model is trained as follows: c1, according to the preset splicing rules, splices the complexity identifier, sample reasoning chain and sample answer in the second sample data to obtain the benchmark sequence corresponding to the second sample data.

[0098] The second sample data is a sample data in the training set.

[0099] Specifically, the preset splicing rules are used to combine complexity identifiers, sample reasoning chains, and sample answers into a continuous text sequence according to a preset format.

[0100] For example, when the second sample question is a simple question, the " <simple>The format is "-long reasoning chain-short reasoning chain", which is "question:[question text]". <simple><Long Thinking>[Detailed Reasoning Process] <finalanswer>Answer: [Final Answer]<Short Thinking> [Concise Reasoning Process]<Final Answer> Answer: [Final Answer] <simple>The purpose of retaining both long and short thought chains in the problem samples is to provide additional training supervision for the short reasoning chains through the long reasoning chains, and to prevent the answer generation model from being affected by the long reasoning chain reasoning ability due to learning the short reasoning chains.

[0101] Of course, when the second sample problem is a simple problem, " <simple>The format for splicing "-short reasoning chain".

[0102] For example, when the second sample question is a complex question, a long inference chain format is used: "Question:[Question Text]"<Long Thinking> [Detailed Reasoning Process]<Final Answer> Answer: [Final Answer]

[0103] In this embodiment, the complexity identifier, sample reasoning chain, and sample answer in the sample data are the core information that the answer generation model needs to learn. By splicing the contents of the sample data into a baseline sequence according to the preset splicing rules, the model optimization chaos caused by multi-objective training can be avoided, and the model output structure can be ensured to be consistent.

[0104] c2 inputs the sample questions from the second sample data into the answer generation model to obtain the prediction sequence corresponding to the second sample data.

[0105] Specifically, the baseline sequence serves as the target reference sequence for the answer generation model's prediction. The prediction sequence is the text sequence output by the answer generation model after the second sample data is input, which includes a prediction complexity indicator, a prediction inference chain, and the predicted answer, and is structurally aligned with the baseline sequence.

[0106] c3. After obtaining the baseline and prediction sequences corresponding to all sample data in the training set, the parameters in the answer generation model are updated based on the baseline and prediction sequences corresponding to all sample data in the training set, thus obtaining the trained answer generation model.

[0107] Specifically, the trained answer generation model compares the baseline sequence and the predicted sequence of all sample data, calculates the error through the loss function, and updates the parameters through backpropagation to obtain the optimized model.

[0108] Optionally, based on the baseline and predicted sequences corresponding to all sample data in the training set, the parameters in the answer generation model are updated to obtain the trained answer generation model, specifically including the following steps: d1, based on the prediction result corresponding to the first position in the first prediction sequence and the benchmark result corresponding to the first position in the first benchmark sequence, determine the preset loss function value corresponding to the first position in the first prediction sequence.

[0109] Wherein, the first predicted sequence is the predicted sequence corresponding to a sample data in the training set, the first reference sequence is the reference sequence corresponding to the first predicted sequence, and the first position is a position in the first predicted sequence.

[0110] Specifically, the position in the sequence is the index position corresponding to the semantic unit (such as token level, character level). For example, if split by token (such as word, phrase, etc.), the token sequence in the first prediction sequence is {"[complexity]: simple", "inference chain:", "25×4 directly calculated", "100", "answer:", "100"}. The first position can be the position corresponding to "[complexity]: simple", the position corresponding to "inference chain:", etc.

[0111] The prediction result corresponding to the first position refers to the semantic unit, such as a token or character, at the first position in the first prediction sequence, which is the prediction output of the answer generation model at that position. Continuing with the token sequence in the first prediction sequence above as an example, if the first position is the first token in the sequence, then the prediction result corresponding to the first position is "[Complexity]: Simple".

[0112] The benchmark result corresponding to the first position is the semantic unit in the first benchmark sequence that has the same index as the first position, which serves as the target output of the answer generation model at that position.

[0113] The preset loss function value is the error between the prediction result calculated using the preset loss function and the baseline result, used to quantify the prediction deviation at a single location. For example, for the first location mentioned above, the preset loss function value is determined using the cross-entropy loss function.

[0114] d2, after determining the preset loss function values ​​corresponding to all positions in all predicted sequences, determine the model parameter adjustment strategy based on the preset loss function values ​​corresponding to all positions in all predicted sequences.

[0115] Specifically, model parameter adjustment strategies include, but are not limited to, adjusting the direction of the parameters (i.e., increasing or decreasing the parameters) and adjusting the step size.

[0116] Optionally, the model parameter tuning strategy can be determined as follows: First, the average loss function value is determined based on the preset loss function values ​​corresponding to all positions in all predicted sequences.

[0117] Specifically, the average loss function value is obtained by taking the arithmetic mean of the preset loss function values ​​at all positions in all predicted sequences, and is used to reflect the overall error level of model training.

[0118] Secondly, the target loss position is determined based on the average loss function value and the preset loss function value corresponding to each position.

[0119] For example, the location corresponding to a preset loss function value that is greater than the average loss function value is determined as the target loss location. This is because if the preset loss function value at a certain location is significantly higher than the average loss function value, it will be the focus of model parameter adjustment.

[0120] For example, the position where the difference between the average loss function value and the preset loss function value is greater than the preset difference threshold is taken as the target loss position.

[0121] Next, based on the target loss location, the parameters to be adjusted in the answer generation model are determined.

[0122] Specifically, the output at different locations in the model is determined by different parameters. For example, different heads in the attention layer correspond to different semantic units, and the error at the target loss location is directly related to specific parameters. The parameters corresponding to the target loss location can be determined by pre-setting the correspondence between the location and the parameters in the model (such as the semantic mapping logic of the model architecture). In this way, determining the parameters to be adjusted by pre-setting the loss function value avoids blind parameter tuning and ensures that the adjustment strategy directly applies to the error location.

[0123] For example, based on the target loss location and the preset mapping relationship between the location and model parameters, the parameters to be adjusted in the answer generation model are determined. For instance, inference chain representation bias corresponds to the weights of the self-attention layer, and complexity labeling errors correspond to the classification head parameters.

[0124] Then, the adjustment direction of the parameter to be adjusted is determined based on the gradient of the preset loss function, and the adjustment range is determined based on the difference between the average loss function value and the preset loss function value.

[0125] Specifically, the gradient of a pre-defined loss function refers to the partial derivative of the pre-defined loss function (such as cross-entropy loss) with respect to a parameter to be adjusted. It indicates how the parameter should be changed to reduce the loss, and the sign of the gradient determines the direction of adjustment. For example, the pre-defined loss function is the cross-entropy loss function, and the value of the pre-defined loss function is the numerical value corresponding to the cross-entropy loss function.

[0126] For example, the parameter to be adjusted is decreased when the gradient of the preset loss function is positive, and increased when the gradient of the preset loss function is negative. The larger the difference between the average loss function value and the preset loss function value, the larger the adjustment.

[0127] Finally, the parameters to be adjusted are adjusted based on the direction and magnitude of the adjustment.

[0128] In this way, by using the average loss function value as the error benchmark, the target loss position can be accurately screened, the parameters to be adjusted can be determined, and further, by combining the gradient and the error difference, the adjustment direction and magnitude can be determined, and the weak links with prominent errors can be corrected in a targeted manner, thereby improving the efficiency and effectiveness of parameter adjustment.

[0129] d3, based on the model parameter adjustment strategy, updates the parameters in the answer generation model to obtain the trained answer generation model.

[0130] In this way, by calculating the preset loss function value position by position, the source of error can be accurately determined. Then, based on the error, the parameters in the answer generation model can be precisely adjusted to ensure that the final answer generation model learns the unit-by-unit mapping relationship of the question-target sequence, improves the structural consistency and accuracy of the output sequence, and makes the predicted sequence output by the model highly consistent with the benchmark sequence in terms of structure and details, thereby improving the accuracy of the reasoning answer and strengthening the simplicity of the reasoning chain and the degree of matching of the complexity indicator.

[0131] In this embodiment, the answer generation model is a base model (such as Qwen2.5-72B), and complexity identifiers and target inference model identifiers are added by configuring the model's tokenizer. The training parameters of the answer generation model can be a sequence length of 16384 tokens to ensure that it can accommodate complete long-short inference chains.

[0132] S203, based on the validation set, determines the accuracy and first average inference chain length of the trained answer generation model.

[0133] Specifically, accuracy refers to the proportion of consistency between the sample answers output by the trained model on the validation set and the corresponding preset reference answers. It is used to measure the accuracy of the reasoning results of the trained answer generation model. For example, if the validation set has 10,000 sample data points and the trained answer generation model outputs 9,000 correct answers, then the accuracy rate is 90%.

[0134] The first average inference chain length is the average length of all inference chains generated by the trained answer generation model on the validation set, used to measure the conciseness of the inference chains generated by the trained answer generation model.

[0135] S204, based on the validation set, determine the second average inference chain length of the answer generation model.

[0136] Specifically, the second average inference chain length is the average length of all inference chains generated on the validation set before training in the current training epoch of the answer generation model. By comparing the first average inference chain length and the second average inference chain length, the optimization effect of the current training epoch can be indicated.

[0137] S205, if the preset iteration stopping condition is not met based on the accuracy, the first average inference chain length, and the second average inference chain length, then proceed to the next training round until the preset iteration stopping condition is met based on the accuracy, the first average inference chain length, and the second average inference chain length, and then obtain the final answer generation model.

[0138] Specifically, the preset iteration stopping condition is a predefined standard for terminating model training, ensuring the accuracy and inference efficiency of the answer generation model. The final answer generation model is an optimized model obtained after multiple rounds of training iterations, satisfying the preset iteration stopping condition, and possessing high accuracy and a streamlined inference chain that adapts to varying complexity.

[0139] For example, if the accuracy is less than or equal to the highest historical accuracy in two consecutive training rounds, and the first average inference chain length is greater than the second average inference chain length, then the preset iteration stopping condition is determined to have been met.

[0140] If the accuracy rate does not exceed the historical high for two consecutive rounds, it indicates that the accuracy of the answer generation model is close to its limit. If the first average inference chain length is greater than the second average inference chain length for two consecutive rounds, it indicates that the answer generation model has not optimized the overthinking problem in subsequent iterations, but instead shows a trend of increasing inference chain redundancy. At this point, training should be stopped to ensure that the final answer generation model has both accuracy and the ability to streamline the inference chain.

[0141] For example, if the accuracy of the current training round is greater than a preset accuracy threshold, and the absolute value of the difference between the first average inference chain length and the second average inference chain length is less than a preset absolute value threshold, then the preset iteration stop condition is determined to have been met.

[0142] In this embodiment, by combining multiple dimensions such as accuracy and the average length of the inference chain generated before and after training the answer generation model, the performance of the model is gradually optimized. This ensures that the final answer generation model can achieve high standards of inference accuracy and generate inference chains that are adapted to the complexity of the problem. While maintaining high inference accuracy, it significantly reduces the length of the inference chain and the computational cost, adapts to lightweight deployment in resource-constrained scenarios, and can cope with inference tasks of different complexities.

[0143] In some embodiments, based on any of the foregoing embodiments, this application provides an answer generation method, which includes the following: First, obtain the problem to be reasoned.

[0144] Then, the question to be reasoned is input into the final answer generation model mentioned above to obtain the reasoning chain and answer corresponding to the question to be reasoned.

[0145] For example, when the final answer generation model receives an input question to be reasoned, such as "What is 56 multiplied by 78?", the model converts the question into the model input format: "Question: [User Input Question]", without adding any length instructions or complexity markers. Next, the trigger marker recognition and reasoning mode switch is initiated, including: setting the initial generation length (e.g., setting the initial generation length to 1 token (used to generate trigger markers)) and the model generating the initial marker (i.e., if the generated marker is...). <hard>If the generated tag is..., then switch to long inference mode; if the generated tag is... <simple>If the answer is not generated, the system switches to short reasoning mode. For long reasoning mode, the answer generation model continues to generate long reasoning chains until a solution is generated.<Final Answer> Then output the final answer. For short reasoning mode, the model automatically... <simple>Replace mark with<Short Thinking> Then generate short inference chains until a short inference chain is generated.<Final Answer> Output the final answer. Finally, remove the markers from the reasoning results, such as complexity markers, reasoning mode markers, and answer markers, retaining only the reasoning chain and the answer.

[0146] Taking the sample question "What are the most basic troubleshooting steps for a server that is not responding when it is powered on?" as an example, the short inference chain generated by the answer generation model can be "Check the power supply → Check the power button → Check the hardware connection", and the generated answer can be "1. Confirm that the power supply is connected; 2. Check if the power button is working properly; 3. Check if the core hardware wiring is loose".

[0147] Taking the complex problem "Frequent blue screen restarts of rack-mounted servers, how to troubleshoot step by step by combining system logs and hardware status?" as an example, the long inference chain generated by the answer generation model can be "First check the system blue screen logs to determine the type of fault → then check the memory / hard drive hardware status → verify the heat dissipation and power supply stability → check driver / system compatibility → finally locate the fault point." The generated answer can be "1. Check the system blue screen logs to determine whether it is a hardware or software fault; 2. Use testing tools to test whether there are bad sectors / errors in the memory and hard drive; 3. Check the working status of the cooling fan and power module to confirm that there is no overheating / unstable power supply; 4. Check recently updated driver / system patches and roll back for testing; 5. Replace the suspected hardware one by one to locate the specific fault point."

[0148] In this embodiment, the final answer generation model can generate trigger markers (e.g., simple markers / complex markers) during the inference phase and automatically switch to long inference mode or short inference mode based on the trigger markers. In short inference mode, no model weight adjustment is required; simply replacing the simple markers with short inference markers generates a short inference chain. This switching mechanism, based on trigger markers, eliminates the need to adjust model weights, achieving lightweight dynamic switching.

[0149] The following is an exemplary description of the process for obtaining training data for an answer generation model, using a complete embodiment. The process includes the following steps: Step 1: Using the second inference model, obtain multiple first-class results corresponding to each initial sample question in the initial dataset. Each first-class result includes an inference chain (i.e., a long inference chain) and an answer.

[0150] Step 2: Verify the accuracy of multiple Class I results for each initial sample problem. If one Class I result passes the verification, then the initial sample problem is used as the sample problem in S101 above.

[0151] For example, if one of the multiple answers matches the preset reference answer, the first type of result is determined to have passed verification, and the initial sample question is used as the sample question in S101 above. If none of the multiple answers match the preset reference answer, the initial sample question is re-inputted into the second inference model to obtain an updated first type of result. If none of the answers in the updated first type of result match the preset reference answer, the initial sample question is discarded. If any answer in the updated first type of result matches the preset reference answer, the initial sample question is used as the sample question in S101 above.

[0152] At the same time, the shortest reasoning chain is selected from the results that match the preset reference answer, and the corresponding answer is selected as the first type of sample reasoning chain and the first type of sample answer corresponding to the sample question.

[0153] Step 3: Using the first reasoning model, obtain multiple second-class results corresponding to each sample question. Each second-class result includes a reasoning chain (i.e., a short reasoning chain) and an answer.

[0154] Step 4: Verify the accuracy of multiple second-type results for each sample question in Step 3.

[0155] For example, if one of the multiple answers matches the preset reference answer, the sample question is classified as a simple question. If none of the multiple answers match the preset reference answer, the sample question is classified as a complex question.

[0156] Step 5: For simple problems, organize the simple problem, its corresponding complexity identifier, first type of sample reasoning chain, first type of sample answer, second type of sample reasoning chain, and second type of sample answer according to the first preset data format; for complex problems, concatenate the complex problem, its corresponding complexity identifier, first type of sample reasoning chain, and first type of sample answer according to the second preset data format.

[0157] For example, for simple problems, use " <simple>The "tag-long inference chain-short inference chain" concatenation format is used for complex problems. <hard>The "tag-long inference chain" concatenation format.

[0158] In this embodiment of the application, a labeled dataset containing long inference chains, short inference chains, and special tags is constructed to train a large language model to autonomously determine the complexity of the inference problem and dynamically select to generate a long or short inference chain.

[0159] The above mainly describes the solution provided by the embodiments of this application from a methodological perspective.

[0160] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method.

[0161] This application also provides a device for acquiring training data for an answer generation model. This device is used to implement the above embodiments and preferred embodiments, and will not be repeated hereafter. As used below, the term "module" can be a combination of software and / or hardware that performs a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0162] This embodiment provides a device for acquiring training data for an answer generation model, such as... Figure 3 As shown, it includes: The first acquisition module 301 is used to acquire multiple sample questions; The second acquisition module 302 is used to acquire the answer corresponding to the first sample question using the first reasoning model, wherein the first sample question is one of multiple sample questions; The first determining module 303 is used to determine the complexity identifier of the first sample question based on the answer corresponding to the first sample question and the preset reference answer, wherein the complexity identifier is used to characterize the complexity of the first sample question; The second determining module 304 is used to determine the sample reasoning chain and sample answer corresponding to the first sample question based on the complexity identifier of the first sample question. The third determining module 305 is used to determine the first sample data based on the first sample question, the complexity identifier of the first sample question, the sample inference chain and the sample answer, until the operation stops after obtaining sample data based on all sample questions. All sample data are used to train the pre-built answer generation model to obtain the final answer generation model.

[0163] In one possible implementation, the first determining module 303 is specifically used to determine the complexity identifier of the first sample problem as a simple problem when at least one of the answers matches the preset reference answer. or, When all answers do not match the preset reference answer, the complexity identifier of the first sample problem is determined to indicate that the first sample problem is a complex problem.

[0164] In one possible implementation, the second determining module 304 is specifically used to determine the target inference model corresponding to the first sample problem based on the complexity identifier of the first sample problem and the preset mapping relationship between the complexity identifier and the inference model. Using the target reasoning model, determine the sample reasoning chain and sample answer corresponding to the first sample question.

[0165] In one possible implementation, the second determining module 304 is specifically used to determine the second inference model as the target inference model if the complexity identifier of the first sample problem indicates that the first sample problem is a complex problem, wherein, for the same input problem, the length of the inference chain generated by the second inference model is greater than the length of the inference chain generated by the first inference model. or, If the complexity indicator of the first sample problem indicates that the first sample problem is a simple problem, then the first inference model is determined as the target inference model.

[0166] In one possible implementation, the second determining module 304 is specifically used to input the first sample question into the target reasoning model in each loop according to a preset first loop number, and obtain the candidate reasoning chain and candidate answer corresponding to the first sample question in each loop; Based on the preset reference answer corresponding to the first sample question, all candidate answers are filtered to obtain the filtered candidate answers; Based on all the filtered candidate answers and the candidate reasoning chain corresponding to each filtered candidate answer, determine the sample reasoning chain and sample answer corresponding to the first sample question.

[0167] In one possible implementation, the second determining module 304 is specifically used to select the sample inference chain corresponding to the first sample question from the candidate inference chains corresponding to all the selected candidate answers based on the inference chain length of the candidate inference chains corresponding to all the selected candidate answers respectively. The candidate answers corresponding to the sample inference chain of the first sample question are determined as the sample answers corresponding to the first sample question.

[0168] In one possible implementation, the second determining module 304 is specifically used to cluster all the filtered candidate answers to obtain at least one set of candidate answers; Based on the number of answers in each candidate answer set, determine the target answer set from at least one candidate answer set; Determine the semantic similarity between the first candidate answer and the candidate inference chain corresponding to the first candidate answer, where the first candidate answer is an answer in the target answer set; Once the semantic similarity of each candidate answer in the target answer set is determined, the sample answer and sample inference chain corresponding to the first sample question are determined based on the semantic similarity of each candidate answer in the target answer set.

[0169] In one possible implementation, when the target reasoning model is the first reasoning model, the third determining module 305 is specifically used to determine the reasoning chain that satisfies the preset conditions obtained by inputting the first sample question into the second reasoning model as a supplementary reasoning chain, and to determine the answer corresponding to the supplementary reasoning chain as a supplementary answer. The first sample question, the complexity identifier of the first sample question, the sample reasoning chain, the sample answer, as well as the supplementary reasoning chain and the supplementary answer are identified as the first sample data.

[0170] For a description of the features in the embodiment corresponding to the device for acquiring training data of the answer generation model, please refer to the relevant description in the embodiment corresponding to the method for acquiring training data of the answer generation model, which will not be repeated here.

[0171] This application also provides a training device for an answer generation model, which is used to implement the above embodiments and preferred embodiments, and will not be repeated hereafter. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0172] This embodiment provides a training device for an answer generation model, such as... Figure 4 As shown, it includes: The third acquisition module 401 is used to acquire the training set and the validation set, wherein the sample data in the training set and the validation set are obtained based on the acquisition method of the training data of the answer generation model mentioned above. Training module 402 is used to train the pre-built answer generation model based on the training set in the current training round to obtain the trained answer generation model; Validation module 403 is used to determine the accuracy and first average inference chain length of the trained answer generation model based on the validation set. The fourth determination module 404 is used to determine the second average inference chain length of the answer generation model based on the validation set; The fifth determination module 405 is used to enter the next training round if the preset iteration stopping condition is not met based on the accuracy, the first average inference chain length, and the second average inference chain length, until the preset iteration stopping condition is met based on the accuracy, the first average inference chain length, and the second average inference chain length, and then obtain the final answer generation model.

[0173] In one possible implementation, the fifth determining module 405 is specifically used to determine that the preset iteration stopping condition has been reached if the accuracy is less than or equal to the historical highest accuracy in two consecutive training rounds and the first average inference chain length is greater than the second average inference chain length.

[0174] In one possible implementation, the training module 402 is specifically used to concatenate the complexity identifier, sample inference chain, and sample answer in the second sample data according to a preset concatenation rule to obtain a benchmark sequence corresponding to the second sample data, wherein the second sample data is a sample data in the training set. Input the sample questions from the second sample data into the answer generation model to obtain the prediction sequence corresponding to the second sample data; After obtaining the baseline and prediction sequences corresponding to all sample data in the training set, the parameters in the answer generation model are updated based on the baseline and prediction sequences corresponding to all sample data in the training set, resulting in the trained answer generation model.

[0175] In one possible implementation, the training module 402 is specifically used to determine the preset loss function value corresponding to the first position of the first prediction sequence based on the prediction result corresponding to the first position in the first prediction sequence and the benchmark result corresponding to the first position in the first benchmark sequence. Here, the first prediction sequence is the prediction sequence corresponding to a sample data in the training set, the first benchmark sequence is the benchmark sequence corresponding to the first prediction sequence, and the first position is a position in the first prediction sequence. Once the preset loss function values ​​for each position in all predicted sequences are determined, a model parameter adjustment strategy is determined based on these preset loss function values. Based on the model parameter adjustment strategy, the parameters in the answer generation model are updated to obtain the trained answer generation model.

[0176] For a description of the features in the embodiment corresponding to the training device of the answer generation model, please refer to the relevant description of the embodiment corresponding to the training method of the answer generation model, which will not be repeated here.

[0177] Embodiments of this application also provide an electronic device, such as... Figure 5 As shown, it includes a memory 10 and a processor 20. The memory 10 stores a computer program, and the processor 20 is configured to run the computer program to perform the steps in any of the above-described methods for obtaining training data for the answer generation model or for training the answer generation model.

[0178] Embodiments of this application also provide a computer-readable storage medium storing a computer program, wherein the computer program is configured to execute the steps in any of the above-described methods for obtaining training data for an answer generation model or for training an answer generation model.

[0179] In one exemplary embodiment, the aforementioned computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard disk, magnetic disk, or optical disk.

[0180] The embodiments of this application also provide a computer program product, which includes a computer program. When the computer program is executed by a processor, it implements the steps in any of the above-described methods for obtaining training data for the answer generation model or the training method for the answer generation model.

[0181] Embodiments of this application also provide another computer program product, including a non-volatile computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it implements the steps in any of the above-described methods for obtaining training data for the answer generation model or for training the answer generation model.

[0182] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0183] The foregoing has provided a detailed description of the method for obtaining training data for an answer generation model and the method for training the answer generation model provided in this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only intended to help understand the method and core ideas of this application. It should be noted that those skilled in the art can make various improvements and modifications to this application without departing from its principles, and these improvements and modifications also fall within the protection scope of the claims of this application.< / hard> < / simple> < / simple> < / simple> < / hard> < / simple> < / simple> < / finalanswer> < / simple> < / simple>

Claims

1. A method for obtaining training data for an answer generation model, characterized in that, The method includes: The problem of obtaining multiple samples; Using a first reasoning model, the answer corresponding to a first sample question is obtained, wherein the first sample question is one of the plurality of sample questions; Based on the answer corresponding to the first sample question and the preset reference answer, a complexity identifier for the first sample question is determined, wherein the complexity identifier is used to characterize the complexity of the first sample question; Based on the complexity identifier of the first sample question, determine the sample reasoning chain and sample answer corresponding to the first sample question; Based on the first sample question, as well as the complexity identifier, sample inference chain, and sample answer of the first sample question, the first sample data is determined. The operation stops after obtaining sample data based on all the sample questions. All the sample data are used to train the pre-built answer generation model to obtain the final answer generation model.

2. The method according to claim 1, characterized in that, The answers corresponding to the first sample question include multiple options; determining the complexity identifier of the first sample question based on the answers corresponding to the first sample question and a preset reference answer includes: When at least one of the answers matches the preset reference answer, the complexity identifier of the first sample problem is determined to indicate that the first sample problem is a simple problem. or, When all the answers do not match the preset reference answer, the complexity identifier of the first sample problem is determined to indicate that the first sample problem is a complex problem.

3. The method according to claim 1, characterized in that, The step of determining the sample inference chain and sample answer corresponding to the first sample question based on the complexity identifier of the first sample question includes: Based on the complexity identifier of the first sample problem and the preset mapping relationship between the complexity identifier and the inference model, the target inference model corresponding to the first sample problem is determined. Using the target reasoning model, determine the sample reasoning chain and sample answer corresponding to the first sample question.

4. The method according to claim 3, characterized in that, The step of determining the target inference model corresponding to the first sample problem based on the complexity identifier of the first sample problem and the preset mapping relationship between the complexity identifier and the inference model includes: If the complexity identifier of the first sample problem indicates that the first sample problem is a complex problem, then the second reasoning model is determined as the target reasoning model, wherein, for the same input problem, the reasoning chain length generated by the second reasoning model is greater than the reasoning chain length generated by the first reasoning model; or, If the complexity identifier of the first sample problem indicates that the first sample problem is a simple problem, then the first reasoning model is determined as the target reasoning model.

5. The method according to claim 3 or 4, characterized in that, The step of using the target reasoning model to determine the sample reasoning chain and sample answer corresponding to the first sample question includes: According to the preset first number of loops, the first sample question is input into the target reasoning model in each loop to obtain the candidate reasoning chain and candidate answer corresponding to the first sample question in each loop; Based on the preset reference answer corresponding to the first sample question, all the candidate answers are filtered to obtain the filtered candidate answers; Based on all the filtered candidate answers and the candidate reasoning chain corresponding to each filtered candidate answer, the sample reasoning chain and sample answer corresponding to the first sample question are determined.

6. The method according to claim 5, characterized in that, The step of determining the sample inference chain and sample answer corresponding to the first sample question based on all the filtered candidate answers and the candidate inference chain corresponding to each filtered candidate answer includes: Based on the inference chain length of the candidate inference chains corresponding to all the filtered candidate answers, a sample inference chain corresponding to the first sample question is selected from the candidate inference chains corresponding to all the filtered candidate answers. The candidate answers corresponding to the sample inference chain of the first sample question are determined as the sample answers corresponding to the first sample question.

7. The method according to claim 5, characterized in that, The step of determining the sample inference chain and sample answer corresponding to the first sample question based on all the filtered candidate answers and the candidate inference chain corresponding to each filtered candidate answer includes: Cluster all the filtered candidate answers to obtain at least one set of candidate answers; A target answer set is determined from the at least one candidate answer set based on the number of answers in each candidate answer set; Determine the semantic similarity between the first candidate answer and the candidate inference chain corresponding to the first candidate answer, wherein the first candidate answer is an answer in the target answer set; After determining the semantic similarity of each of the candidate answers in the target answer set, the sample answer and sample reasoning chain corresponding to the first sample question are determined based on the semantic similarity of each of the candidate answers in the target answer set.

8. The method according to claim 4, characterized in that, When the target reasoning model is the first reasoning model, determining the first sample data based on the first sample question, the complexity identifier of the first sample question, the sample reasoning chain, and the sample answer includes: The reasoning chain that satisfies the preset conditions obtained by inputting the first sample question into the second reasoning model is determined as the supplementary reasoning chain, and the answer corresponding to the supplementary reasoning chain is determined as the supplementary answer; The first sample question, the complexity identifier of the first sample question, the sample reasoning chain, the sample answer, the supplementary reasoning chain, and the supplementary answer are determined as the first sample data.

9. A training method for an answer generation model, characterized in that, The method includes: A training set and a validation set are obtained, wherein the sample data in the training set and the validation set are obtained based on the method for obtaining training data of the answer generation model according to any one of claims 1-8; In the current training round, the pre-built answer generation model is trained based on the training set to obtain the trained answer generation model; Based on the validation set, the accuracy and first average inference chain length of the trained answer generation model are determined. Based on the validation set, determine the second average inference chain length of the answer generation model; If, based on the accuracy, the first average inference chain length, and the second average inference chain length, it is determined that the preset iteration stopping condition has not been met, then the next training round is entered until, based on the accuracy, the first average inference chain length, and the second average inference chain length, it is determined that the preset iteration stopping condition has been met, and then the final answer generation model is obtained.

10. The method according to claim 9, characterized in that, The method further includes: If, in two consecutive training rounds, the accuracy rate is less than or equal to the historical highest accuracy rate, and the first average inference chain length is greater than the second average inference chain length, then the preset iteration stop condition is determined to have been met.

11. The method according to claim 9 or 10, characterized in that, The step of training the pre-built answer generation model based on the training set to obtain the trained answer generation model includes: According to the preset splicing rules, the complexity identifier, sample reasoning chain, and sample answer in the second sample data are spliced ​​together to obtain the benchmark sequence corresponding to the second sample data, wherein the second sample data is a sample data in the training set; The sample questions in the second sample data are input into the answer generation model to obtain the prediction sequence corresponding to the second sample data; After obtaining the baseline sequence and prediction sequence corresponding to all the sample data in the training set, the parameters in the answer generation model are updated based on the baseline sequence and prediction sequence corresponding to all the sample data in the training set, and the trained answer generation model is obtained.

12. The method according to claim 11, characterized in that, The step of updating the parameters in the answer generation model based on the baseline sequence and predicted sequence corresponding to all the sample data in the training set to obtain the trained answer generation model includes: Based on the prediction result corresponding to the first position in the first prediction sequence and the benchmark result corresponding to the first position in the first benchmark sequence, a preset loss function value corresponding to the first position in the first prediction sequence is determined, wherein the first prediction sequence is the prediction sequence corresponding to a sample data in the training set, the first benchmark sequence is the benchmark sequence corresponding to the first prediction sequence, and the first position is a position in the first prediction sequence. Once the preset loss function values ​​corresponding to all positions in all the predicted sequences are determined, a model parameter adjustment strategy is determined based on the preset loss function values ​​corresponding to all positions in all the predicted sequences. Based on the model parameter adjustment strategy, the parameters in the answer generation model are updated to obtain the trained answer generation model.

13. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor, configured to execute the computer program to implement the steps of the method for acquiring training data of the answer generation model as described in any one of claims 1-8, or the steps of the method for training the answer generation model as described in any one of claims 9-12.

Images