Question and answer model training method and device, electronic equipment, computer readable storage medium and computer program product

Abstract

The application provides a question and answer model training method and device, electronic equipment, computer program product and computer readable storage medium; the method comprises the following steps: determining a sample question set required for a current training iteration process of a question and answer model, generating a first training data set of the current training iteration process based on the sample question set; based on the first training data set, performing the current training iteration on the question and answer model obtained in the previous training iteration process to obtain the question and answer model after the current training iteration; determining a first target question based on the question and answer model after the current training iteration; based on the first target question, determining a second training data set of the next training iteration process, and training the question and answer model after the current training iteration by using the second training data set until a training end condition is reached to obtain a trained question and answer model. Through the application, the training efficiency of the question and answer model can be improved, and the question and answer accuracy of the trained question and answer model can be improved.

Description

Technical Field

[0001] This application relates to computer technology, and more particularly to a question-answering model training method, apparatus, electronic device, computer-readable storage medium, and computer program product. Background Technology

[0002] Large language models are now widely used across various fields. When performing mathematical reasoning tasks, they require both complex mathematical semantic understanding and multi-step reasoning capabilities, making mathematical reasoning tasks still quite challenging. In particular, a large amount of high-quality mathematical training data is the fundamental fuel for mathematical reasoning. However, due to the diversity and complexity of mathematical problems, high-quality natural mathematical data is relatively scarce. Therefore, related technologies still suffer from a severe lack of high-quality mathematical training data when training the mathematical reasoning capabilities of large language models. Summary of the Invention

[0003] This application provides a question-answering model training method, apparatus, computer-readable storage medium, and computer program product, which can generate multiple training datasets in multiple training iterations, thereby improving the training efficiency of the question-answering model and the question-answering accuracy of the trained question-answering model.

[0004] The technical solution of this application embodiment is implemented as follows:

[0005] This application provides a question-answering model training method, the method comprising:

[0006] Determine the sample question set required for the current training iteration of the question answering model, and generate a first training dataset for the current training iteration based on the sample question set. The first training dataset includes multiple synthetic questions and the question answer corresponding to each synthetic question.

[0007] Based on the first training dataset, the question answering model obtained in the previous training iteration is used for the current training iteration to obtain the question answering model after the current training iteration.

[0008] Based on the question-answering model after the current training iteration, a first target question is determined from the sample question set and the plurality of synthetic questions. The first target question is the question that the question-answering model after the current training iteration failed to answer.

[0009] Based on the first target question, a second training dataset is determined for the next training iteration, and the question-answering model after the current training iteration is trained using the second training dataset until the training termination condition is met, resulting in a well-trained question-answering model.

[0010] This application provides a question-answering model training device, including:

[0011] The determination module is used to determine the sample question set required for the current training iteration of the question answering model, and generate a first training dataset for the current training iteration based on the sample question set. The first training dataset includes multiple synthetic questions and the question answer corresponding to each synthetic question.

[0012] The training module is used to perform the current training iteration on the question-answering model obtained in the previous training iteration based on the first training dataset, so as to obtain the question-answering model after the current training iteration.

[0013] The determining module is further configured to determine a first target question from the sample question set and the plurality of synthetic questions based on the question-answering model after the current training iteration, wherein the first target question is a question that the question-answering model after the current training iteration failed to answer;

[0014] The determining module is further configured to determine the second training dataset for the next training iteration based on the first target question, and use the second training dataset to train the question-answering model after the current training iteration until the training termination condition is met, thereby obtaining the trained question-answering model.

[0015] This application provides an electronic device, the electronic device comprising:

[0016] Memory is used to store executable instructions or computer programs.

[0017] When a processor executes computer-executable instructions or computer programs stored in the memory, it implements the question-answering model training method provided in the embodiments of this application.

[0018] This application provides a computer-readable storage medium storing a computer program or computer-executable instructions, which, when executed by a processor, implements the question-answering model training method provided in this application.

[0019] This application provides a computer program product, including a computer program or computer executable instructions. When the computer program or computer executable instructions are executed by a processor, they implement the question-answering model training method provided in this application.

[0020] The embodiments of this application have the following beneficial effects:

[0021] In this embodiment, a sample question set required for the current training iteration of the question-answering model is determined. A first training dataset for the current training iteration is generated based on the sample question set. The first training dataset includes multiple synthetic questions and their corresponding answers. Based on the first training dataset, the question-answering model obtained in the previous training iteration is used for the current training iteration to obtain the question-answering model after the current training iteration. In this way, in each training iteration, multiple synthetic questions and answers can be generated to perform the current training iteration on the question-answering model obtained in the previous training iteration, thereby ensuring that multiple training datasets can be generated cyclically in multiple training iterations, improving the quality and richness of the training data, and improving the accuracy of model training. Then, based on the question-answering model after the current training iteration, a first target question is determined from the sample question set and multiple synthetic questions. The first target question is the question that the question-answering model failed to answer in the current training iteration. Based on the first target question, a second training dataset for the next training iteration is determined, and the question-answering model after the current training iteration is trained using the second training dataset. This ensures that the training dataset for each training iteration includes the questions the model failed to answer in the previous iteration. This allows for targeted training of the model each time it is trained, fully utilizing previously unsolved problems and further improving the accuracy of the question-answering model. Finally, the training continues until the termination condition is met, resulting in a well-trained question-answering model that improves both accuracy and performance. Attached Figure Description

[0022] Figure 1 This is a schematic diagram of the structure of the model training system provided in the embodiments of this application;

[0023] Figure 2 This is a schematic diagram of the structure of the server 200 provided in the embodiments of this application;

[0024] Figure 3A This is a flowchart illustrating the question-answering model training method provided in the embodiments of this application;

[0025] Figure 3B This is a flowchart illustrating the process of determining the sample problem set provided in an embodiment of this application;

[0026] Figure 3C This is a flowchart illustrating the process of determining a candidate problem set provided in an embodiment of this application;

[0027] Figure 3D This is another flowchart illustrating the process of determining the sample problem set provided in an embodiment of this application;

[0028] Figure 3E This is a flowchart illustrating the process of determining the answer to a question, as provided in an embodiment of this application.

[0029] Figure 3F This is another flowchart illustrating the process of determining the answer to a question provided in an embodiment of this application;

[0030] Figure 3G This is a flowchart illustrating the process of determining a reference problem as provided in an embodiment of this application;

[0031] Figure 3H This is a flowchart illustrating the process of determining the first target problem provided in an embodiment of this application;

[0032] Figure 3I This is a flowchart illustrating the process of determining a trained question-answering model provided in an embodiment of this application;

[0033] Figure 4 This is a schematic diagram of the model iterative training process provided in the embodiments of this application. Detailed Implementation

[0034] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limitations on this application. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0035] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0036] In the following description, the terms "first, second, third" are used merely to distinguish similar objects and do not represent a specific ordering of objects. It is understood that "first, second, third" may be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0037] In this application embodiment, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0038] Unless otherwise defined, all technical and scientific terms used in the embodiments of this application have the same meaning as commonly understood by one of ordinary skill in the art. The terminology used in the embodiments of this application is for the purpose of describing the embodiments of this application only and is not intended to limit this application.

[0039] In the implementation of this application, the collection and processing of relevant data should strictly comply with the requirements of relevant laws and regulations, obtain the informed consent or separate consent of the personal information subject, and carry out subsequent data use and processing within the scope of laws and regulations and the authorization of the personal information subject.

[0040] Before providing a further detailed description of the embodiments of this application, the nouns and terms involved in the embodiments of this application will be explained, and the nouns and terms involved in the embodiments of this application shall be interpreted as follows.

[0041] 1) Synthetic Data: Training large language models requires massive amounts of data. In addition to natural data, corresponding data can be synthesized using the model and incorporated into the training process. This data is called synthetic data. Synthetic data can be of various types. In this embodiment, the synthetic data is a question type, referred to as a synthetic question.

[0042] 2) Large language model (LLM): A type of artificial intelligence model based on deep learning technology with a large number of parameters, designed to process and generate human language-related tasks.

[0043] 3) Mathematical task: It includes two parts: mathematical problem and mathematical answer.

[0044] 4) Problem generation: refers to the process of creating or conceiving a problem. In this embodiment, it refers to the process of generating corresponding mathematical problems based on a large language model.

[0045] 5) Answer generation: This refers to the process of creating or conceiving the correct answer based on a given problem or task. In this embodiment, it refers to the process of generating the corresponding answer to a given mathematical problem based on a large language model, which may include the solution process for the mathematical problem.

[0046] 6) Retrieval-augmented generation (RAG): An artificial intelligence technique that combines information retrieval and natural language generation (NLG) capabilities to improve the accuracy and relevance of generated content. In this embodiment, it refers to the large language model retrieving other questions and answers related to the input mathematical question during the answer generation process and inputting them into the large language model to assist it in generating more accurate and factual answers.

[0047] 7) Difficulty Controller: A system used in educational and training environments that dynamically adjusts the difficulty of tasks based on the target's skill level and performance, aiming to provide an adaptive learning experience for the target and ensure that tasks are both challenging and achievable. In this embodiment, the difficulty-based controller can select more informative mathematical problems based on their difficulty and perform cyclical synthetic data generation to improve the relevance and efficiency of synthetic data generation.

[0048] In related technologies, large language models require both complex mathematical semantic understanding and multi-step reasoning capabilities when performing mathematical reasoning tasks. Therefore, mathematical reasoning tasks remain a significant challenge for large language models. Some well-known large language models consider mathematical tasks as one of their fundamental capabilities, and high-quality mathematical training data is the basic fuel for mathematical reasoning. However, due to the diversity and complexity of mathematical problems, high-quality natural mathematical data is relatively scarce. Therefore, the mathematical data available in related technologies falls far short of the needs of large language models in pre-training. Consequently, synthetic data is widely used to augment training data.

[0049] To address this, related technologies typically employ different strategies for generating synthetic problems, such as problem restatement, problem verification, and problem back-translation. The synthetic problem generation methods in these technologies mainly focus on improving the Supervised Fine-Tuning (SFT) stage using millions of synthetic mathematical data points, and have the following problems:

[0050] 1) Low synthesis efficiency: Existing synthesis problem generation strategies mostly rely on complex algorithms and large-scale computing resources. A lot of text processing and calculation are required during the generation process, and the time taken for a single synthesis is long, making it difficult to quickly meet the needs of large-scale data generation.

[0051] 2) Repetition in synthesis problems: Due to the limitations of the generation strategy and the similarity of data samples, repetition problems are likely to occur frequently when generating a large number of synthesis problems, resulting in data redundancy;

[0052] 3) For more difficult synthesis problems, it is impossible to obtain accurate synthesis answers: Complex synthesis problems often involve the integration of knowledge from multiple fields and deep nesting of logic. The methods in related technologies are difficult to generate answer generation logic that matches them, or cannot guarantee the accuracy and completeness of the synthesized answer;

[0053] 4) High labor costs: After the synthetic questions are generated, manual annotation is usually required to check the accuracy and rationality of the questions and answers. However, the synthetic data is large in volume and of varying quality. Annotators need to have professional knowledge and rich experience, which makes the annotation work difficult, time-consuming and labor costs high.

[0054] This application provides a question-answering model training method, apparatus, device, computer-readable storage medium, and computer program product. It can automatically generate multiple training datasets in multiple training iterations, thereby improving the training efficiency of the question-answering model and the question-answering accuracy of the trained model. The following describes exemplary applications of the electronic devices provided in this application. These electronic devices can be implemented as various types of terminals such as laptops, tablets, desktop computers, set-top boxes, smartphones, smart speakers, smartwatches, smart TVs, and in-vehicle terminals, or as servers. The following describes exemplary applications when the device is implemented as a server.

[0055] See Figure 1 , Figure 1 This is a schematic diagram of the structure of the model training system provided in the embodiments of this application. Figure 1 The system involves a database 100, a server 200, a network 300, and a terminal 400. Terminal 400 connects to server 200 via network 300, which can be a wide area network (WAN), a local area network (LAN), or a combination of both. The trained question-answering model can be stored in database 100, which can be independent of server 200 or deployed on server 200. Figure 1 The Sino-Israeli database 100 is shown independently of server 200. The trained question-answering model can be compressed using techniques such as pruning and deployed locally on terminal 400, or it can be deployed on server 200. Terminal 400 submits a question, and server 200 generates an answer using the trained question-answering model and returns it to terminal 400.

[0056] When model training is required, a model training trigger operation can be performed through terminal 400. In response to the trigger operation, terminal 400 generates a corresponding model training instruction and sends it to server 200 via network 300. Upon receiving the model training instruction, server 200 begins iterative training of the question-answering model to be trained. Specifically, for each training iteration, server 200 determines the sample question set required for the current training iteration of the question-answering model, and generates a first training dataset based on the sample question set. The first training dataset includes multiple synthetic questions and their corresponding answers. Then, based on the first training dataset, server 200 performs the current training iteration on the question-answering model obtained from the previous training iteration, obtaining the question-answering model after the current training iteration. Based on the question-answering model after the current training iteration, it determines a first target question from the sample question set and multiple synthetic questions. The first target question is the question that the question-answering model failed to answer in the current training iteration. Afterwards, server 200 determines the second training dataset for the next training iteration based on the first target question, and uses the second training dataset to train the question answering model after the current training iteration until the training termination condition is met, thereby obtaining the trained question answering model and storing it in database 100.

[0057] Taking the server mentioned above as an example, which is the electronic device used for model training, see [link to relevant documentation]. Figure 2 , Figure 2 This is a schematic diagram of the structure of the server 200 provided in the embodiments of this application. Figure 2 The server 200 shown includes at least one processor 210, memory 230, and at least one network interface 220. The various components of server 200 are coupled together via a bus system 240. It is understood that the bus system 240 is used to implement communication between these components. In addition to a data bus, the bus system 240 also includes a power bus, a control bus, and a status signal bus. However, for clarity, ... Figure 2 The general labeled all buses as Bus System 240.

[0058] Processor 210 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc., wherein the general-purpose processor can be a microprocessor or any conventional processor, etc.

[0059] The memory 230 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state memory, hard disk drives, optical disk drives, etc. The memory 230 may optionally include one or more storage devices physically located away from the processor 210.

[0060] The memory 230 may include volatile memory or non-volatile memory, or both. The non-volatile memory may be read-only memory (ROM), and the volatile memory may be random access memory (RAM). The memory 230 described in this application embodiment is intended to include any suitable type of memory.

[0061] In some embodiments, memory 230 is capable of storing data to support various operations, examples of which include programs, modules, and data structures or subsets or supersets thereof, as illustrated below.

[0062] Operating system 231 includes system programs for handling various basic system services and performing hardware-related tasks, such as the framework layer, core library layer, driver layer, etc., for implementing various basic business functions and handling hardware-based tasks;

[0063] The network communication module 232 is used to reach other electronic devices via one or more (wired or wireless) network interfaces 220, exemplary network interfaces 220 including Bluetooth, WiFi, and Universal Serial Bus (USB).

[0064] In some embodiments, the apparatus provided in this application can be implemented in software. Figure 2 A model training apparatus 233 stored in memory 230 is shown. This apparatus can be software in the form of programs and plugins, and includes the following software modules: a determination module 2331 and a training module 2332. These modules are logically linked and can therefore be arbitrarily combined or further separated according to their implemented functions. The functions of each module will be described below.

[0065] In other embodiments, the apparatus provided in this application can be implemented in hardware. For example, the apparatus provided in this application can be a processor in the form of a hardware decoding processor, which is programmed to execute the question-answering model training method provided in this application. For example, the processor in the form of a hardware decoding processor can be one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), or other electronic components.

[0066] The question-answering model training method provided in this application will be described in conjunction with exemplary applications and implementations of the terminals provided in the embodiments of this application.

[0067] The question-answering model training method provided in the embodiments of this application will be described below. As mentioned above, the electronic device implementing the question-answering model training method in the embodiments of this application can be a terminal, a server, or a combination of both. Therefore, the executing entity of each step will not be described again below.

[0068] It should be noted that the model training examples below are illustrated using a question-answering model as an example. Those skilled in the art can apply the question-answering model training method provided in the embodiments of this application to the model training process of other types of models based on their understanding of the following text.

[0069] See Figure 3A , Figure 3A This is a flowchart illustrating the question-answering model training method provided in this application embodiment, which will be combined with... Figure 3A The steps shown are explained.

[0070] In step 101, the sample question set required for the training iteration process of the question answering model is determined, and the first training dataset for the current training iteration process is generated based on the sample question set.

[0071] Here, the training of the model in this embodiment includes multiple training iterations, and the current training iteration refers to the currently ongoing training iteration. The sample problem set consists of representative and diverse sample problems required for training, used to generate the first training dataset. The first training dataset is used to train the solution model and includes multiple synthetic problems and the corresponding answers for each synthetic problem. A synthetic problem refers to a new problem generated by combining, transforming, or deriving existing sample problems from the sample problem set. The answer is the solution to the synthetic problem and may include both the answer and the solution process. It should be noted that the types of problems in this embodiment can be various, such as social problems, philosophical problems, technical problems, and problems from different disciplines (mathematical problems, physical problems, historical problems, chemical problems), etc. The following explanation mainly uses a mathematical problem as an example.

[0072] In some embodiments, when the current training iteration is the first training iteration, see [link to documentation]. Figure 3B The step 101, "determining the set of sample questions required for the current training iteration of the question-answering model," can be achieved through steps 1011A to 1014A, including:

[0073] In step 1011A, a set of candidate questions is obtained.

[0074] Here, the candidate problem set includes multiple candidate problems obtained by preprocessing the initial problem set. The initial problem set comprises numerous initial problems collected from various data sources, typically numbering in the millions. These mathematical problems collected from different data sources essentially cover most practical mathematical tasks. By preprocessing each initial problem in the initial problem set, multiple candidate problems are obtained and constructed into a candidate problem set. The purpose of preprocessing is to ensure the quality and applicability of these mathematical problems. For example, preprocessing may include data cleaning, handling missing values, and data deduplication. If the collected initial problems include mathematical problems in multiple different language types, then translation processing is also required during preprocessing. For example, translating mathematical problems expressed in English into Chinese, or vice versa, ultimately ensuring linguistic consistency among the multiple candidate problems in the candidate problem set.

[0075] In some embodiments, see Figure 3C The candidate problem set can be obtained through steps 201 to 204, including:

[0076] In step 201, the initial problem set is obtained.

[0077] Here, the initial problems set includes initial problems in multiple languages. Multiple languages ​​refer to several different languages, such as Chinese, English, Spanish, French, Russian, Arabic, etc. The initial problems are collected from multiple data sources, typically numbering in the millions. These mathematical problems collected from different data sources essentially cover most practical mathematical tasks.

[0078] In step 202, the initial problem set is cleaned to obtain a cleaned initial problem set.

[0079] The purpose of cleaning the initial question set is to remove noisy, erroneous, or irrelevant questions, making the questions more accurate, standardized, and effective. During the cleaning process, it's possible to check for missing parts in the initial questions, such as incomplete questions or missing key information. For missing information that can be supplemented, attempts are made to fill in the gaps using other sources or context; if the missing information cannot be supplemented and the question is severely incomplete and affects usability, then the question should be deleted. The cleaning process can also correct spelling and grammar errors, remove irrelevant or low-quality questions, thereby improving the quality of the initial question set and making it more suitable for subsequent analysis and training.

[0080] In step 203, for each initial problem in the cleaned initial problem set, the problems that are the same as the initial problems are identified and deleted, resulting in a deleted initial problem set.

[0081] Here, to remove duplicate problems from the cleaned initial problem set, it is necessary to identify and delete similar problems for each initial problem. This can be done by converting the initial problems into vector representations and then calculating the cosine of the similarity between the vectors corresponding to two initial problems. The closer the similarity is to 1, the more similar the two initial problems are. A similarity threshold, such as 0.8, can be set; when the similarity between two problems exceeds this threshold, they are considered similar and deduplication is performed.

[0082] In step 204, the initial questions in the deleted initial question set, excluding each preset language type, are translated to obtain a candidate question set.

[0083] Here, the preset language type refers to a pre-determined language, such as Chinese, English, or French. Since the initial question set includes questions in multiple language types, the deleted initial question set also includes questions in multiple language types. Therefore, it is necessary to translate each initial question outside the preset language type into the preset language type, resulting in a candidate question set. At this point, the candidate question set contains multiple candidate questions in the preset language type; that is, each candidate question in the candidate question set is in the preset language type.

[0084] In this embodiment, an initial question set is obtained, which includes initial questions in multiple language types. The initial question set is then cleaned to obtain a cleaned initial question set. For each initial question in the cleaned initial question set, questions identical to the initial question are identified and deleted, resulting in a deleted initial question set. Each initial question in the deleted initial question set that is not in a preset language type is translated to obtain a candidate question set, which includes multiple candidate questions in preset language types. This approach, by collecting initial questions in multiple languages, greatly enriches the diversity of the initial questions and expands the coverage of the final candidate question set. Furthermore, by cleaning and deduplicating the initial questions, errors or redundancy in the candidate question set can be avoided, making the candidate question set more accurate, complete, and reliable.

[0085] In step 1012A, based on the problem category of each candidate problem in the candidate problem set, multiple candidate problems are clustered to obtain multiple problem clusters.

[0086] Here, the problem category can refer to the grade level (e.g., primary school, middle school, high school, university) corresponding to the candidate problem, or the test point (e.g., quadratic equations, functions, analytic geometry, trigonometry, sequences, etc.) corresponding to the candidate problem, or the solution method (calculation problem, application problem, inquiry problem, etc.). Understandably, the problem category can be set adaptively according to different candidate problems; the specific setting method for the problem category in this embodiment is not limited. Since the candidate problems are existing problems collected from reality, the corresponding problem category is usually determined during the collection process. Alternatively, the problem category corresponding to each candidate problem can be determined by inputting the candidate problems into a classification model. After determining the problem category corresponding to each candidate problem, multiple candidate problems are clustered according to the problem category to obtain multiple problem clusters. Among them, multiple candidate problems in each problem cluster correspond to the same problem category.

[0087] In step 1013A, the first target sample of the first number is determined from each problem cluster.

[0088] Here, the first number is predetermined, and the first number corresponding to different problem clusters can be the same or different. The first target sample is the candidate problem determined from the problem cluster.

[0089] In some embodiments, the first number can be determined by the following process:

[0090] For each problem cluster, determine the first difficulty level corresponding to each candidate problem in the problem cluster; based on each first difficulty level, determine the second difficulty level corresponding to the problem cluster; determine the sampling ratio corresponding to the second difficulty level, the sampling ratio being positively correlated with the second difficulty level; based on the sampling ratio and the total number of problems in the problem cluster, determine the first number.

[0091] Here, the first difficulty level describes the difficulty of the candidate problem. The difficulty level can be numerical (e.g., an integer between 0 and 10) or percentage (e.g., 60%, 78%, etc.). A higher difficulty level indicates a more difficult candidate problem. The first difficulty level can be determined based on the problem category. For example, if candidate problem 1 is categorized as elementary school and candidate problem 2 as university, then the first difficulty level for candidate problem 1 would be 3, and for candidate problem 2, it would be 8. Alternatively, the first difficulty level can be determined based on the number of knowledge points included in the candidate problem. For example, if candidate problem 1 involves only one basic and common mathematical knowledge point, and candidate problem 2 involves multiple medium-difficulty knowledge points, then the first difficulty level for candidate problem 1 would be 1, and for candidate problem 2, it would be 9. Furthermore, the first difficulty level can also be determined based on the number of solution steps for each candidate problem. If the first difficulty level is determined based on the problem category, then each candidate problem within the same problem cluster has the same first difficulty level. In this case, the first difficulty level of the candidate problem can be directly determined as the second difficulty level of the problem cluster. If the first difficulty levels of candidate questions within the same problem cluster are not equal, the second difficulty level of the problem cluster can be determined based on the average or mode of each first difficulty level. The sampling ratio refers to the proportion between the number of the first target samples to be extracted from the problem cluster and the total number of candidate questions in the cluster, such as 50% or 70%. The sampling ratio is positively correlated with the second difficulty level. If the second difficulty level is a percentage, it can be directly used as the sampling ratio. If the second difficulty level is a numerical value, such as an integer between 0 and 10, it can be divided by the largest second difficulty level, and the result is used as the sampling ratio. For example, if the second difficulty level is 5, 5 ÷ 10 = 50%, so the sampling ratio is 50%. Then, the product of the sampling ratio and the total number of questions in the problem cluster is used as the first number. For example, if the sampling ratio for problem cluster 1 is 20% and the total number of questions in cluster 1 is 10,000, then the first number for cluster 1 is 20% × 10,000 = 2,000.

[0092] In this embodiment, for each problem cluster, a first difficulty level is determined for each candidate problem within the cluster; a second difficulty level is determined based on each first difficulty level; a sampling ratio is determined corresponding to the second difficulty level, and the sampling ratio is positively correlated with the second difficulty level; a first number is determined based on the sampling ratio and the total number of problems in the problem cluster. In this way, the first number corresponding to each problem cluster can be adaptively determined based on the total number of problems in each cluster and the corresponding second difficulty level. For more difficult problem clusters, the determined first number can be larger, thereby improving the flexibility and relevance of determining the first number.

[0093] In some embodiments, the first target sample of the first number can be determined from each problem cluster by the following process:

[0094] For each problem cluster, based on the first difficulty level corresponding to each candidate problem in the problem cluster, each candidate problem is sorted in descending order to obtain a sorted problem sequence; the first candidate problem in the problem sequence is determined as the first target sample of the first number.

[0095] Here, for each problem cluster, each candidate problem is sorted in descending order based on its first difficulty level. For example, if candidate problem 1 has a first difficulty level of 2, candidate problem 2 has a first difficulty level of 8, and candidate problem 3 has a first difficulty level of 5, then the sorted problem sequence is: candidate problem 2, candidate problem 3, candidate problem 1. Then, the candidate problem of the first number in the problem sequence is determined as the first target sample of that number. For example, if the first number is 2, and the sorted problem sequence is: candidate problem 2, candidate problem 3, candidate problem 1, then the first target sample includes: candidate problem 2 and candidate problem 3.

[0096] In this embodiment, for each problem cluster, each candidate problem is sorted in descending order based on its first difficulty level, resulting in a sorted problem sequence. The first candidate problem in the problem sequence is then identified as the first target sample. This ensures that more difficult candidate problems are prioritized from the problem cluster as the first target samples, improving the quality and effectiveness of determining the first target samples and thus increasing the efficiency of subsequent model training.

[0097] In step 1014A, the sample problem set required for the current training iteration process is constructed using the first target sample of the first number corresponding to each problem cluster.

[0098] Here, the first target samples of the first number determined in each problem cluster are combined together to construct the sample problem set required for the current training iteration process.

[0099] In this embodiment, when the current training iteration is determined to be the first training iteration, a candidate problem set is obtained. This candidate problem set includes multiple candidate problems obtained by preprocessing the initial problem set. Based on the problem category of each candidate problem in the candidate problem set, the multiple candidate problems are clustered to obtain multiple problem clusters. A first target sample is determined from each problem cluster. Using the first target sample corresponding to the first number of each problem cluster, a sample problem set required for the current training iteration is constructed. This ensures that a subset of candidate problems are specifically identified from each problem cluster as the first target samples, and the sample problem set required for the current training iteration is constructed, improving the data richness and reliability of the sample problem set, thereby improving the efficiency of model training.

[0100] In some embodiments, when the current training iteration is the k-th training iteration, where k is an integer greater than 1, see [reference needed]. Figure 3D The step 101, "determining the set of sample questions required for the current training iteration of the question-answering model," can be achieved through steps 1011B to 1015B, including:

[0101] In step 1011B, the second objective problem and the objective problem cluster corresponding to the (k-1)th training iteration are obtained.

[0102] Here, when the current training iteration is the k-th iteration (where k is an integer greater than 1), it indicates that the current training iteration is not the first iteration. For example, the current training iteration could be the 3rd or 5th iteration. In this case, the second target question corresponding to the (k-1)-th training iteration is obtained; that is, the question that the question-answering model failed to answer in the previous training iteration. The target question cluster corresponding to each second target question is then determined. The second target question can be a sample question from the previous training iteration or a synthesized question. If it is a sample question, the question cluster containing the sample question itself is determined as the target question cluster. If it is a synthesized question, the question cluster containing the sample question corresponding to the synthesized question is determined as the target question cluster. For example, if synthesized question 1 is generated based on sample question 1, then synthesized question 1 should belong to the same question category as sample question 1. Therefore, the question cluster containing sample question 1 can be determined as the target question cluster corresponding to the second target question.

[0103] In step 1012B, the third number is determined based on the second and first numbers of the second target problem corresponding to the target problem cluster.

[0104] Here, the second number refers to the number of second target problems corresponding to each target problem cluster, and the third number is the difference between the first and second numbers. For example, second target problem 1 corresponds to target problem cluster 1, second target problem 2 corresponds to target problem cluster 3, second target problem 3 corresponds to target problem cluster 1, second target problem 4 corresponds to target problem cluster 1, and second target problem 5 corresponds to target problem cluster 2. Therefore, the second number of the second target problems corresponding to target problem cluster 1 is 3, the second number of the second target problems corresponding to target problem cluster 2 is 1, and the second number of the second target problems corresponding to target problem cluster 3 is 1. The first number is determined during the first training iteration. For example, the first number corresponding to target problem cluster 1 is 5, the first number corresponding to target problem cluster 2 is 8, and the first number corresponding to target problem cluster 3 is 2. Therefore, the third number corresponding to target problem cluster 1 is 5-3=2, the third number corresponding to target problem cluster 2 is 8-1=7, and the third number corresponding to target problem cluster 3 is 2-1=1.

[0105] In step 1013B, the second target sample of the third number is determined from the target problem cluster.

[0106] Here, from each target problem cluster, the third candidate problem is selected as the second target sample. Specifically, each candidate problem in the target problem cluster can be sorted in descending order based on its first difficulty level, resulting in a sorted problem sequence. Then, the third candidate problem in the first three parts of this problem sequence is selected as the third target sample.

[0107] In step 1014B, the third target sample of the first number is determined from each of the other problem clusters.

[0108] Here, "other question clusters" refers to question clusters other than the target question cluster. A first number of third target samples are determined from each of these other question clusters. Since no second target question belongs to any of the other question clusters (meaning they do not contain questions the question-answering model failed to answer in the previous training iteration), the questions in these other question clusters can be considered to be of lower difficulty. Therefore, in some embodiments, fewer than the first number of candidate questions can be determined from each of these other question clusters as the third target samples.

[0109] In step 1015B, the sample problem set required for the current training iteration process is constructed using the second target problem, the second target sample, and the third target sample.

[0110] Here, each second objective question, second objective sample, and third objective sample is combined to construct the sample question set required for the current training iteration. Alternatively, a subset of the second objective questions, second objective samples, and third objective samples can be combined to construct the sample question set required for the current training iteration.

[0111] In this embodiment, when the current training iteration is determined to be the k-th training iteration, the second target problem and the target problem cluster corresponding to the (k-1)-th training iteration are obtained, where k is an integer greater than 1. Based on the second and first numbers of the second target problem corresponding to the target problem cluster, the third number is determined. The second target sample of the third number is determined from the target problem cluster. The third target sample of the first number is determined from each of the other problem clusters, where the other problem clusters are problem clusters other than the target problem cluster. Using the second target problem, the second target sample, and the third target sample, the sample problem set required for the current training iteration is constructed. Thus, when the current training iteration does not belong to the first training iteration, the second target problem that the question-answering model failed to solve in the previous training iteration, as well as the newly determined second and third target samples from the problem clusters, can be used to construct the sample problem set required for the current training iteration. Therefore, the sample problem set of each training iteration can be associated with the mathematical capabilities of the continuously evolving question-answering model, enabling the model training direction to more specifically address unsolved mathematical problems, thereby improving the model training efficiency and relevance.

[0112] In some embodiments, the sample problem set includes multiple sample problems corresponding to each problem cluster. The step 101, "generating the first training dataset for the current training iteration based on the sample problem set," can be implemented through the following process:

[0113] Multiple synthetic questions are generated based on multiple sample questions corresponding to each question cluster, and the answer to each synthetic question is determined. Each synthetic question and its corresponding answer are then used as the first training dataset for the current training iteration.

[0114] Here, composite problems can be obtained by changing the numbers and parameters in the original sample problems. Alternatively, composite problems can be obtained by altering the problem context, expanding the depth and breadth of the sample problems, adding other conditions to make them more complex, or combining multiple related problems into comprehensive problems. Furthermore, composite problems can be obtained by changing the type of sample problems—for example, turning calculation problems into word problems, or using reverse thinking to generate new problems from the conditions and conclusions of the sample problems. After obtaining the composite problems, each composite problem is solved to obtain its corresponding answer. Finally, each composite problem and its corresponding answer are used as the first training dataset for the current training iteration.

[0115] In this embodiment, the sample question set includes multiple sample questions corresponding to each question cluster. Multiple synthetic questions are generated based on the multiple sample questions corresponding to each question cluster, and the answer to each synthetic question is determined. Each synthetic question and its corresponding answer are defined as the first training dataset for the current training iteration. This improves the data diversity of the first training dataset, thereby increasing the efficiency of training the question-answering model using the first training dataset.

[0116] In some embodiments, a trained synthesis processing model can be used to generate multiple synthetic questions based on multiple sample questions corresponding to each question cluster, including: obtaining a trained synthesis processing model; generating prompt words for each question cluster based on multiple sample questions corresponding to the question cluster; and generating multiple synthetic questions based on the prompt words using the trained synthesis processing model.

[0117] Here, the trained synthesis processing model is a pre-trained model used to generate synthetic questions based on sample questions. For each question cluster corresponding to multiple sample questions, a prompt word corresponding to that question cluster is generated. This prompt word is then input into the trained synthesis processing model, which will generate and output multiple synthetic questions based on the prompt word. For example, the prompt word could be: Referring to the given questions below, generate 10 similar elementary probability questions: Sample Question 1, Sample Question 2; Ensure the new questions conform to common sense; Ensure the new questions can be answered without the given questions.

[0118] In this embodiment, a trained synthesis processing model is obtained; for each question cluster, prompt words are generated based on multiple sample questions corresponding to the question cluster; based on the prompt words, multiple synthesized questions are generated using the trained synthesis processing model. In this way, generating synthesized questions through a trained synthesis processing model can improve the accuracy and efficiency of generating synthesized questions.

[0119] In some embodiments, see Figure 3E The answer to each synthesis question can be determined through steps 301 to 304, including:

[0120] In step 301, for each synthesis question, multiple first question-answer results corresponding to the synthesis question are determined.

[0121] Here, each synthesis question is input into multiple different large language models, which then perform question-and-answer operations on the synthesis question, resulting in multiple first-response results. These first-response results may be the same or different. A first-response result can be the model's solution process and answer to the synthesis question, or it can be a message indicating that the model cannot answer the synthesis question.

[0122] In step 302, at least one first answer is obtained from the first question-and-answer results that represent a successful answer.

[0123] Here, if the first question-and-answer result is the model's solution process and answer to the synthesized question, then the first question-and-answer result represents a successful solution. The first answer refers to the final answer given for the synthesized question in the first question-and-answer result representing a successful solution. Since different models may give different answers to the same question, there may be multiple different first answers in the first question-and-answer result representing a successful solution. For example, for synthesized question 1, if model 1 outputs answer A, model 2 outputs answer B, and model 3 outputs answer A, then the first answers include A and B.

[0124] In step 303, based on the first question-and-answer result representing a successful answer, a first percentage of each first answer is determined, and based on each first percentage, a first maximum percentage is determined.

[0125] Here, the first percentage represents the proportion of each first answer among all first-answer results, and the first maximum percentage is the largest first percentage among all first percentages. For example, if there are 10 first-answer results indicating a successful answer, with the corresponding answers being: Answer A, Answer A, Answer B, Answer A, Answer A, Answer A, Answer B, Answer A, Answer A, Answer A, Answer A, Answer A; then the first answers include Answer A and Answer B. Answer A appears 8 times, corresponding to a first percentage of 8 / 10 = 80%, and Answer B appears 2 times, corresponding to a first percentage of 2 / 10 = 20%. Therefore, the first maximum percentage is 80%.

[0126] In step 304, when the first maximum percentage is greater than or equal to the percentage threshold, the first answer corresponding to the first maximum percentage is determined as the answer to the question.

[0127] Here, the percentage threshold is preset and can be flexibly adjusted according to actual needs, usually greater than 50%. When the first maximum percentage is greater than or equal to the percentage threshold, it means that the probability of the first answer corresponding to the first maximum percentage being the correct answer is high. At this time, the first answer corresponding to the first maximum percentage can be determined as the answer to the question.

[0128] In this embodiment, for each synthesized question, multiple first question-and-answer results corresponding to the synthesized question are determined; at least one first answer is obtained from the first question-and-answer results representing successful answers; based on the first question-and-answer results representing successful answers, a first proportion of each first answer is determined, and based on each first proportion, a first maximum proportion is determined; when the first maximum proportion is greater than or equal to a proportion threshold, the first answer corresponding to the first maximum proportion is determined as the question answer. This improves the accuracy and efficiency of determining the question answer corresponding to each synthesized question.

[0129] In some embodiments, when multiple first question-answer results all indicate a failed answer, or when the first maximum proportion is less than a proportion threshold, it indicates that no relatively accurate answer to the synthesized question has been determined. In this case, see [link to relevant documentation]. Figure 3F Furthermore, steps 401 to 403 can be used to determine the answer to the synthesized question, including:

[0130] In step 401, a reference problem related to the synthesis problem and a reference answer corresponding to the reference problem are determined.

[0131] Here, the reference problem is a high-quality problem similar to the composition problem, and the reference answer is the exact answer corresponding to the reference problem.

[0132] In some embodiments, see Figure 3G Step 401 can be implemented through steps 4011 to 4013, including:

[0133] In step 4011, a retrieval process is performed based on the synthesis problem to obtain multiple retrieval results.

[0134] Here, the search results include candidate questions and their corresponding answers. You can first extract the key concepts, terms, or numbers from the composite question and use them as keywords to search for candidate questions related to the composite question. Then, retrieve the answers for each candidate question to obtain multiple search results.

[0135] In step 4012, the correlation results for each candidate problem and the synthesis problem are determined.

[0136] Here, the correlation result is used to characterize the degree of correlation or similarity between the candidate problem and the composite problem. Specifically, the similarity between the candidate problem and the composite problem can be calculated as the correlation result.

[0137] In step 4013, a reference problem is determined from multiple candidate problems based on the correlation results.

[0138] Here, the relevance result of the reference problem is greater than the relevance threshold, which is predetermined and can be adjusted according to actual needs. First, target candidate problems with relevance results greater than the relevance threshold are identified. If the number of target candidate problems is less than the quantity threshold, these target candidate problems are directly designated as reference problems. If the number of target candidate problems is greater than or equal to the quantity threshold, the target candidate problems need further screening to select the target candidate problems that meet the quantity threshold, and these are then designated as reference problems.

[0139] In this embodiment, a retrieval process is performed based on the synthesized question to obtain multiple retrieval results, including candidate questions and their corresponding answers. The relevance between each candidate question and the synthesized question is determined. Based on the relevance results, a reference question is determined from the multiple candidate questions, where the relevance of the reference question is greater than a relevance threshold. This improves the efficiency and accuracy of determining the reference question.

[0140] In step 402, based on the synthesis question, the reference question, and the reference answer, at least one second answer corresponding to the synthesis question is determined, and a second proportion of each second answer is determined.

[0141] Here, the synthesized question is again input into multiple different large language models, with prompts indicating that the models should use reference questions and answers as reference data to answer the synthesized question. At this point, the multiple large language models will use the reference questions and answers as references to answer the synthesized question, thus obtaining at least one second answer. Then, the second percentage of each second answer among all question-answering results representing successful question-answering is determined.

[0142] In step 403, a second maximum percentage is determined based on each second percentage. When the second maximum percentage is greater than or equal to the percentage threshold, the second answer corresponding to the second maximum percentage is determined as the answer to the question.

[0143] Here, the second largest percentage is the second largest percentage among all second percentages. The percentage threshold is preset and can be flexibly adjusted according to actual needs, usually greater than 50%. When the second largest percentage is greater than or equal to the percentage threshold, it means that the second answer corresponding to the second largest percentage has a high probability of being the correct answer. At this time, the second answer corresponding to the second largest percentage can be determined as the answer to the question.

[0144] In this embodiment, when multiple first question-and-answer results indicate failure to answer, or when the first maximum percentage is less than a percentage threshold, a reference question related to the synthesized question and a reference answer corresponding to the reference question are determined. Based on the synthesized question, the reference question, and the reference answer, at least one second answer corresponding to the synthesized question is determined, and a second percentage for each second answer is determined. Based on each second percentage, a second maximum percentage is determined. When the second maximum percentage is greater than or equal to the percentage threshold, the second answer corresponding to the second maximum percentage is determined as the question answer. Thus, by using reference questions and reference answers, the success rate and accuracy of determining the question answer can be further improved.

[0145] In some embodiments, when the second maximum proportion is less than the proportion threshold, the synthesized question is sent to the terminal; the processing result sent by the terminal based on the synthesized question is received, and the processing result is determined as the answer to the question.

[0146] Here, if the second largest proportion is still less than the proportion threshold, it indicates that the current large language model cannot answer the synthesis question. In this case, the synthesis question needs to be sent to the terminal for manual answering. Then, the receiving terminal sends the processing result based on the synthesis question, and the processing result is determined as the answer. In this way, even for a few synthesis questions that the model cannot answer, the corresponding answer can still be determined manually, improving the reliability of determining the answer to the synthesis question.

[0147] In step 102, based on the first training dataset, the question-answering model obtained in the previous training iteration is used for the current training iteration to obtain the question-answering model after the current training iteration.

[0148] Here, the question-answering model obtained in the previous training iteration refers to the question-answering model trained in the previous training iteration, i.e., the question-answering model after the previous training iteration. The question-answering model after the current training iteration refers to the question-answering model obtained through the current training iteration. If the current training iteration is the first training iteration, then the question-answering model obtained in the previous training iteration is the initial question-answering model to be trained. For each synthetic question in the first training dataset, the synthetic question is input into the question-answering model obtained in the previous training iteration, so that the question-answering model obtained in the previous training iteration can answer the synthetic question, thus obtaining the answer result corresponding to each synthetic question. Then, based on the answer to each question and each answer result in the first training dataset, the model loss value is determined. This loss value is backpropagated to the question-answering model obtained in the previous training iteration. Based on the loss value, the parameters of the question-answering model obtained in the previous training iteration are adjusted using the gradient descent algorithm to obtain the question-answering model after the current training iteration.

[0149] In step 103, based on the question-answering model after the current training iteration, the first target question is determined from the sample question set and multiple synthetic questions.

[0150] Here, the first target problem is the problem that the question-answering model fails to solve after the current training iteration. Specifically, each sample problem and each synthesized problem in the sample problem set are input into the question-answering model after the current training iteration, and the problems that the model fails to solve are identified as the first target problem.

[0151] In some embodiments, see Figure 3H Step 103 can be implemented through steps 1031 to 1033, including:

[0152] In step 1031, for each sample question in the sample question set, the sample question is processed based on the question-answering model after the current training iteration to obtain the third question-answering result corresponding to the sample question.

[0153] Here, for each sample question in the sample question set, the sample question is input into the question-answering model after the current training iteration, causing the model to answer the sample question and obtain the third question-answer result. The third question-answer result can be either a successful answer or a failed answer. A successful answer indicates that the question-answering model after the current training iteration has output an answer to the sample question, but the answer is not necessarily correct.

[0154] In step 1032, for each of the multiple synthesis questions, the synthesis question is processed based on the question-answering model after the current training iteration to obtain the fourth question-answering result corresponding to the synthesis question.

[0155] Here, for each of the multiple synthetic questions, the synthetic question is input into the question-answering model after the current training iteration, enabling the model to answer the synthetic question and obtain the fourth question-answer result. The fourth question-answer result can be either a successful answer or a failed answer.

[0156] In step 1033, the sample question corresponding to the third question-and-answer result that represents a failed or incorrect answer, and the synthetic question corresponding to the fourth question-and-answer result that represents a failed or incorrect answer, are determined as the first target question.

[0157] Here, the corresponding question answers are usually collected simultaneously with the sample questions. Therefore, when the third question-answer result indicates a successful solution, it can be compared with the collected question answers; if they are inconsistent, the solution is incorrect. For the synthetic question, when the fourth question-answer result indicates a successful solution, it can be compared with the question answers corresponding to the synthetic question in the first training dataset; if they are inconsistent, the solution is incorrect. Then, the sample questions corresponding to the third question-answer results indicating failed or incorrect solutions, and the synthetic questions corresponding to the fourth question-answer results indicating failed or incorrect solutions, are all designated as the first target questions.

[0158] In this embodiment, for each sample question in the sample question set, the sample question is processed based on the question-answering model after the current training iteration to obtain the third question-answering result corresponding to the sample question; for each synthetic question among multiple synthetic questions, the synthetic question is processed based on the question-answering model after the current training iteration to obtain the fourth question-answering result corresponding to the synthetic question; the sample question corresponding to the third question-answering result representing a failed or incorrect answer, and the synthetic question corresponding to the fourth question-answering result representing a failed or incorrect answer, are determined as the first target question. In this way, the question-answering model after the current training iteration can filter out questions that the model cannot currently answer from the sample question set and multiple synthetic questions as the first target question, thereby improving the efficiency and accuracy of determining the first target question.

[0159] In step 104, based on the first target question, the second training dataset for the next training iteration is determined, and the question-answering model after the current training iteration is trained using the second training dataset until the training termination condition is met, thus obtaining the trained question-answering model.

[0160] Here, the first target question and question cluster are determined, and the sample question set required for the next training iteration is determined. Based on the sample question set required for the next training iteration, a second training dataset for the next training iteration is generated. Then, the question-answering model after the current training iteration is trained using the second training dataset until the training termination condition is met, resulting in a trained question-answering model. The training termination condition can be determined based on the number of training iterations or the current performance of the question-answering model.

[0161] In some embodiments, step 104, "training the question-answering model after the current training iteration using the second training dataset until the training termination condition is met, and obtaining the trained question-answering model," can be implemented through the following process, including:

[0162] Using the second training dataset, the question-answering model after the current training iteration is trained to obtain the question-answering model after the next training iteration; the number of training iterations corresponding to the question-answering model after the next training iteration is determined; when the number of training iterations reaches the preset number, the training termination condition is determined, and the question-answering model after the next training iteration is determined to be the trained question-answering model.

[0163] Here, the preset number of iterations is a pre-set number of training iterations, such as 3, 5, or 10. When the preset number of training iterations is reached, the training termination condition can be determined, and the question-answering model after the next training iteration is then selected as the trained question-answering model. In this way, the number of training iterations can be set according to actual needs, improving the model's generalization ability and avoiding overfitting, thus improving the efficiency of model training and the performance of the trained question-answering model.

[0164] In some embodiments, see Figure 3I The trained question-answering model can also be determined through steps 501 to 504, including:

[0165] In step 501, the verification dataset is obtained.

[0166] Here, the validation dataset includes multiple validation questions and the corresponding label data for each validation question.

[0167] In step 502, for each verification question, the verification question is answered based on the question-answering model after the next training iteration, and the fifth question-answering result of the verification question is obtained.

[0168] Here, each verification question is input into the question-answering model after the next training iteration, so that the question-answering model after the next training iteration can answer the verification question and obtain the fifth question-answer result corresponding to the verification question.

[0169] In step 503, based on the fifth question-answer result and label data corresponding to each verification question, the verification result of the question-answering model after the next training iteration is determined.

[0170] Here, the validation result of the question-answering model after the next training iteration is determined based on the difference between the fifth question-answer result for each validation question and the corresponding label data. For example, if the fifth question-answer result for the validation question is the same as the corresponding label data, or the difference is less than a preset difference, the validation result is determined to be successful; if the fifth question-answer result for the validation question is different from the corresponding label data, or the difference is greater than a preset difference, the validation result is determined to be unsuccessful.

[0171] In step 504, when the verification result characterization passes the verification, the question-answering model after the next training iteration is determined to be the trained question-answering model.

[0172] Here, when the validation result indicates that the validation has passed, it means that the model's performance has reached the preset standard. At this point, the question-answering model after the next training iteration is determined to be a well-trained question-answering model. In this way, the performance of the question-answering model can be validated by validation questions, and model training can be stopped when the model performance reaches the standard, resulting in a well-trained question-answering model with satisfactory performance, thus improving the efficiency of model training.

[0173] In this embodiment, a sample question set required for the current training iteration of the question-answering model is determined. A first training dataset for the current training iteration is generated based on the sample question set. The first training dataset includes multiple synthetic questions and their corresponding answers. Based on the first training dataset, the question-answering model obtained in the previous training iteration is used for the current training iteration to obtain the question-answering model after the current training iteration. In this way, in each training iteration, multiple synthetic questions and answers can be generated to perform the current training iteration on the question-answering model obtained in the previous training iteration, thereby ensuring that multiple training datasets can be generated cyclically in multiple training iterations, improving the quality and richness of the training data, and improving the efficiency and accuracy of model training. Then, based on the question-answering model after the current training iteration, a first target question is determined from the sample question set and multiple synthetic questions. The first target question is the question that the question-answering model failed to answer in the current training iteration. Based on the first target question, a second training dataset for the next training iteration is determined, and the question-answering model after the current training iteration is trained using the second training dataset. This ensures that the training dataset for each training iteration includes the questions the model failed to answer in the previous iteration. This allows for targeted training of the model each time it is trained, fully utilizing previously unsolved problems and further improving training efficiency. Finally, the training continues until the termination condition is met, resulting in a well-trained question-answering model that improves both question-answering accuracy and model performance.

[0174] Question-answering models are a common type of natural language processing model that understands user questions and provides corresponding answers. For example, in intelligent customer service systems, question-answering models are used to automatically respond to customer inquiries and provide 24 / 7 support, commonly found in e-commerce websites and banking services. Users send questions to the intelligent customer service system through websites, mobile applications, or instant messaging tools; the question-answering model deployed on the server receives the question, performs natural language understanding and processing, retrieves information from databases or knowledge bases, generates a relevant answer, and returns it to the user. In intelligent assistant applications, question-answering models act as intelligent assistants, answering various user questions such as weather forecasts and schedules. Users ask questions to the intelligent assistant via voice or text on the terminal; the question-answering model on the server receives and processes these questions, calling relevant services (such as weather information and schedule management services) to generate answers. In educational tutoring applications, question-answering models on online education platforms can help students resolve their learning difficulties. In this scenario, students ask questions via chat or voice input on an online platform. A question-answering model on the server analyzes the questions, extracts relevant information from an educational content library, and provides explanations or answers. The terminal is typically the user interface; users input questions, which are then sent to the server. The question-answering model running on the server processes these questions, potentially involving natural language processing, information retrieval, and knowledge base queries, before returning the generated answer to the terminal, completing the interaction between the user and the system.

[0175] The following will describe an exemplary application of the embodiments of this application in a real-world application scenario.

[0176] In the training process of question-answering models, question data is crucial training fuel, playing a decisive role in the final performance of the trained model. This application proposes an iterative model training framework and related algorithms that can continuously generate high-quality synthetic mathematical data (including synthetic questions, answers, and solution steps) to assist question-answering models in improving their mathematical reasoning abilities. This can be applied to the pre-training or post-training stages of large language models such as mixed-language models. Specifically, this application can automatically expand new high-quality synthetic mathematical data based on a certain initial question set and the large language model. This synthetic data will be used in the training of the question-answering model, ultimately improving its mathematical capabilities. This application mainly includes modules for diverse synthetic question generation, accurate question answer generation, and a difficulty-based loop controller. Furthermore, the question-answering model training method provided in this application is intelligently iterative based on question difficulty, capable of cyclically generating more synthetic data. The generated synthetic mathematical data can help improve the mathematical reasoning ability of the question-answering model.

[0177] In generating high-quality mathematical synthesis questions through a large language model, this application focuses on three key factors: (1) Difficulty of the synthesis question: In this application, how to construct more difficult synthesis questions is the key to generating synthesis questions. This is because the goal of this application is to improve the mathematical capability boundary of the current large language model. Therefore, it is necessary to determine the difficulty of each synthesis question and construct more challenging synthesis questions; (2) Accuracy of the answer: Generally speaking, for more difficult mathematical questions, the accuracy of the generated answer will decrease, which will seriously weaken the quality of the training data. Therefore, in order to ensure the quality of the training data, the accuracy of the answer generation must be guaranteed; (3) Diversity of the synthesis question: In order to provide a large amount of synthetic training data required for question answering model training, it is necessary to ensure the diversity of the generated synthesis questions.

[0178] like Figure 4 As shown, Figure 4 This is a schematic diagram of the model iterative training process provided in the embodiments of this application, embodying the concept of efficiently and economically generating synthetic mathematical data. Figure 4 In the first training iteration, the collected diverse initial questions are preprocessed to obtain a candidate question set 41, which is the fundamental source of mathematical problem diversity. Then, each candidate question in the candidate question set 41 is clustered to obtain multiple question clusters 42. Subsequently, in each training iteration, in the synthetic question generation module 43, based on the questions in the question clusters 42, diverse synthetic questions 44 are generated using a many-to-many mathematical question generation strategy (guided by enhanced diversity prompts, aiming to generate diverse synthetic mathematical questions). In the question answer generation module 45, multiple strategies such as retrieval enhancement, multi-model validation, and manual answering are used to generate question answers 46 corresponding to each synthetic question 44. In the model training module 47, the question answering model is trained using the synthetic questions 44 and question answers 46. In the difficulty controller 48, the question answering model identifies the unsolved questions as the first target questions 49, and the next training iteration is performed using the first target questions 49 and the questions in the question clusters 42. Repeat the above training iteration process until the training termination condition is met, and a well-trained question-answering model is obtained.

[0179] Here, the generation of diverse synthesis questions is based on a large language model. It takes a set of sample questions as input and outputs a variety of new synthesis questions. Considering the practicality, simplicity, and scalability of the synthesis question generation process, the diversity of synthesis questions is mainly ensured through the following three strategies:

[0180] 1) Data collection from multiple data sources: Since the data sources of mathematical problems in the real world are naturally diverse and distributed, we first collect millions of initial problems from multiple data sources in the real world. After careful data cleaning and deduplication, we obtain a set of candidate problems. We assume that these mathematical problems collected from different data sources cover almost all practical mathematical tasks.

[0181] 2) Many-to-Many Synthesis Problem Generation Method: After obtaining the candidate problem set, a trained synthesis processing model uses diverse prompts to generate synthetic problems based on the problems in the candidate problem set (e.g., prompting the trained synthesis processing model: you need to generate mathematical problems as diverse and accurate as possible). To further improve the diversity of synthetic problems, this application provides a many-to-many synthesis problem generation method. Specifically, firstly, each candidate problem is divided according to the problem category of the candidate problem set (e.g., high school algebra, middle school geometry, etc.) to obtain different problem clusters. Then, multiple candidate problems are selected from one cluster as input, and the trained synthesis processing model outputs multiple synthetic problems. This diversity-focused many-to-many approach can stimulate the trained synthesis processing model to generate more diverse mathematical synthesis problems.

[0182] 3) Multilingual Mathematical Problem Transfer: In actual experiments, this application's embodiments found that the coverage and reasoning ability of mathematical problems in English were far superior to those in other languages. Therefore, during data collection, this application's embodiments also collected mathematical problems in English or other languages ​​and translated these mathematical problems into a specified language (such as Chinese) to enhance the multilingual coverage of mathematical problems.

[0183] Finally, after data quality verification and deduplication, this embodiment of the application generates a large number of diverse synthesis problems based on the trained synthesis processing model. These problems are closely related to mainstream mathematical needs. It should be noted that the trained synthesis processing model in this embodiment of the application uses a mixed-language model; however, other open-source large language models can also be used as the trained synthesis processing model.

[0184] Generating accurate answers to questions is crucial for ensuring the quality of training data. This application embodiment inputs the aforementioned diverse synthetic questions into a large language model and outputs the corresponding answers and solution steps for these mathematical synthetic questions. The accuracy of the generated answers can be improved through the following strategies:

[0185] 1) Multi-Language Model Joint Validation: This application embodiment employs multiple large language models to automatically measure the correctness of multiple generated first question-and-answer results. Multiple different large language models are used to perform question-and-answer operations on the synthesized question, resulting in multiple first question-and-answer results. From the first question-and-answer results representing successful answers, at least one first answer is obtained, and the first proportion of each first answer in all first question-and-answer results is determined. This collaboratively judges the quality of the first question-and-answer results, discarding first question-and-answer results with significant discrepancies between different large language model validators, determining the first maximum proportion, and then, when the first maximum proportion is greater than or equal to a proportion threshold, determining the first answer corresponding to the first maximum proportion as the question answer.

[0186] 2) Improving the accuracy of question answer generation through retrieval-enhanced generation: Sometimes, even state-of-the-art large language models cannot accurately answer challenging synthetic questions, which are precisely the mathematical problems that this application aims to teach the question-answering model during training. Therefore, this application proposes a retrieval-enhanced generation method that retrieves high-quality mathematical problems related to the input synthetic question as reference questions to generate more accurate question answers. Specifically, the prompts input to the large language model for solving the problem include not only the original prompts and the input synthetic question, but also reference answers to relevant reference questions.

[0187] 3) Manual annotation: For some difficult problems that cannot be solved even through search-enhanced generation, this application's embodiments introduce limited expert-level annotation, using manual annotation to focus on solving the most difficult batch of mathematical problems and obtain the answers to the problems.

[0188] Based on these accuracy-oriented strategies, the embodiments of this application greatly improve the accuracy of answers to more challenging synthesis problems.

[0189] Here, the generation of the synthesis problem and the training of the model are not instantaneous processes; more challenging mathematical problems that the model cannot solve continuously emerge. Therefore, a guide for iteratively generating training data and iteratively training the model is essential. The generation of the synthesis problem should have an appropriate guiding direction closely related to practical needs. Since simple synthesis problems have been widely solved, it is necessary to focus on training the problem model to solve difficult mathematical problems. In this embodiment, a difficulty controller is used to guide the direction of synthesis problem generation. Specifically, the difficulty controller in this embodiment defines the difficulty of the synthesis problem in the following way:

[0190] 1) Static difficulty (corresponding to the first difficulty level in other embodiments): Mathematical problems collected from different data sources usually contain labels such as level or difficulty (e.g., elementary school math problems, university math problems, etc.), which can be directly used as static difficulty markers. In the process of generating synthetic problems, this embodiment of the application consciously selects more candidate problems from the more difficult problem clusters to generate many-to-many synthetic problems.

[0191] 2) Dynamic Difficulty: In addition to static difficulty, the difficulty is further considered in relation to the dynamics of model training. Specifically, at the end of each synthetic question generation cycle, the question-answering model obtained from the previous training iteration is first trained again based on the generated synthetic questions and their answers. The enhanced question-answering model after training becomes the question-answering model after the current training iteration. Then, using the question-answering model after the current training iteration, questions are randomly selected from the sample question set or synthetic questions for answering. Those questions that still cannot be correctly solved are identified as the first target questions. The first target questions and the questions randomly sampled from the question cluster are then combined to form the new sample questions needed for the next synthetic question generation. Unlike static difficulty, dynamic difficulty is related to the mathematical ability of the continuously evolving question-answering model, which allows the model training direction to more specifically address unsolved mathematical problems.

[0192] In the embodiments of this application, based on the standards of static difficulty and dynamic difficulty, more difficult synthesis problems can be used to generate new synthesis problems in the generation iteration of synthesis problems, thereby gradually breaking through the mathematical capability boundaries of the question-answering model.

[0193] Refer to Table 1, which is a performance comparison table of the models provided in the embodiments of this application. Among them, Model 1 is the question-answering model trained by the question-answering model training method provided in the embodiments of this application, and Models 2 to 5 are other question-answering models in related technologies. GSM8k, MATH and CMATH are different test datasets. The values ​​in the table represent the test results of the models on the corresponding datasets. The higher the value, the better the model performs on the dataset.

[0194] Table 1: Model Performance Comparison Table

[0195] Model Model parameters GSM8k MATH CMATH Model 1 1.5B 65.4 32.9 81.5 Model 2 1.5B 59.2 23.6 61.7 Model 1 8B 86.2 47.2 88.0 Model 3 8B 70.4 35.8 - Model 4 8B 58.2 17.0 52.7 Model 5 8B 81.9 29.6 68.7

[0196] As can be seen from Table 1, the question-answering model obtained by the question-answering model training method provided in this application embodiment has significant performance improvement under different model parameters or different test datasets.

[0197] The synthetic question evolution framework provided in this application is simple, efficient, and capable of continuous iteration. Furthermore, this application employs various strategies, including many-to-many question generation for diverse questions, multilingual data transfer, multi-model validation, and retrieval-enhanced generation of accurate answers, thus enabling the generation of higher-quality mathematical questions and more accurate answers. This application also provides a difficulty controller to determine which mathematical questions the model should focus on, thereby more effectively improving the mathematical capabilities of the question-answering model, gradually resolving currently unsolved, misunderstood, or fully explored mathematical problems, and improving the accuracy of the trained question-answering model.

[0198] The following description continues to illustrate the exemplary structure of the model training device 233 provided in the embodiments of this application as a software module. In some embodiments, such as Figure 2 As shown, the software modules stored in the model training device 233 of the memory 230 may include:

[0199] The determining module 2331 is used to determine the sample question set required for the current training iteration process of the question answering model, and generate a first training dataset for the current training iteration process based on the sample question set. The first training dataset includes multiple synthetic questions and the question answer corresponding to each synthetic question.

[0200] Training module 2332 is used to perform the current training iteration on the question answering model obtained in the previous training iteration based on the first training dataset, so as to obtain the question answering model after the current training iteration.

[0201] The determining module 2331 is further configured to determine a first target question from the sample question set and the plurality of synthetic questions based on the question-answering model after the current training iteration, wherein the first target question is a question that the question-answering model after the current training iteration failed to answer;

[0202] The determining module 2331 is further configured to determine the second training dataset for the next training iteration based on the first target question, and use the second training dataset to train the question-answering model after the current training iteration until the training termination condition is met, thereby obtaining the trained question-answering model.

[0203] In some embodiments, the determining module 2331 is further configured to: obtain a candidate problem set when the current training iteration process is the first training iteration process; the candidate problem set includes multiple candidate problems obtained by preprocessing the collected initial problem set; perform clustering processing on the multiple candidate problems based on the problem category of each candidate problem in the candidate problem set to obtain multiple problem clusters; determine a first target sample of the first number from each of the problem clusters; and construct the sample problem set required for the current training iteration process using the first target sample of the first number corresponding to each of the problem clusters.

[0204] In some embodiments, the determining module 2331 is further configured to: when the current training iteration process is the k-th training iteration process, obtain the second target problem corresponding to the (k-1)-th training iteration process and the target problem cluster corresponding to the second target problem, where k is an integer greater than 1; determine the third number based on the second number and the first number of the second target problem corresponding to the target problem cluster; determine the second target sample of the third number from the target problem cluster; determine the third target sample of the first number from each other problem cluster, where the other problem clusters are problem clusters other than the target problem cluster; and construct the sample problem set required for the current training iteration process using the second target problem, the second target sample, and the third target sample.

[0205] In some embodiments, the determining module 2331 is further configured to: determine a first difficulty level corresponding to each candidate question in each question cluster; determine a second difficulty level corresponding to the question cluster based on each first difficulty level; determine a sampling ratio corresponding to the second difficulty level, wherein the sampling ratio is positively correlated with the second difficulty level; and determine a first number based on the sampling ratio and the total number of questions in the question cluster.

[0206] In some embodiments, the determining module 2331 is further configured to, for each of the problem clusters, sort each of the candidate problems in descending order based on the first difficulty level corresponding to each candidate problem in the problem cluster, to obtain a sorted problem sequence; and determine the first candidate problem in the problem sequence as the first target sample of the first number.

[0207] In some embodiments, the determining module 2331 is further configured to generate multiple synthetic questions based on multiple sample questions corresponding to each question cluster, and determine the question answer corresponding to each synthetic question; and determine each synthetic question and the question answer corresponding to the synthetic question as the first training dataset of the current training iteration process.

[0208] In some embodiments, the determining module 2331 is further configured to obtain a trained synthesis processing model; generate prompt words based on multiple sample questions corresponding to each question cluster; and generate multiple synthesized questions based on the prompt words and the trained synthesis processing model.

[0209] In some embodiments, the determining module 2331 is further configured to: determine a plurality of first question-and-answer results corresponding to each of the synthesized questions; obtain at least one first answer from the first question-and-answer results representing successful answers; determine a first percentage of each first answer based on the first question-and-answer results representing successful answers, and determine a first maximum percentage based on each first percentage; when the first maximum percentage is greater than or equal to a percentage threshold, determine the first answer corresponding to the first maximum percentage as the question answer.

[0210] In some embodiments, the determining module 2331 is further configured to: determine a reference question related to the synthesized question and a reference answer corresponding to the reference question when multiple first question-and-answer results all indicate that the answer has failed, or when the first maximum percentage is less than a percentage threshold; determine at least one second answer corresponding to the synthesized question based on the synthesized question, the reference question, and the reference answer, and determine a second percentage for each second answer; determine a second maximum percentage based on each second percentage, and when the second maximum percentage is greater than or equal to the percentage threshold, determine the second answer corresponding to the second maximum percentage as the question answer.

[0211] In some embodiments, the determining module 2331 is further configured to perform retrieval processing based on the synthetic question to obtain multiple retrieval results, the retrieval results including candidate questions and answers corresponding to the candidate questions; determine the relevance result of each candidate question and the synthetic question; and determine a reference question from the multiple candidate questions based on the relevance result, wherein the relevance result corresponding to the reference question is greater than a relevance threshold.

[0212] In some embodiments, the determining module 2331 is further configured to send the synthesis question to the terminal when the second maximum proportion is less than the proportion threshold; receive the processing result sent by the terminal based on the synthesis question; and determine the processing result as the answer to the question.

[0213] In some embodiments, the determining module 2331 is further configured to process each sample question in the sample question set based on the question-answering model after the current training iteration to obtain a third question-answering result corresponding to the sample question; process each synthetic question in the plurality of synthetic questions based on the question-answering model after the current training iteration to obtain a fourth question-answering result corresponding to the synthetic question; and determine the sample question corresponding to the third question-answering result representing a failed or incorrect answer, and the synthetic question corresponding to the fourth question-answering result representing a failed or incorrect answer, as the first target question.

[0214] In some embodiments, the training module 2332 is further configured to use the second training dataset to train the question-answering model after the current training iteration to obtain the question-answering model after the next training iteration; determine the number of training iterations corresponding to the question-answering model after the next training iteration; when the number of training iterations reaches a preset number, determine that the training termination condition has been met, and determine that the question-answering model after the next training iteration is the trained question-answering model.

[0215] In some embodiments, the training module 2332 is further configured to acquire a verification dataset, the verification dataset including multiple verification questions and label data corresponding to each verification question; for each verification question, based on the question-answering model after the next training iteration, answer the verification question to obtain a fifth question-answering result for the verification question; based on the fifth question-answering result and label data corresponding to each verification question, determine the verification result of the question-answering model after the next training iteration; when the verification result indicates that the verification is passed, determine the question-answering model after the next training iteration as the trained question-answering model.

[0216] In some embodiments, the determining module 2331 is further configured to: obtain an initial question set, the initial question set including initial questions in multiple language types; clean the initial question set to obtain a cleaned initial question set; for each initial question in the cleaned initial question set, identify and delete questions that are identical to the initial question to obtain a deleted initial question set; and translate each initial question in the deleted initial question set other than a preset language type to obtain a candidate question set, the candidate question set including multiple candidate questions in a preset language type.

[0217] This application provides a computer program product, which includes a computer program or computer-executable instructions stored in a computer-readable storage medium. The processor of an electronic device reads the computer-executable instructions from the computer-readable storage medium and executes the computer-executable instructions, causing the electronic device to perform the question-answering model training method described above in this application.

[0218] This application provides a computer-readable storage medium storing computer-executable instructions or a computer program. When the computer-executable instructions or the computer program are executed by a processor, the processor will execute the question-answering model training method provided in this application. For example, ... Figure 3A The question-answering model training method is shown.

[0219] In some embodiments, the computer-readable storage medium may be a memory such as RAM, ROM, flash memory, magnetic surface memory, optical disk, or CD-ROM; or it may be a variety of devices including one or any combination of the above-mentioned memories.

[0220] In some embodiments, computer-executable instructions may take the form of programs, software, software modules, scripts, or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as stand-alone programs or as modules, components, subroutines, or other units suitable for use in a computing environment.

[0221] As an example, computer-executable instructions may, but do not necessarily, correspond to files in a file system. They may be stored in a portion of a file that holds other programs or data, for example, in one or more scripts in a Hyper Text Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple co-located files (e.g., a file that stores one or more modules, subroutines, or code sections).

[0222] As an example, computer-executable instructions can be deployed to execute on a single electronic device, or on multiple electronic devices located at one location, or on multiple electronic devices distributed across multiple locations and interconnected via a communication network.

[0223] In summary, in this embodiment, a sample question set required for the current training iteration is determined. A first training dataset for the current training iteration is generated based on the sample question set. The first training dataset includes multiple synthetic questions and their corresponding answers. Based on the first training dataset, the question-answering model obtained in the previous training iteration is used for the current training iteration to obtain the question-answering model after the current training iteration. In this way, in each training iteration, multiple synthetic questions and answers can be generated to perform the current training iteration on the question-answering model obtained in the previous training iteration, thereby ensuring that multiple training datasets can be generated cyclically in multiple training iterations, improving the quality and richness of the training data, and improving the efficiency and accuracy of model training. Then, based on the question-answering model after the current training iteration, a first target question is determined from the sample question set and multiple synthetic questions. The first target question is the question that the question-answering model failed to answer in the current training iteration. Based on the first target question, a second training dataset for the next training iteration is determined, and the question-answering model after the current training iteration is trained using the second training dataset. This ensures that the training dataset for each training iteration includes the questions the model failed to answer in the previous iteration. This allows for targeted training of the model each time it is trained, fully utilizing previously unsolved problems and further improving training efficiency. Finally, the training continues until the termination condition is met, resulting in a well-trained question-answering model that improves both question-answering accuracy and model performance.

[0224] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, and improvements made within the spirit and scope of this application are included within the scope of protection of this application.

Claims

1. A question-answering model training method, characterized in that, The method includes: Determine the sample question set required for the current training iteration of the question answering model, and generate a first training dataset for the current training iteration based on the sample question set. The first training dataset includes multiple synthetic questions and the question answer corresponding to each synthetic question. Based on the first training dataset, the question answering model obtained in the previous training iteration is used for the current training iteration to obtain the question answering model after the current training iteration. Based on the question-answering model after the current training iteration, a first target question is determined from the sample question set and the plurality of synthetic questions. The first target question is the question that the question-answering model after the current training iteration failed to answer. Based on the first target question, a second training dataset is determined for the next training iteration, and the question-answering model after the current training iteration is trained using the second training dataset until the training termination condition is met, resulting in a well-trained question-answering model.

2. The method according to claim 1, characterized in that, The determination of the sample problem set required for the current training iteration includes: When the current training iteration process is the first training iteration process, a candidate problem set is obtained, which includes multiple candidate problems obtained by preprocessing the collected initial problem set; Based on the question category of each candidate question in the candidate question set, multiple candidate questions are clustered to obtain multiple question clusters; From each of the aforementioned problem clusters, determine the first target sample of the first number; Using the first target sample of the first number corresponding to each of the problem clusters, construct the sample problem set required for the current training iteration process.

3. The method according to claim 2, characterized in that, The determination of the sample problem set required for the current training iteration includes: When the current training iteration process is the k-th training iteration process, the second target problem and the target problem cluster corresponding to the (k-1)-th training iteration process are obtained, where k is an integer greater than 1; Based on the second number of the second target problem corresponding to the target problem cluster and the first number, determine the third number; Determine the second target sample of the third number from the target problem cluster; A third target sample of the first number is determined from each of the other problem clusters, wherein the other problem clusters are problem clusters other than the target problem cluster; Using the second target problem, the second target sample, and the third target sample, a sample problem set required for the current training iteration process is constructed.

4. The method according to claim 2, characterized in that, The method further includes: For each of the aforementioned problem clusters, determine the first difficulty level corresponding to each candidate problem in the problem cluster; Based on each of the first difficulty levels, determine the second difficulty level corresponding to the problem cluster; Determine the sampling ratio corresponding to the second difficulty level, wherein the sampling ratio is positively correlated with the second difficulty level; Based on the sampling ratio and the total number of problems in the problem cluster, the first number is determined.

5. The method according to claim 2, characterized in that, The process of determining the first target sample of the first number from each of the problem clusters includes: For each of the aforementioned problem clusters, based on the first difficulty level corresponding to each candidate problem in the problem cluster, each of the candidate problems is sorted in descending order to obtain a sorted problem sequence; The candidate problem of the first number in the problem sequence is determined as the first target sample of the first number.

6. The method according to claim 1, characterized in that, The sample problem set includes multiple sample problems corresponding to each problem cluster. Generating the first training dataset for the current training iteration based on the sample problem set includes: Multiple synthetic questions are generated based on multiple sample questions corresponding to each question cluster, and the answer to each synthetic question is determined. Each of the synthesized questions and the corresponding answers to the synthesized questions are determined as the first training dataset for the current training iteration process.

7. The method according to claim 6, characterized in that, The generation of multiple synthetic questions based on multiple sample questions corresponding to each question cluster includes: Obtain the trained synthetic processing model; For each of the aforementioned question clusters, prompt words are generated based on multiple sample questions corresponding to the question cluster; Based on the prompt words, the trained synthesis processing model generates multiple synthesis questions.

8. The method according to claim 6, characterized in that, Determining the answer to each of the synthesis questions includes: For each of the aforementioned synthesis questions, determine multiple first question-answer results corresponding to the synthesis question; Obtain at least one first answer from the first question-and-answer results that indicate a successful solution; Based on the first question-and-answer result representing a successful answer, determine the first percentage for each of the first answers, and based on each of the first percentages, determine the first maximum percentage; When the first maximum percentage is greater than or equal to the percentage threshold, the first answer corresponding to the first maximum percentage is determined as the answer to the question.

9. The method according to claim 8, characterized in that, The method further includes: When multiple first question-and-answer results indicate that the solution has failed, or when the first maximum proportion is less than the proportion threshold, a reference question related to the synthesized question and a reference answer corresponding to the reference question are determined. Based on the synthesis question, the reference question, and the reference answer, at least one second answer corresponding to the synthesis question is determined, and a second proportion of each second answer is determined; A second maximum percentage is determined based on each of the second percentages. When the second maximum percentage is greater than or equal to the percentage threshold, the second answer corresponding to the second maximum percentage is determined as the answer to the question.

10. The method according to claim 9, characterized in that, The determination of the reference problem related to the synthesis problem includes: Based on the synthesized question, a retrieval process is performed to obtain multiple retrieval results, which include candidate questions and the corresponding answers to the candidate questions; Determine the correlation results between each of the candidate problems and the synthesis problem; Based on the correlation results, a reference problem is determined from a plurality of candidate problems, wherein the correlation result of the reference problem is greater than a correlation threshold.

11. The method according to claim 9, characterized in that, The method further includes: When the second maximum proportion is less than the proportion threshold, the synthesis problem is sent to the terminal; The system receives the processing result sent by the terminal based on the synthesized question and determines the processing result as the answer to the question.

12. The method according to any one of claims 1 to 11, characterized in that, The step of determining the first target question from the sample question set and the plurality of synthesized questions based on the question-answering model after the current training iteration includes: For each sample question in the sample question set, the sample question is processed based on the question-answering model after the current training iteration to obtain the third question-answering result corresponding to the sample question; For each of the plurality of synthesis questions, the synthesis question is processed based on the question-answering model after the current training iteration to obtain the fourth question-answering result corresponding to the synthesis question; The sample question corresponding to the third question-and-answer result that represents a failed or incorrect answer, and the composite question corresponding to the fourth question-and-answer result that represents a failed or incorrect answer, are determined as the first target question.

13. The method according to any one of claims 1 to 11, characterized in that, The step of training the question-answering model using the second training dataset after the current training iteration until the training termination condition is met, to obtain a trained question-answering model, includes: Using the second training dataset, the question-answering model after the current training iteration is trained to obtain the question-answering model after the next training iteration; The number of training iterations corresponding to the question-answering model after the next training iteration is determined. When the number of training iterations reaches a preset number, the training termination condition is determined to be met, and the question-answering model after the next training iteration is determined to be the trained question-answering model.

14. The method according to claim 13, characterized in that, The method further includes: Obtain a validation dataset, which includes multiple validation questions and label data corresponding to each validation question; For each verification question, the verification question is answered based on the question-answering model after the next training iteration, to obtain the fifth question-answering result for the verification question; Based on the fifth question-answer result and label data corresponding to each of the verification questions, the verification result of the question-answering model after the next training iteration is determined; When the verification result indicates that the verification is successful, the question-answering model after the next training iteration is determined to be the trained question-answering model.

15. The method according to any one of claims 2 to 11, characterized in that, The method further includes: Obtain an initial question set, which includes initial questions in multiple language types; The initial problem set is cleaned to obtain a cleaned initial problem set; For each initial problem in the cleaned initial problem set, identify and delete problems that are identical to the initial problem, thus obtaining a deleted initial problem set; The initial questions in the deleted initial question set, excluding each preset language type, are translated to obtain the candidate question set, which includes multiple candidate questions in the preset language type.

16. A question-answering model training device, characterized in that, The device includes: The determination module is used to determine the sample question set required for the current training iteration of the question answering model, and generate a first training dataset for the current training iteration based on the sample question set. The first training dataset includes multiple synthetic questions and the question answer corresponding to each synthetic question. The training module is used to perform the current training iteration on the question-answering model obtained in the previous training iteration based on the first training dataset, so as to obtain the question-answering model after the current training iteration. The determining module is further configured to determine a first target question from the sample question set and the plurality of synthetic questions based on the question-answering model after the current training iteration, wherein the first target question is a question that the question-answering model after the current training iteration failed to answer; The determining module is further configured to determine the second training dataset for the next training iteration based on the first target question, and use the second training dataset to train the question-answering model after the current training iteration until the training termination condition is met, thereby obtaining the trained question-answering model.

17. An electronic device, characterized in that, The electronic device includes: Memory is used to store executable instructions or computer programs. A processor, when executing computer-executable instructions or computer programs stored in the memory, implements the method according to any one of claims 1 to 15.

18. A computer-readable storage medium storing computer-executable instructions or a computer program, characterized in that, When the computer-executable instructions or computer program are executed by a processor, they implement the method described in any one of claims 1 to 15.

19. A computer program product comprising computer-executable instructions or a computer program, characterized in that, When the computer-executable instructions or computer program are executed by a processor, they implement the method according to any one of claims 1 to 15.

Images