Mathematical test question generation and solution collaborative enhancement method and system based on large model

Abstract

The invention provides a mathematical test question generation and solution collaborative enhancement method and system based on a large model, and relates to the technical field of large language models.The method comprises the steps that multiple sets of annotation data are obtained and input into a basic large language model for fine tuning training, and a first large language model is obtained; generating a plurality of test question request instructions, inputting the test question request instructions into the first large language model, correspondingly outputting mathematical questions and solving processes, obtaining a plurality of groups of combined data, and screening the combined data to obtain a supervision fine tuning data set and a preference optimization data set; performing optimization training on the first large language model by utilizing the preference optimization data set to obtain a second large language model; inputting the supervision fine tuning data set into a second large language model for next round of iterative training; continuously repeating to obtain a target large language model; and inputting a current test question request instruction into the trained target large language model, and outputting a mathematical test question comprising a mathematical question and a solving process. According to the method, the problems of low big language model training efficiency and poor optimization effect are solved.

Description

Technical Field

[0001] The present invention relates to the technical field of large language models, and in particular to a method and system for collaboratively enhancing the generation and solution of mathematical test questions based on large models. Background Art

[0002] In recent years, large language models (LLMs) have achieved significant breakthroughs in natural language processing (NLP) fields such as translation, question answering, and text summarization. As the models scale, they have shown great potential in more complex areas, particularly in math problem generation (MPG) and math reasoning (MR). MPG aims to automatically generate math problems that meet teaching requirements, while MR focuses on automatically solving the corresponding math problems and providing detailed solution procedures. These capabilities provide technical support and theoretical foundations for educational applications such as intelligent tutoring systems and automated assessments.

[0003] There is a mutually reinforcing relationship between math problem generation and mathematical reasoning ability. Generating excellent math questions is not only reflected in the expression of the math questions, but also requires a complete and accurate solution process. Therefore, the quality of math questions can be improved by enhancing the reasoning ability of the large language model, making the questions more solvable and rigorous; or by training the large language model to generate high-quality math questions including a complete solution process, it can also help improve its mathematical reasoning ability. In short, the synergistic improvement of MPG and MR helps to optimize the comprehensive performance of the large language model in mathematical tasks and improve the credibility and practicality of the generated content.

[0004] Large language models, leveraging their powerful natural language understanding and reasoning capabilities, have made significant breakthroughs in improving mathematical problem generation and reasoning. However, existing methods primarily train large language models to improve math problem generation by constructing high-quality mathematical datasets. These methods focus on generating math problems within math exams, neglecting the equally important generation of the solution process. Generating math problems and generating the solution process within math exams are often treated as two separate tasks, ignoring the underlying connection between them. When optimizing the model, both the math problem generation model and the solution generation model need to be optimized separately. This wastes data resources and leads to problems such as poor stability, low training efficiency, and weak mathematical logic in the large language model. Furthermore, existing methods rely heavily on manually annotated data, relying solely on iterative optimization based on this data. This manual annotation process is not only time-consuming and costly, but also, due to the complexity of math exams, can contain logical loopholes or biases, resulting in poor training data quality and impacting the learning quality and continuous learning ability of the large language model. Furthermore, existing methods typically employ a one-time fine-tuning strategy, making it difficult to support the long-term evolution and continuous optimization of the model. When new data is added or the training data is modified, the model often needs to be retrained, resulting in high computational costs and the potential for catastrophic forgetting. Furthermore, existing large language models using supervised fine-tuning are typically optimized for specific tasks and struggle to generalize to math problems across different topics or complex exams. This results in limited training data and an inability to adapt to math problems of varying difficulty levels, leading to insufficient reasoning capabilities for the large language models. These issues with large language models contribute to the poor quality of the math problems they produce. Summary of the Invention

[0005] In response to the above-mentioned problems, the present invention provides a collaborative enhancement method and system for generating and solving mathematics test questions based on a large model, which is used to solve the problem of poor quality of mathematics test questions caused by low training efficiency and poor optimization effect of the current large language model.

[0006] In a first aspect, the present invention provides a collaborative enhancement method for generating and solving mathematical test questions based on a large model, the method comprising: Acquire multiple sets of annotated data, input the multiple sets of annotated data into a basic large language model for fine-tuning training, and obtain a first large language model for generating math problems and solution processes; wherein the annotated data includes test question annotation instructions, math problems, and solution processes; generating a plurality of test question request instructions, inputting the plurality of test question request instructions into the first language model respectively, and outputting mathematical problems and solution processes corresponding to the test question request instructions, thereby obtaining a plurality of sets of composite data; Screening multiple sets of composite data to obtain a supervised fine-tuning dataset and a preference optimization dataset; optimizing and training the first large language model using the preference optimization dataset to obtain a second large language model; inputting the supervised fine-tuning dataset into the second large language model for the next round of iterative training; and repeating the process until the model converges to obtain a target large language model. The current test question request instruction is input into the trained target large language model, and a math test question including a math question and a solution process is output.

[0007] Furthermore, the method also includes: obtaining a forgetting score function by measuring the loss difference between the current round of synthesized data and the historical synthesized data in the output of the second largest language model, and dynamically screening important key historical synthesized data based on the forgetting score function; supplementing the key historical synthesized data into the supervised fine-tuning dataset to achieve the next round of iterative training.

[0008] Furthermore, for any historical synthetic data , the forgetfulness score function for: ; in, Representation Model In historical synthetic data The loss value on ; Represents the model obtained after the current round of training based on the addition of the key historical synthetic data; Representation Model The loss value on the synthetic data in this round; Represents the model obtained after the current round of training without supplementing the key historical synthetic data; Indicates the question request instruction. Represents the corresponding mathematical problem and solution process.

[0009] Furthermore, the generation of the test question request instruction includes: The math problem is defined as multiple dimensional information, and the test question request instruction is generated by randomly combining the multiple dimensional information.

[0010] Furthermore, a set of the synthetic data includes a test question request instruction, a math question and a solution process corresponding to each other.

[0011] Furthermore, multiple groups of data are combined and screened through a double verification mechanism; The dual verification mechanism includes: Obtaining a first signal based on a self-game verification mechanism; Obtaining a second signal based on a multi-agent collaboration mechanism; The synthesized data is screened and classified according to the first signal and the second signal.

[0012] Furthermore, the self-game verification mechanism is: Copy the first language model of this round to generate a copy shadow model that independently obtains the solution process; Decompose the math questions in the synthetic data into the problem statement and the original solution process; Inputting the problem statement into the copy shadow model and outputting a comparative solution process; It is determined whether the logical steps and final answers of the original solution process and the comparative solution process are consistent, and a first signal is obtained.

[0013] Furthermore, the multi-agent collaboration mechanism is: Introducing multiple agent models with different expertise, Use multiple intelligent agent models to evaluate the quality of mathematical problems in synthetic data and output evaluation opinions; Determine whether the evaluation opinions output by multiple agents are consistent and obtain a second signal.

[0014] Furthermore, the first signal is , the second signal is ;1 means consistent, 0 means inconsistent; like , then directly add the supervised fine-tuning dataset; like , then add the preference optimization data set after feedback optimization; like , then the synthesized data is discarded.

[0015] In a second aspect, the present invention provides a collaborative enhancement method system for generating and solving mathematics test questions based on a large model, comprising a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of any of the above methods.

[0016] In general, the present invention provides a method and system for collaboratively enhancing the generation and solving of mathematical test questions based on a large model. The technical solution conceived by the present invention can achieve the following beneficial effects compared with the existing technology: (1) The present invention takes "mutual promotion of mathematical problem generation and mathematical reasoning ability" as its core concept and constructs a continuous learning and training framework with the collaboration of mathematical problem generation and solution process as the core. It breaks away from the previous practice of dividing the two tasks into independent processing and constructs training samples that include both mathematical problems and complete solution processes. By utilizing the deep connection between mathematical problems and solution processes, it establishes a collaborative continuous optimization path in which mathematical problem generation drives mathematical reasoning ability and mathematical reasoning ability feeds back to mathematical problem generation, thus achieving the simultaneous improvement of mathematical problem generation and reasoning ability; solving the current problems of low training efficiency and poor optimization effect of large language models, and outputting high-quality mathematical test questions.

[0017] (2) The present invention uses synthetic data to replace manually annotated data for high-quality model training, breaking through the dependence on large-scale manually annotated data. By generating synthetic data to train and optimize the model, a model-driven synthetic data closed-loop framework is constructed, so that the generated synthetic data has good quality in terms of semantic consistency, language perplexity, etc., avoiding problems such as logical loopholes or deviations caused by manual annotation alone, and improving the quality of training data; and the synthetic data of the present invention can be directly used for model training, providing a practical technical solution for building high-quality and low-cost models.

[0018] (3) The present invention introduces a data replay mechanism based on loss difference. By measuring the loss difference between the current round of synthetic data and the historical synthetic data, a forgetting score function is obtained, thereby dynamically screening important key historical synthetic data and adding them to the supervised fine-tuning dataset. In this way, the model's ability to remember key knowledge is maintained without significantly increasing the training burden or retraining. This not only enhances the training quality and reduces the training cost, but also solves the catastrophic forgetting problem that may occur in large language models during multiple rounds of continuous training, thereby improving the continuous learning ability and training effect of large language models.

[0019] (4) The present invention introduces a dual verification mechanism of self-game verification mechanism and multi-agent collaboration mechanism to realize the screening of synthetic data; introduces the self-game mechanism into mathematical tasks to realize the self-generation, self-solution and self-evaluation of the model without manually labeled data; at the same time, introduces the multi-agent collaboration mechanism, and uses multiple agent models to evaluate the quality of synthetic data from multiple different perspectives, which can effectively improve the logic, correctness and diversity of synthetic data, and provide high-quality training samples for subsequent training. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0021] Figure 1 This is a schematic diagram of the method steps of a collaborative enhancement method system for generating and solving mathematical test questions based on a large model provided by the present invention; Figure 2 This is a schematic diagram of synthetic data screening for a collaborative enhancement method system for generating and solving mathematical test questions based on a large model provided by the present invention; Figure 3 It is a schematic diagram of experimental results of quality analysis of synthetic data by a collaborative enhancement method system for generating and solving mathematical test questions based on a large model provided by the present invention. DETAILED DESCRIPTION

[0022] To make the objectives, technical solutions, and advantages of the present invention more clear, the technical solutions of the present invention will be clearly and completely described below in conjunction with the accompanying drawings and embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0023] It should be noted that, in the description of the embodiments of the present invention, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a method, step, or apparatus comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such method, step, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the method, step, or apparatus comprising the element.

[0024] This paper proposes a collaborative enhancement method for math problem generation and solution based on a large model. This method focuses on simultaneously optimizing math problem generation and mathematical reasoning capabilities. By leveraging the inherent connection between these two capabilities and generating high-quality synthetic data, it enhances the collaborative capabilities of large language models in these two tasks. This addresses the current problems of poor math problem quality caused by low training efficiency and poor optimization effects of large language models. Compared to traditional methods, this paper not only focuses on the generation of math problems, but also emphasizes the construction of a complete solution process to ensure the rationality and solvability of math problems.

[0025] Specifically, if Figure 1 As shown, the method includes: S101: Acquire multiple sets of annotated data, input the multiple sets of annotated data into a basic large language model for fine-tuning training, and obtain a first large language model for generating math problems and solving the problem.

[0026] The labeled data includes test question labeling instructions, math problems, and solution procedures. Labeled data is manually labeled data. A set of labeled data consists of a test question labeling instruction, a math problem, and a solution procedure, which correspond to each other. This is the test question labeling instruction - math problem - solution procedure.

[0027] It should be noted that the math test questions in the present invention include math questions and solution processes.

[0028] In the supervised fine-tuning stage, labeled data is used to fine-tune the basic large language model so that it has the ability to generate mathematical problems and mathematical reasoning, thereby obtaining the first large language model.

[0029] During the fine-tuning training process, the first language model can be fine-tuned by minimizing the loss between the predicted results and the true labeled data. Preferably, a cross-entropy loss function is used. Specifically, the loss function of the first language model can be: ; in, Indicates that in all data pairs Take the average of the above; Indicates test question marking instructions; Indicates the math problems and solution processes outputted corresponding to the test question marking instructions; Represents the current training dataset; Indicates the sequence length; Represents the current model; Indicates the tokens; Indicates that from to tokens.

[0030] S102: Generate multiple test question request instructions, input the multiple test question request instructions into the first language model respectively, and output the math questions and solution processes corresponding to the test question request instructions to obtain multiple groups of composite data.

[0031] In other words, the synthetic data includes a test question request instruction, a math problem, and a solution process. A set of synthetic data includes a corresponding test question request instruction, a math problem, and a solution process. In other words, the synthetic data is obtained by combining the three data sets of test question request instruction, math problem, and solution process.

[0032] In the synthetic data generation stage, the goal is to improve the model's reasoning and generalization capabilities by generating high-quality mathematical problems and corresponding solution processes.

[0033] As an embodiment, the generation of the test question request instruction includes: The math questions are defined as multiple dimensional information, and the test question request instructions are generated by randomly combining the multiple dimensional information.

[0034] For example, math questions can be defined into three core dimensions: question type, knowledge point classification, and difficulty level.

[0035] Question types: These can include a variety of basic question types, such as multiple-choice questions, fill-in-the-blank questions, application questions, and proof questions, covering all scenarios from basic calculations to complex reasoning.

[0036] Knowledge point classification: You can establish a knowledge point library to integrate a variety of mathematical knowledge points, covering the core content of mathematical knowledge points, such as fraction operations, quadratic functions, geometric transformations, etc., and ensure the correlation between levels through tree structure organization.

[0037] Difficulty grading: Use a neural network to quantify the difficulty of the input math problems into the interval [0.1, 0.9], and label the difficulty of the math problems in order to obtain the difficulty label of the math problems.

[0038] Based on these three core dimensions, test question request instructions are generated through a random combination strategy.

[0039] For example, by randomly selecting one to three knowledge points from the knowledge point library and combining the question type and difficulty tag, a question request instruction template can be generated, such as "Please generate a math problem covering the knowledge points of quadratic function expressions and the method of undetermined coefficients, with a problem type of application problem and a difficulty tag of 0.7, and provide a step-by-step solution." Based on this instruction template, multiple question request instructions are generated by varying the content. These are then input into the first language model to obtain the corresponding math problem and solution, thus ensuring the diversity and balanced distribution of the synthetic data.

[0040] The present invention uses synthetic data instead of manually annotated data for high-quality model training, so that the generated synthetic data has good quality in terms of semantic consistency and language perplexity, avoiding problems such as logical loopholes or deviations caused by manual annotation alone, and improving the quality of training data.

[0041] S103: Filter the multiple groups of synthesized data to obtain a supervised fine-tuning dataset and a preference optimization dataset; use the preference optimization dataset to optimize and train the first large language model to obtain a second large language model; input the supervised fine-tuning dataset into the second large language model for the next round of iterative training; repeat until the model converges to obtain the target large language model.

[0042] In order to ensure the high quality and diversity of the synthesized data, the present invention screens multiple sets of synthesized data through a double verification mechanism.

[0043] As an embodiment, the dual verification mechanism includes: Obtaining a first signal based on a self-game verification mechanism; Obtaining a second signal based on a multi-agent collaboration mechanism; The synthesized data is screened and classified according to the first signal and the second signal.

[0044] It should be noted that this invention uses a self-game mechanism to allow the model to act as both problem setter and solver, screening out high-quality math problems and their solutions based on consistency of solution. Through a multi-agent collaboration mechanism, multiple external agent models are introduced to evaluate the rationality of the math problems and the accuracy of the solution process from different perspectives, thereby achieving quality control of the synthesized data and ensuring the correctness and diversity of the math problems and solutions. Finally, a data classification mechanism is used to screen and classify the synthesized data to optimize the model's continuous learning process, effectively improving the logic, correctness, and diversity of the synthesized data and providing high-quality training samples for subsequent training.

[0045] The self-game verification mechanism acts as both a question setter and a problem solver, screening high-quality data through consistency in problem solving. As an example, the self-game verification mechanism can be: Create a copy shadow model: copy the first language model of this round , generate a copy shadow model of the independent solution process ; Question decoupling: decompose the output current math question into question statements With the original solution process ; Independent solution: State the problem Input into the copy shadow model and output the comparison solution process ; Consistency judgment: judge the original solution process Comparison and solution process The first signal is obtained by checking whether the logical steps are consistent with the final answer.

[0046] Furthermore, the first signal is a binary signal and can be expressed as , if the original solution process Comparison and solution process The logical steps are consistent with the final answer, then the first signal On the contrary, if the original solution process Comparison and solution process If the logical steps are inconsistent with the final answer, the first signal .

[0047] Different agent models can bring novel ways of thinking to knowledge reasoning. They can consider issues from different perspectives and capture problems that are difficult to identify from a single perspective. The multi-model collaboration mechanism of this invention ensures the quality of synthetic data by introducing multiple external agent models with different expertise to evaluate them from multiple perspectives.

[0048] As an embodiment, the multi-agent collaboration mechanism may be: Introducing multiple agent models with different expertise, Use multiple intelligent agent models to evaluate the quality of mathematical problems in synthetic data and output evaluation opinions; Determine whether the evaluation opinions output by multiple agents are consistent and obtain a second signal.

[0049] Furthermore, the second signal is a binary signal and can be expressed as If the domain views output by multiple agents are consistent, the second signal On the contrary, if the domain views output by multiple agents are inconsistent, the second signal .

[0050] For example, first introduce two intelligent agent models: A1 and A2, which independently evaluate the quality of math problems and give evaluation opinions within their fields. The process of independent analysis and answering ensures that each intelligent agent model will not be disturbed by the opinions of other intelligent agents, thereby ensuring the independence of the answers. Then introduce an intelligent agent A3 to analyze the evaluation opinions of the intelligent agent models A1 and A2. If the opinions of A1 and A2 are consistent, A3 will synthesize the opinions of the two to obtain a complete and hierarchical opinion system to ensure that the final solution covers important information from all aspects. If there is a conflict between the opinions of A1 and A2, A3 will conduct a detailed analysis of the differences between the two, clearly point out the specific controversial points and their respective positions, and then evaluate each difference to find a reasonable compromise. Finally, A3 generates a binary quality assessment signal, which is the second signal .

[0051] The synthesized data is screened and classified according to the first signal and the second signal.

[0052] As an embodiment, the first signal is , the second signal is ;1 means consistent, 0 means inconsistent; like , then directly add the supervised fine-tuning dataset (SFT dataset,); the supervised fine-tuning dataset is used as the data for the next round of supervised fine-tuning. The dataset mainly includes high-quality mathematical problems and their complete solution process.

[0053] like , then it is added to the Direct Preference Optimization dataset (DPO dataset) after feedback optimization; this indicates that this type of data has some defects but can be optimized and needs to be re-added to the Direct Preference Optimization dataset after feedback optimization. The Direct Preference Optimization dataset guides the first language model to learn to prefer high-quality math problems and solution processes, enhancing its ability to identify high-quality problems and high-quality reasoning paths. This dataset mainly includes paired example data, one of which is an optimized math problem and solution process output by the first language model, and the other is a flawed math problem and solution process. This is used to train the first language model to output more correct and reasonable solutions.

[0054] like , then the synthetic data is discarded. The data is low-quality data. This dataset includes math problems and solution processes that are judged to be wrong, unreasonable or low-quality during self-game or agent evaluation.

[0055] As a specific embodiment, Figure 2 As shown in the figure, a task space including 6 question types, 1200+ knowledge point classifications and 3 levels of difficulty is constructed, and 50K question request instructions are randomly generated. The question request instructions are input into the first language model to obtain the corresponding math questions and solution processes, thereby obtaining 50K combined data.

[0056] Use the self-game mechanism to make consistency judgments and get the first signal ; Use multi-agent collaboration mechanism to make quality assessment and judgment, and get the second signal ; Finally, the data that has passed the double verification will be combined into multiple groups for screening and classification, thereby ensuring the diversity and balanced distribution of the synthetic data.

[0057] This paper employs a direct preference optimization approach, using positive and negative examples to reinforce the model's learning of high-quality problem-solving processes, ensuring they align more closely with mathematical logic. Specifically, a preference-optimized dataset is used to optimize and train a first language model, yielding a second language model. This phase trains the first language model using positive and negative examples, enabling it to learn high-quality mathematical problems and avoid common errors. In other words, the first language model is fine-tuned directly based on the preference-optimized dataset, bypassing the reward modeling stage to maximize the objective function.

[0058] Among them, the objective function can be: ; in, Indicates that in all data pairs Take the average of the above; Indicates the test question request instruction; Indicates the defective mathematical problem and solution process corresponding to the question request instruction; Indicates the optimized math problem and solution process corresponding to the question request instruction; Represents the current preference optimization dataset; Represents the sigmoid function; represents a hyperparameter; Indicates the probability that the model to be optimized generates high-quality mathematical problems and solution processes; represents the probability that the reference model generates high-quality mathematical problems and solution processes; Indicates the probability that the model to be optimized generates low-quality mathematical problems and solution processes; Represents the probability that the reference model generates low-quality mathematical problems and solution processes.

[0059] In order to further improve the generalization ability and long-term stability of the model, this paper adopts a data replay mechanism to continuously utilize high-quality historical data during the training process to reduce the forgetting effect.

[0060] Specifically, by introducing a dynamic data replay mechanism based on loss difference, in the dynamic data replay stage, the loss difference is used to measure the impact of the introduction of new knowledge on the old knowledge, so as to identify which historical synthetic data are more likely to be forgotten, thereby selectively retaining key historical synthetic data, alleviating the catastrophic forgetting problem, and improving the long-term stability of the model.

[0061] As an example, a forgetfulness scoring function is derived by measuring the difference in loss between the current round of synthesized data and historical synthesized data in the output of the second largest language model. Important key historical synthesized data is dynamically selected based on the forgetfulness scoring function. This key historical synthesized data is then added to the supervised fine-tuning dataset for the next round of iterative training. This maintains the model's ability to retain key knowledge without significantly increasing the training burden or requiring retraining. This not only enhances training quality and reduces training costs, but also addresses the catastrophic forgetting problem that can occur in large language models during multiple rounds of continuous training, improving their continuous learning capabilities and training effectiveness.

[0062] It should be noted that for any historical synthetic data , forgetfulness score function for: ; in, Representation Model In historical synthetic data The loss value on ; Represents the model obtained after the current round of training, supplemented with key historical synthetic data; Representation Model The loss value on the synthetic data in this round; Represents the model obtained after the current round of training without supplementing key historical synthetic data; Indicates the question request instruction. Represents the math problem and solution process output corresponding to the test question request instruction. If the forgetting degree of the historical synthetic data in this round is greater than the average loss of all historical synthetic data on the model, it is considered forgotten.

[0063] S104: Input the current test question request instruction into the trained target large language model, and output the math test question including the math question and the solution process.

[0064] The present invention prevents catastrophic forgetting through dynamic sample selection and knowledge transfer accumulation, screens high-value samples based on historical training loss differences, and prioritizes the return of easily forgotten math questions and high-difficulty math questions that have not been fully learned, thereby maintaining the iterative stability of the model in the process of acquiring new knowledge and improving the model's learning efficiency and learning effect on new tasks.

[0065] On the other hand, the present invention also provides a collaborative enhancement system for generating and solving mathematical test questions based on a large model, comprising a memory, a processor and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of any of the above methods.

[0066] Specifically, the system can utilize multiple modules to achieve collaborative optimization of mathematical problem generation and mathematical reasoning capabilities, including a supervised fine-tuning module, a data generation module, a direct preference optimization module, and a data replay module.

[0067] The supervised fine-tuning module is used to initialize and fine-tune the basic large language model using labeled data, so that it has basic mathematical problem generation and solving capabilities.

[0068] The data generation module is used to generate and screen high-quality synthetic data using the self-game mechanism and multi-agent collaboration mechanism, thereby improving the quality of training data and the generalization ability of the model.

[0069] The direct preference optimization module is used to further enhance the model's preference for high-quality mathematical problems and solution processes using positive and negative examples.

[0070] The data replay module is used to solve the forgetting problem by selectively retaining key historical synthetic data.

[0071] This paper focuses on leveraging the synergistic relationship between mathematical problem generation and mathematical reasoning. By training a model with synthetic data consisting of both mathematical problems and complete solution processes, it simultaneously improves performance on both tasks. Compared to approaches that optimize either problem generation or reasoning alone, this method ensures that the generated mathematical problems possess a clear logical structure while simultaneously improving the model's ability to understand and solve complex mathematical problems.

[0072] To comprehensively validate the performance of the target large language model, the present invention designed targeted evaluation metrics and systems for the two tasks of mathematical problem generation and mathematical reasoning ability. In the mathematical problem generation task, a scoring method based on an advanced model (GPT-4o) was used to score the generated math problems based on eight dimensions: language fluency (LF), logical correctness (LC), content completeness (CC), analytical completeness (AC), answer accuracy (AA), knowledge relevance (KR), difficulty adaptability (DA), and question type adaptability (TA). Each dimension was scored out of 10, and the average score was used to measure the overall quality of the problem. In the mathematical reasoning task, standard accuracy was used as the primary evaluation metric, judging whether the final output of the target large language model was completely consistent with the reference answer, ensuring objectivity and reproducibility of the evaluation. Furthermore, to verify the quality of the synthesized data, the present invention introduced multiple automatic evaluation metrics, including semantic similarity (BERTScore) and perplexity (PPL), which measure the degree of similarity between the synthesized data and the annotated data from the perspectives of semantic rationality and linguistic naturalness, respectively.

[0073] Specifically, the present invention sets up multiple basic large language models for comparative experiments, mainly including mainstream large language models such as Qwen2-7B-Instruct, GLM4-9B-Chat, Gemma-2-9b-it, Llama-3.1-8B-Instruct, and specialized models trained in the field of mathematics such as DeepSeekMath-7B-RL, Internlm2-math-plus-20b, and NuminaMath-7B-CoT.

[0074] The verification experiment is divided into four parts: mathematical problem generation task evaluation experiment, mathematical reasoning ability task evaluation experiment, continuous learning effect and synthetic data quality analysis.

[0075] (1) Mathematical Problem Generation Task Evaluation Experiment Table 1 Mathematical problem generation performance of each LLM

[0076] Table 1 shows the performance of math problem generation for each LLM. The bold indicates the best score, and the underline indicates the suboptimal score. It can be found that: The model of the present invention performs well in all eight dimensions, with an average score of 9.401, which is superior to other baseline models in all aspects. Among them, KR is 9.786, AA is 9.982, TA is 9.76, and AC is 9.27, which are particularly outstanding; the present invention is significantly better than the field baseline, such as DeepSeek-Coder-V2-Lite-Instruct, which has a KR of 9.476, AA of 9.07, TA of 9.108, and AC of 8.556. This shows that the model of the present invention can accurately revolve around the specified knowledge points when generating math test questions, and ensure that the question type matches the requirements and the solution process and results are accurate. In addition, in terms of content quality, the CC of the model of the present invention is 9.11 and the LC is 9.882, both of which reach a high level, reflecting its ability to generate test questions with rigorous structure and clear logic.

[0077] (2) Mathematical reasoning ability task assessment experiment Table 2 Performance of the solution process of each LLM

[0078] Table 2 shows the performance of the solution process of each LLM. The bold represents the best score, and the underline represents the suboptimal score. It can be found that: Models trained using our method have demonstrated excellent performance on multiple mathematical reasoning benchmarks. On English benchmarks, our model achieved a top-performing score of 90.1% on the GSM8K benchmark, surpassing all baseline models. On the more challenging MATH benchmark, it achieved an accuracy of 60.7%, only 0.3 percentage points lower than the top-ranked DeepSeek-Coder-V2-Lite-Instruct (61.0%). On challenging math benchmarks such as Minerva Math, GaoKao 2023 En, and College Math, our model achieved scores of 27.9%, 54.3%, and 39.5%, respectively, outperforming most baseline models and trailing only DeepSeek-Coder-V2-Lite-Instruct. However, it's worth noting that this model has a parameter size of 16 bytes, far exceeding the parameter size of our model (7 bytes).

[0079] In terms of Chinese benchmark tests, the performance of the present invention is even more outstanding. On the CMATH test set, the model achieved an accuracy of 91.0%, surpassing the equally strong professional mathematics model DeepSeek-Coder-V2-Lite-Instruct (89.8%) and the general language model GLM4-9B-Chat (86.0%). This advantage is also obvious in the CN-Middle-School-24 benchmark test. The model of the present invention achieved a score of 74.3%, which is 7% higher than the optimal baseline DeepSeekMath-7B-RL's 67.3%.

[0080] (3) Continuous learning effect The continuous learning strategy of the present invention shows significant performance in both mathematical problem generation and mathematical reasoning ability tasks.

[0081] Table 3 Performance changes of models in various dimensions after different training iterations in mathematical problem generation

[0082] Table 3 shows the performance changes of the models across various dimensions on the math problem generation task after different numbers of training iterations. M0 represents the base model, M1 represents the model after one training iteration, M2 represents the model after two training iterations, and so on. As shown in Table 3, the base model M0 achieved an average score of 8.639 across the eight evaluation dimensions on the math problem generation task. After the first iteration, the average score increased to 8.936. After the second iteration, the average score further improved to 9.401. This demonstrates the effectiveness of continuous learning in the generation task. The improvements in core metrics such as LC, AA, and TA were particularly significant.

[0083] Table 4 Performance changes of models in mathematical reasoning ability in various dimensions after different training iterations

[0084] Table 4 shows the performance changes of the models in various dimensions after different training iterations on the mathematical reasoning ability task. M0 represents the basic model, M1 represents the model after one training, M2 represents the model after two trainings, and so on. As can be seen from Table 3, Continuous learning also led to sustained performance improvements in mathematical reasoning tasks. The base model, M0, achieved an average accuracy of 45.933% across nine benchmarks. After the first iteration (M1), model performance significantly improved, reaching an average accuracy of 52.389% (+6.456%). The gains were particularly pronounced on MATH and CN-Middle-School-24, increasing by 5.1% and 11.8%, respectively. The second iteration (M2) further enhanced the model's capabilities, raising the average accuracy to 55.411%, achieving improvements ranging from 2% to 8% on most benchmarks. However, performance on challenging competitions like the Olympiad Bench fluctuated slightly, indicating that the model still has room for improvement when handling challenging mathematical problems.

[0085] (4) Synthetic data quality analysis In order to verify the quality and effectiveness of the synthetic data, this paper uses the external reward model InternLM2-7B-Reward to evaluate the preference pairs. This model judges whether the improved answer is better than the initial answer. The experimental results are shown in Figure 2. Figure 3 As shown, in the preference data synthesized in the first round of iterations, 86% of the improved answers were evaluated as better than their initial versions; the preference data synthesized in the second round of iterations maintained a high consistency rate of 84%. The stable and high consistency rate derived from the independent reward model provides strong empirical support for the reliability of the preference data synthesis method of the present invention. It not only verifies that the model can modify its output based on self-feedback and external feedback, but also verifies the effectiveness of combining self-game and multi-model collaboration mechanisms to generate high-quality preference pairs.

[0086] Table 5 Quality analysis of synthetic data

[0087] Table 5 is a quality analysis of the synthesized data, where PPL represents perplexity, and BERT Score F1 represents the comparison of the synthesized data of each data set with the annotated data. As can be seen from Table 5, the present invention uses two commonly used indicators to compare the quality of the SFT data synthesized by the present invention and the annotated data (5,000 samples were randomly selected respectively): PPL and BERT Score; among them, a lower PPL means a better output. The present invention uses Qwen2.5-72B-Instruct to calculate PPL, and uses BERT Score to calculate the semantic similarity of the synthesized SFT data, and compares it with the annotated data written by humans to determine whether the synthesized data has comparable quality to the annotated data. It can be seen that although the PPL of the SFT data synthesized in the first two rounds of iterations is lower than that of the data written by humans, the performance of the synthesized data on BERT Score is very close to that of the annotated data.

[0088] In summary, the present invention takes "mutual promotion of mathematical problem generation and mathematical reasoning ability" as its core concept, utilizes the deep connection between mathematical problems and the solution process, establishes a collaborative and continuous optimization path in which mathematical problem generation drives mathematical reasoning ability, and mathematical reasoning ability feeds back to mathematical problem generation, thereby achieving the simultaneous improvement of mathematical problem generation and reasoning ability; solves the current problems of low training efficiency and poor optimization effect of large language models, and outputs high-quality mathematical test questions.

[0089] It should be noted that for the aforementioned embodiments, for simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that this application is not limited by the order of the actions described, because according to this application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in this specification are all preferred embodiments, and the actions and modules involved are not necessarily required by this application.

[0090] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0091] In the several embodiments provided in this application, it should be understood that the disclosed methods or systems can be implemented in other ways. For example, the embodiments described above are merely illustrative, and the division of the units described is merely a logical functional division. In actual implementation, other division methods may be used, such as combining or integrating multiple units or components into another system, or ignoring or not implementing certain features.

[0092] The above is only an exemplary embodiment of the present disclosure and cannot be used to limit the scope of the present disclosure. That is, any equivalent changes and modifications made according to the teachings of the present disclosure are still within the scope of the present disclosure. After considering the specification and practicing the disclosure herein, those skilled in the art will easily think of the implementation scheme of the present disclosure. This application is intended to cover any variation, use or adaptation of the present disclosure, which follows the general principles of the present disclosure and includes common knowledge or customary technical means in the art that are not recorded in the present disclosure. The description and examples are to be regarded as exemplary only, and the scope and spirit of the present disclosure are defined by the claims.

[0093] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0094] It will be easily understood by those skilled in the art that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A collaborative enhancement method for generating and solving mathematical test questions based on a large model, characterized in that: The method comprises: Acquire multiple sets of annotated data, input the multiple sets of annotated data into a basic large language model for fine-tuning training, and obtain a first large language model for generating math problems and solution processes; wherein the annotated data includes test question annotation instructions, math problems, and solution processes; generating a plurality of test question request instructions, inputting the plurality of test question request instructions into the first language model respectively, and outputting mathematical problems and solution processes corresponding to the test question request instructions, thereby obtaining a plurality of sets of composite data; Screening multiple sets of composite data to obtain a supervised fine-tuning dataset and a preference optimization dataset; optimizing and training the first large language model using the preference optimization dataset to obtain a second large language model; inputting the supervised fine-tuning dataset into the second large language model for the next round of iterative training; and repeating the process until the model converges to obtain a target large language model. The current test question request instruction is input into the trained target large language model, and a math test question including a math question and a solution process is output.

2. A collaborative enhancement method for generating and solving mathematical questions based on a large model according to claim 1, characterized in that: The method also includes: obtaining a forgetting score function by measuring the loss difference between the current round of synthesized data and the historical synthesized data in the output of the second largest language model, dynamically screening important key historical synthesized data based on the forgetting score function; and supplementing the key historical synthesized data into the supervised fine-tuning dataset to implement the next round of iterative training.

3. A collaborative enhancement method for generating and solving mathematical questions based on a large model according to claim 2, characterized in that: For any historical synthetic data , the forgetfulness score function for: ； in, Representation Model In historical synthetic data The loss value on ; Represents the model obtained after the current round of training based on the addition of the key historical synthetic data; Representation Model The loss value on the synthetic data in this round; Represents the model obtained after the current round of training without supplementing the key historical synthetic data; Indicates the question request instruction. Represents the corresponding mathematical problem and solution process.

4. A collaborative enhancement method for generating and solving mathematical questions based on a large model according to claim 1, characterized in that: The generation of the test question request instruction includes: The math problem is defined as multiple dimensional information, and the test question request instruction is generated by randomly combining the multiple dimensional information.

5. The collaborative enhancement method for generating and solving mathematical questions based on a large model according to claim 1 is characterized in that: A set of synthetic data includes a test question request instruction, a math question and a solution process corresponding to each other.

6. A collaborative enhancement method for generating and solving mathematical questions based on a large model according to claim 1, characterized in that: Multiple groups of data are combined for screening through a double verification mechanism; The dual verification mechanism includes: Obtaining a first signal based on a self-game verification mechanism; Obtaining a second signal based on a multi-agent collaboration mechanism; The synthesized data is screened and classified according to the first signal and the second signal.

7. A collaborative enhancement method for generating and solving mathematical test questions based on a large model according to claim 6, characterized in that: The self-game verification mechanism is: Copy the first language model of this round to generate a copy shadow model that independently obtains the solution process; Decompose the math questions in the synthetic data into the problem statement and the original solution process; Inputting the problem statement into the copy shadow model and outputting a comparative solution process; It is determined whether the logical steps and final answers of the original solution process and the comparative solution process are consistent, and a first signal is obtained.

8. The collaborative enhancement method for generating and solving mathematical questions based on a large model according to claim 6 is characterized in that: The multi-agent collaboration mechanism is: Introducing multiple agent models with different expertise, Use multiple intelligent agent models to evaluate the quality of math test questions in synthetic data and output evaluation opinions; Determine whether the evaluation opinions output by multiple intelligent agents are consistent and obtain a second signal.

9. A collaborative enhancement method for generating and solving mathematical test questions based on a large model according to claim 6, 7 or 8, characterized in that: The first signal is , the second signal is ;1 means consistent, 0 means inconsistent; like , then directly add the supervised fine-tuning dataset; like , then add the preference optimization data set after feedback optimization; like , then the synthesized data is discarded.

10. A mathematics test system based on a large language model, comprising a memory, a processor, and a computer program stored in the memory, characterized in that: The processor executes the computer program to implement the steps of the method according to any one of claims 1 to 9.

Images