Methods, computing devices, computer-readable storage media, and computer program products for synthesizing training datasets.

Abstract

This invention relates to computer systems utilizing computational models, and discloses methods, computing devices, computer-readable storage media, and computer program products for synthesizing training datasets. One method for synthesizing training datasets includes: iteratively determining multiple word sequences based on input by inference using one of a first model and a second model as the target model; generating answer words associated with the final answer by the second model based on the current word sequence obtained in the last iteration; and generating a synthesized word sequence based on the current word sequence and answer words obtained in the last iteration, as a training dataset. The method according to this invention overcomes the limitations of existing knowledge distillation methods, such as catastrophic forgetting and performance degradation of student models, and style mismatch, providing a training dataset synthesis framework that alternately uses teacher and student models during the synthesis sequence generation process, thereby improving the performance of student models.

Description

Technical Field

[0001] This invention generally relates to computer systems utilizing computational models, and more specifically to methods, computing devices, computer-readable storage media, and computer program products for synthesizing training datasets. Background Technology

[0002] In recent years, reasoning models have become the mainstream of flagship large language models (LLMs). The responses generated by these models are typically divided into two parts: "thinking content" for fine-grained reasoning and "final answer content" for providing solutions. By introducing explicit thinking processes, LLMs have achieved breakthroughs in logic-intensive tasks such as code generation.

[0003] Considering the differences in model capabilities and inference costs, a widely adopted paradigm in academia and industry is to use a large-scale model (teacher model) to generate training data and then perform supervised fine-tuning (SFT) on a small-scale model (student model) to achieve performance comparable to that of the large-scale model on a specific task.

[0004] The most common approach is to directly use high-quality synthetic data generated independently by the teacher model to train the student model using standard SFT. For example, code data generated by GPT-OSS-120B can be used to train Qwen3-8B. This method is common and effective for both Base and Instruction versions of the model.

[0005] However, when applied to emerging inference models like Qwen3-8B, directly using synthetic data generated by the teacher model for SFT often fails to improve students' reasoning abilities and may even lead to catastrophic forgetting. Experiments show that using standard SFT can even cause a 10.02% performance drop on OJBench. Furthermore, teacher and student models have significantly different thinking styles. For example, while GPT-OSS can generate correct problem-solving strategies, its text connection style is noticeably different from Qwen3. Directly using teacher data for training forces student models to adapt to the teacher model's style, which is unnecessary and can cause the model to deviate from its original distribution strategy.

[0006] There is a need in this field for techniques to generate training datasets that can improve the performance of trained student models. Summary of the Invention

[0007] This invention is provided to provide a technique for generating training datasets that can improve the performance of trained student models.

[0008] One aspect of the present invention provides a method for synthesizing a training dataset, comprising the following steps: S1: performing inference based on input by one of a first model and a second model as a target model to iteratively determine a plurality of lexical sequences, the input including a reasoning problem, wherein the first model has stronger reasoning ability and / or a larger parameter scale than the second model, the first model is used to generate lexicals associated with performing a task, and the second model is used to generate lexicals associated with style, the plurality of lexical sequences including lexicals generated by the target model based on the input and associated with the thinking part of thinking about the reasoning problem, S1 comprising, for the current iteration, performing the following steps: S11: generating lexical fragments by the target model based on the current lexical sequence completed up to the previous iteration, the current lexical sequence including the reasoning problem and all lexical sequences determined up to the previous iteration, the lexical fragments including a predetermined number of lexicals; S12: determining truncation positions in the lexical fragments by a boundary predictor associated with the target model. The method is as follows: S13: The boundary predictor is trained to determine, based on a classification algorithm, whether there are non-target type lexical units in the lexical segment that are different from the target type lexical units associated with the target model, wherein the truncation position is determined based on the starting point of the non-target type lexical units in the lexical segment; S14: Based on whether the truncation position is a lexical unit in the lexical segment, one of the first model and the second model is selected as the target model for the next iteration, and the boundary predictor associated with the selected model is selected as the boundary predictor for the next iteration; wherein the initial iteration is performed by the second model as the target model; S2: The second model generates answer lexical units associated with the final answer based on the current lexical unit sequence obtained in the last iteration; and S3: Based on the current lexical unit sequence obtained in the last iteration and the answer lexical units, a synthetic lexical unit sequence is generated as the training dataset.

[0009] As described above, step S12 includes: S121: calculating the probability by the boundary predictor that each lexical in the lexical segment is the starting point of the non-target type lexical; and S122: determining the truncation position based on the calculated probability.

[0010] As described above, step S122 includes: S1221: In response to determining that the non-target type word exists in the word segment, the position of the word with the highest probability is determined as the truncation position.

[0011] As described above, S122 includes: S1222: In response to determining that the non-target type word does not exist in the word segment, the position of the predefined indicator indicating the end of the segment is determined as the truncation position.

[0012] As described above, S13 includes: S131: determining the lexical preceding the truncation position as a lexical sequence associated with the current iteration; and S132: updating the current lexical sequence by adding the lexical sequence determined in the current iteration.

[0013] As described above, S14 includes: S141: In response to determining that the non-target type lexical exists in the lexical fragment, selecting the model that is different from the target model in the first model and the second model as the target model for the next iteration, and determining the corresponding boundary predictor for the next iteration.

[0014] As described above, S14 includes: S142: In response to determining that the non-target type lexical does not exist in the lexical fragment, retaining the target model as the target model for the next iteration, and retaining the boundary predictor associated with the target model as the boundary predictor for the next iteration.

[0015] As described above, step S1 is completed in response to the identification of a predefined token in the current lexical sequence or the current lexical sequence reaching a predetermined length.

[0016] The boundary predictor is trained by using the first model to label samples randomly sampled from the first model and the second model, and the labeled samples are used as training data to train the boundary predictor.

[0017] As described above, the training dataset is used to train a student model with inference ability and / or parameter size similar to the second model.

[0018] As described above, the student model includes the second model.

[0019] As described above, the student model is configured to perform at least one of a code generation task and a mathematical reasoning task.

[0020] In the method described above, the student model is configured to perform a mathematical reasoning task, wherein the lexical units associated with the task include descriptions involving formula derivation, and the lexical units associated with style include conjunctions.

[0021] Another aspect of the present invention provides a computing device, comprising: a processor; and a memory having instructions stored thereon, which, when executed by the processor, implement the steps of the method as described in any of the preceding claims.

[0022] Another aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described in any of the preceding claims.

[0023] Another aspect of the present invention provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the method described in any of the above embodiments.

[0024] The training dataset synthesized using the embodiments of the present invention is used to train the student model, avoiding the student model from adapting to the style of the teacher model and deviating from the original distribution strategy, thereby significantly improving the performance of the student model. Attached Figure Description

[0025] Figure 1 This is a diagram illustrating the knowledge distillation process based on a large language model.

[0026] Figure 2 This is an illustration of a data synthesis framework according to some embodiments of the present invention.

[0027] Figure 3 This is a flowchart of a method for synthesizing training datasets according to some embodiments of the present invention.

[0028] Figure 4 This is a flowchart of a first process associated with a method for synthesizing a training dataset according to some embodiments of the present invention.

[0029] Figure 5 According to some embodiments of the present invention Figure 4 A diagram of the first process in the process.

[0030] Figure 6 This is a flowchart of a second process associated with a method for synthesizing a training dataset according to some embodiments of the present invention.

[0031] Figure 7 The illustration shows an iterative process of collaborative data synthesis using a teacher-student model according to some embodiments of the present invention.

[0032] Figure 8 This is a block diagram of a computing device according to some embodiments of the present invention.

[0033] Figure 9 This is a block diagram of a computer-readable storage medium according to some embodiments of the present invention.

[0034] Figure 10 This is a block diagram of a computer program product according to some embodiments of the present invention.

[0035] In the accompanying drawings, the same reference numerals refer to the same elements. Detailed Implementation

[0036] In this application, ordinal numbers such as "first," "second," and "third" are used to distinguish different instances of objects with the same name. The ordinal numbers "first," "second," and "third" do not indicate a relative order of the indicated objects in time, space, sequence, or other aspects.

[0037] In this application, the term "first model" is used to denote a reasoning model with high reasoning ability and / or a large parameter size, and the term "second model" is used to denote a reasoning model with low reasoning ability and / or a small parameter size. As an example, "first model" can be a teacher model, and "second model" can be a student model. As an example, "first model" can include GPT, etc. (e.g., GPT-OSS), and "second model" can include Qwen3, etc. (e.g., Qwen3-8B).

[0038] In this application, the term "token" refers to the smallest meaningful unit that a reasoning model (e.g., a large language model) can understand and generate.

[0039] With the rapid development of artificial intelligence technology, deep neural network models have achieved excellent performance in various tasks. However, these high-performance models are usually complex in structure and have a large number of parameters, resulting in high computational overhead and storage requirements during the inference phase, which limits their application in resource-constrained environments. In order to reduce deployment costs and improve inference efficiency while maintaining model performance, the knowledge distillation method has been proposed.

[0040] Knowledge distillation, also known as model distillation, is a model compression method based on a teacher-student framework. This method typically uses a complex, high-performance teacher model (which can be a single model or an ensemble of multiple models) as a knowledge source, transferring the feature representations and output distribution information learned during training to a more streamlined student model. The student model also possesses the ability to map input data to probability distributions. This process can significantly reduce model size while maintaining the original accuracy level as much as possible, thereby achieving efficient reasoning in specific scenarios through model logic refinement.

[0041] Figure 1 This is a diagram illustrating the knowledge distillation process based on a large language model. For example... Figure 1As shown, the knowledge distillation process based on a large language model is implemented using teacher model 110, synthetic data module 120 and student model 130.

[0042] In the knowledge distillation process, the teacher model 110 first performs forward reasoning on the original data and outputs the corresponding results. Then, the synthetic data module 120 generates synthetic data based on the results and uses this synthetic data as a training dataset to train the student model 130, thereby guiding the parameter updates of the student model 130. This process aims to achieve reasoning capabilities as close as possible to those of the teacher model 110 with the lower computational cost of the student model 130.

[0043] Figure 2 This is an illustration of a data synthesis framework according to some embodiments of the present invention.

[0044] This invention proposes a collaborative data synthesis framework that, by alternately utilizing a first model 210 and a second model 220 to generate different types of lexical units in the synthesis sequence, preserves the reasoning ability of the first model 210 while ensuring that the data distribution conforms to the style of the second model 220. As an example and not a limitation, the first model 210 can be as described above regarding... Figure 1 The example of teacher model 110, and the second model 220 may be as described above regarding Figure 1 An example of the student model 130.

[0045] In an embodiment, the first model 210 and the second model 220 alternately receive the initial training set input 215 and generate lexical units representing thought processes accordingly. As an example, the initial training set input 215 may include a reasoning question. In an embodiment, the first model 210 is used to generate capability lexical units directly related to task solving. (Capability tokens) (such as code or reasoning steps) are used to ensure correctness. In an embodiment, the second model 220 is used to generate task-independent style tokens. (Style tokens) (such as tone words or conjunctions, like "wait" or "but") are used to keep the generated tokens consistent with the data distribution of the student model. In this embodiment, the second model 220 is also used to generate the final answer 255.

[0046] In this process, the goal of supervised fine-tuning (SFT) is to optimize the student model ( Output distribution With data distribution Alignment. The objective loss function of SFT is typically defined as:

[0047] (1).

[0048] Equation (1) defines the distillation loss of the student model: by minimizing the KL divergence between the output distribution of the student model and the output distribution of the teacher model, the student model can “imitate” the behavior of the teacher model while having a smaller structure and fewer parameters, thereby achieving knowledge transfer and model compression.

[0049] Given the significant style differences between the teacher and student models, directly optimizing the style loss may hinder model learning. Therefore, the core objective of this invention is to construct synthetic data such that the distribution of ability lexical units is sampled from the teacher model (…). The distribution of style terms was sampled from the student model (), The formal objectives are as follows:

[0050] (2),

[0051] (3).

[0052] This invention employs an alternating iterative approach to generate responses. For a single synthetic sample, embodiments of this invention generate multiple segments by alternately switching between a first model 210 and a second model 220. 235 and The sequence consisting of 225 represents the content of the thought process. 245:

[0053] (4).

[0054] in, This represents a style fragment generated by the second model 220. This represents the capability fragment generated by the first model 210.

[0055] Since the output of the inference model typically begins with a stylized phrase (such as "Okay, let's see"), in some embodiments, the generation process defaults to generating the first fragment by the second model 220. .

[0056] Each subsequent segment is generated based on all previously generated segments:

[0057] (5),

[0058] (6).

[0059] After the thinking process is completed, the second model 220 uses the initial training set input 215 and the thinking content as input. 245 generates the final answer 255. Initial training set input 215, thought content 245 and the final answer 255 can be combined to form a training dataset for training the student model. As an example, the training dataset can be used to train the second model 220 used in the process of synthesizing the training dataset. As another example, the training dataset can also be used to train another student model that is different from the second model 220.

[0060] Below are examples of different thought processes obtained by reasoning the same reasoning problem using the teacher model, student model, and teacher-student collaborative data synthesis framework.

[0061] Reasoning problem:

[0062] Given a string s, you may delete any palindromic substring in a single operation. Find the minimum number of operations required to remove the entire string.

[0063] The thought processes derived by the teacher model (e.g., GPT-OSS-120B) through independent reasoning:

[0064] We need to solve a problem...

[0065] We can consider Deleting s[i] alone: ​​dp[i][j] = 1 + dp[i+1][j] ...

[0066] We can try Pair s[i] with the following s[k]: dp[i][j] = min(dp[i][j], dp[i+1][k-1]) +dp[k+1][j]) ...

[0067] The thought process obtained by the student model (e.g., Qwen3-8B) through independent reasoning:

[0068] Okay, let's take a look. The problem is...

[0069] For example, If s[i] == s[j], then perhaps dp[i][j] = min(dp[i][j-1], dp[i+1][j], dp[i+1][j], dp[i+1][j]). 1][j-1]+0) ...

[0070] Or, here's an idea: dp[i][j] can be min(dp[i][j-1], dp[i+1][j], dp[i+1]). [j-1]) ...

[0071] The thought content derived from the teacher-student collaborative data synthesis framework (e.g., GPT-OSS-120B+ Qwen3-8B):

[0072] Okay, let's take a look. The problem is...

[0073] For example, Deleting s[i] alone: ​​dp[i][j] = 1 + dp[i+1][j] ...

[0074] Or, here's an idea: Pair s[i] with the following s[k]: dp[i][j] = min(dp[i][j], dp[i+ 1][k-1]+dp[k+1][j]) ...

[0075] In the above discussion, the underlined parts represent ability-related lexical units, while the ununderlined parts represent style-related lexical units. The purpose of the teacher-student collaborative data synthesis framework in this application is to generate ability-related lexical units from the teacher model and style-related lexical units from the student model.

[0076] According to one aspect of the present invention, a method for synthesizing training datasets is provided.

[0077] Figure 3 This is a flowchart of a method for synthesizing training datasets according to some embodiments of the present invention.

[0078] In some examples, the method may include step S1: inferring from the input using one of a first model and a second model as the target model to iteratively determine multiple lexical sequences. The input may include a reasoning problem. The first model may have stronger reasoning capabilities and / or a larger parameter scale than the second model. The first model may be used to generate lexicals associated with performing the task, and the second model may be used to generate lexicals associated with style. The multiple lexical sequences may include lexicals generated by the target model based on the input and associated with the thinking portion of the reasoning problem.

[0079] In some examples, step S1 may include, for the current iteration, performing step S11: generating a word fragment from the target model based on the current word sequence completed up to the previous iteration. The current word sequence may include the reasoning question and all word sequences determined up to the previous iteration. The word fragment may include a predetermined number of words.

[0080] In some examples, step S1 may include, for the current iteration, performing step S12: determining the truncation position in the lexical segment by a boundary predictor associated with the target model. The boundary predictor may be trained to determine, based on a classification algorithm, whether there are non-target type lexical segments in the lexical segment that differ from the target type lexical segments associated with the target model. The truncation position may be determined based on the starting point of the non-target type lexical segment.

[0081] In some examples, step S1 may include performing step S13 for the current iteration: determining the lexical sequence associated with the current iteration based on the truncation position to obtain the current lexical sequence.

[0082] In some examples, step S1 may include performing step S14 for the current iteration: selecting one of the first model and the second model as the target model for the next iteration based on whether the truncation position is a word in a word segment, and selecting the boundary predictor associated with the selected model as the boundary predictor for the next iteration.

[0083] In some examples, the initial iteration can be performed by a second model as the target model.

[0084] In some examples, the method may include step S2: generating answer lexical units associated with the final answer by a second model based on the current lexical sequence obtained in the last iteration.

[0085] In some examples, the method may include step S3: generating a synthetic lexical sequence based on the current lexical sequence and the answer lexical obtained in the last iteration, as a training dataset.

[0086] According to some embodiments, a first model with strong reasoning ability and / or a large parameter size generates capability lexical units associated with the task, while a second model with weaker reasoning ability and / or a smaller parameter size generates style lexical units associated with the style. The capability and style lexical units form the thought content, and the second model generates the final answer based on the reasoning question and the thought content. This allows the second model to maintain the student model's style during the reasoning process. The training dataset obtained through this process, when used to train the student model, can prevent the student model from deliberately adapting to the teacher model and deviating from its own distribution strategy, thereby improving the performance of the trained student model.

[0087] In some examples, step S1 is completed in response to the identification of a predefined token in the current lexical sequence or the current lexical sequence reaching a predetermined length. In some examples, the boundary predictor is trained by labeling samples randomly sampled from the first and second models using a first model, and using the labeled samples as training data to train the boundary predictor.

[0088] In some examples, the training dataset can be used to train a student model with reasoning ability and / or parameter size similar to the second model. In some examples, the student model may include the second model. In some examples, the student model may be configured to perform at least one of a code generation task and a mathematical reasoning task. In some examples, the student model may be configured to perform a mathematical reasoning task, wherein the lexical units associated with the task include descriptions involving formula derivation, and the lexical units associated with style include conjunctions.

[0089] The process of synthesizing the training dataset according to the method of the present invention is described in summary in Algorithm 1 below.

[0090]

[0091] It should be understood that the above algorithms are illustrative rather than restrictive and are not intended to limit the scope of the invention. Embodiments of the invention may include more or fewer steps than those shown in the above algorithms, and these steps may not be performed in the order shown.

[0092] In the example, as the initial iteration, the second model 220, as the target model, performs reasoning based on the input to obtain an initial fragment of the thought content. This initial fragment includes a predetermined number of lexical units. In the initial iteration, the input only includes the reasoning question. The initial fragment generated by the second model 220 is a style fragment.

[0093] A specially trained boundary predictor is used to determine the truncation points in the generated fragments. For teacher-generated fragments... Using the ability word boundary predictor Identify the position of the last ability word in the fragment, retain that position and the content before it, and truncate the subsequent content. For student-generated fragments... Using style lexical boundary predictor Identify the position of the last style term in the fragment, retain that position and the content before it, and truncate the subsequent content. Formally, the final retained fragment is calculated as follows:

[0094] (7).

[0095] The boundary predictor is implemented as a capability-level classification model (similar to a pointer network in an extractive question-answering task). Training data for training the boundary predictor is obtained by randomly labeling, for example, 100k teacher- and student-generated segments using a prompting teacher model. As an example and not a limitation, the boundary predictor can be trained on Qwen3-0.6B-Base. This is a relatively small model with a short input length, ensuring the operational efficiency of the data synthesis framework of this invention.

[0096] As examples rather than limitations, Tables 1 and 2 show the cue words used to train the style lemma boundary predictor and the ability lemma boundary predictor, respectively.

[0097] Table 1. Cue words used to train the style lemma boundary predictor

[0098]

[0099] Table 2. Cue words used to train the ability lexical boundary predictor

[0100]

[0101] In the initial iteration, the style lexical boundary predictor determines whether a capability lexical exists in the initial segment and determines the truncation position based on the starting point of the capability lexical. Based on this truncation position, the lexical sequence associated with the initial iteration is determined. Based on this truncation position, either the first model 210 or the second model 220 is selected as the target model for the next iteration, and the associated boundary predictor is selected as the boundary predictor for the next iteration.

[0102] In multiple iterations, the first model 210 and the second model 220 gradually determine the word sequence to be added to the thought content. The iteration of the thinking process ends when the thought content reaches a predefined length or when a special identifier appears in the thought content.

[0103] The second model 220 generates the final answer content based on the word sequence obtained up to the completion of the last iteration. The word sequence obtained up to the completion of the last iteration includes the reasoning question and the words that were determined to be added to the thinking content in each iteration.

[0104] A new word sequence is synthesized based on the word sequence obtained up to the completion of the last iteration and the generated final answer content, and used as the training dataset for training the student model. It should be noted that the student model used to train this training dataset can be a second model 220 used in the process of synthesizing the training dataset, or it can be another student model.

[0105] The synthesized training dataset is used to train a student model that can be configured to perform at least one of a code generation task and a mathematical reasoning task. Taking a student model configured to perform a mathematical reasoning task as an example, in this case, the capability lexical units in the data synthesis framework refer to content such as formula derivation that are directly related to the task, while the style lexical units refer to content such as conjunctions.

[0106] As an example and not a limitation, Table 3 shows the thought content generated by the student model, the teacher model, and embodiments of the present invention.

[0107] Table 3. Thought content generated from the student model, teacher model, and embodiments of the present invention

[0108]

[0109]

[0110] The underlined parts in the above discussion are style-related terms, while the ununderlined parts are ability-related terms.

[0111] The following will combine Figures 4-6 Describe in detail the functions and implementation details of each functional module in Algorithm 1.

[0112] Figure 4 This is a flowchart of a first process associated with a method for synthesizing a training dataset according to some embodiments of the present invention. The first process may be... Figure 3 The specific implementation of steps S12 and S13 in the method is described, but the scope of the present invention is not limited thereto.

[0113] In an embodiment, the first process may include step S121: calculating the probability that each word in the word segment is the starting point of a non-target type word using a boundary predictor. In an embodiment, the first process may include step S122: determining the truncation position based on the calculated probabilities. Steps S121 and S122 may be specific implementation details of line 9 in Algorithm 1.

[0114] In one embodiment, the first process may include step S131: determining the lexical units preceding the truncation position as the lexical sequence associated with the current iteration. In another embodiment, the first process may include step S132: updating the current lexical sequence by adding the lexical sequence determined in the current iteration. Steps S131 and S132 may be specific implementation details of lines 10-11 in Algorithm 1.

[0115] The following will combine Figure 5 Explain the example implementation of steps S121, S122, S131, and S132.

[0116] Figure 5 According to some embodiments of the present invention Figure 4 A diagram of the first process in the process.

[0117] In the example, by the second model (e.g., Figure 2 The style fragment generated by the second model (220) in the i-th iteration includes 5 lexical units. , , , and The probability of each lexical term serving as the starting point of a capability lexical term is calculated by the style lexical boundary predictor. , , , and .in, At its highest level, correspondingly, the truncation position is determined as a lexical unit. Place. Accordingly, discard the word element. and And the word before the truncated position , and This sequence is concatenated to the word sequence determined in the previous iteration before the deadline, to update the current word sequence. Here, word... - It refers to the lexical units included in the reasoning problem. - It is the sequence of lexical terms that is finally attached to the capability fragment generated by the first model 210 in the (i-1)th iteration.

[0118] It should be noted that during the iterative generation of segments by the first model 210 or the second model 220, the associated boundary predictor may also determine the truncation position as after the last word in the segment. In this case, no words are discarded, and all words included in the segment generated in the current iteration are appended to the word sequence to update the current word sequence.

[0119] Figure 6 This is a flowchart of a second process associated with a method for synthesizing a training dataset according to some embodiments of the present invention. This second process may be... Figure 4 Step S122 and in the first process Figure 3 The specific implementation of step S14 in the method is described, but the scope of the present invention is not limited thereto.

[0120] In an embodiment, the second process may include, at box 601, determining whether there are non-target type lexical elements in the lexical fragment.

[0121] In one embodiment, in response to determining that a non-target type word exists in the word segment at box 601, the second process may include step S1221: determining the position of the predefined indicator indicating the end of the segment as the truncation position. In another embodiment, in response to determining that a non-target type word exists in the word segment at box 601, the second process may include step S141: selecting the model that differs from the target model in the first and second models as the target model for the next iteration, and determining the corresponding boundary predictor for the next iteration. This step S141 may be a specific implementation detail of lines 12-18 in Algorithm 1.

[0122] In one embodiment, in response to determining that no non-target type lexical unit is present in the lexical segment at box 601, the second process may include step S1222: determining the position of the predefined indicator indicating the end of the segment as the truncation position. In another embodiment, in response to determining that no non-target type lexical unit is present in the lexical segment at box 601, the second process may include step S142: retaining the target model as the target model for the next iteration, and retaining the boundary predictor associated with the target model as the boundary predictor for the next iteration.

[0123] Figure 6The second process illustrates that switching between the first model 210 and the second model 220 is not mandatory between different iterations. Switching to another model and correspondingly switching the boundary predictor is only necessary in the next iteration if the fragment generated by the second model 220 includes unwanted capability terms, or if the fragment generated by the first model 210 includes unwanted style terms.

[0124] As an example, when the fragment generated by the second model 220 includes only style lexics, the truncation position is determined to be the location of the sequence terminator [eos]. In the next iteration, the fragment is still generated by the second model 220, and the truncation position is determined by the style lexicographic boundary predictor. Alternatively, when the fragment generated by the first model 210 includes only capability lexics, the truncation position is determined to be the location of the sequence terminator [eos]. In the next iteration, the fragment is still generated by the first model 210, and the truncation position is determined by the style lexicographic boundary predictor.

[0125] The following is combined with Figure 7 Provide a specific example illustrating the iterative process of collaborative data synthesis in a teacher-student model.

[0126] Figure 7 The illustration shows an iterative process of collaborative data synthesis using a teacher-student model according to some embodiments of the present invention.

[0127] In the example, in iteration 1, the second model receives the terms included in the reasoning question. - As input, and through reasoning, a lexical fragment is generated, which includes lexical units. , and Subsequently, the style lexical boundary predictor associated with the second model determines the truncation position as a lexical. Accordingly, the lexical sequence associated with iteration 1 is determined to include only lexical units. The current word sequence is updated accordingly. The current word sequence after iteration 1 includes words. … and word elements .

[0128] In response to determining the existence of non-target lexical units (at least including lexical units) in iteration 1 In iteration 2, the model is switched to be based on the current word sequence by the first model. … , Perform reasoning to generate lexical fragments, which include lexical units. , and Subsequently, the capability lexical boundary predictor associated with the first model determined the truncation position as... Accordingly, the lexical sequence associated with iteration 2 is determined to include the lexical unit. and The current word sequence is updated accordingly. The current word sequence after iteration 1 includes words. … , , , .

[0129] In response to determining the existence of non-target lexical units (at least including lexical units) in iteration 2 In iteration 3, the model is switched to be based on the current word sequence by the first model. … , , , Perform reasoning to generate lexical fragments, which include lexical units. , and Subsequently, the style lexical boundary predictor associated with the second model determines the truncation position as a lexical. Accordingly, the lexical sequence associated with iteration 3 is determined to include the lexical unit. and The current lexical sequence is then updated accordingly. The current lexical sequence after iteration 3 includes lexical units. … , , , , , .

[0130] exist Figure 7 middle, The input prompt words (i.e., the reasoning question) are represented by these words. Indicates the second model in the 1st century. The first generation generated during the alternation process Each word element, Indicates the first model in the... The first generation generated during the alternation process There are 1000 lexical units. The maximum length generated in each alternation is 1000. ,exist Figure 7 Winning For example, but it should be understood that, It can be any positive integer. The specific number of adopted lemmas is determined and truncated by the boundary predictor. The diagram only illustrates the first three iterations; in actual runtime, this process continues until the answer generation is complete. It should be understood that in actual runtime, the first iteration may also use the first model instead of the second model to generate lemma fragments if needed.

[0131] According to another aspect of the present invention, a computing device is provided.

[0132] Figure 8 This is a block diagram of a computing device 800 according to some embodiments of the present invention.

[0133] The computing device 800 includes a processor 810 and a memory 820. The memory 820 stores instructions 825, which, when executed by the processor 810, implement the above-mentioned combination. Figure 3 , Figure 4 , Figure 6 The steps of each method or process described.

[0134] According to another aspect of the present invention, a computer-readable storage medium is provided.

[0135] Figure 9 This is a block diagram of a computer-readable storage medium 900 according to some embodiments of the present invention.

[0136] A computer-readable storage medium 900 stores a computer program 950. When executed by a processor, the computer program 950 implements the above-mentioned combination. Figure 3 , Figure 4 , Figure 6 The steps of each method or process described.

[0137] According to another aspect of the present invention, a computer program product is provided.

[0138] Figure 10 This is a block diagram of a computer program product 1000 according to some embodiments of the present invention.

[0139] Computer program product 1000 may include computer program 950. Computer program 950, when executed by a processor, implements the above-mentioned... Figure 3 , Figure 4 , Figure 6 The steps of each method or process described.

[0140] Table 4 below shows the experimental results obtained using embodiments of the present invention.

[0141] Table 4. Performance comparison between embodiments of the present invention and the SFT baseline.

[0142]

[0143] In Table 4, the in-domain test set refers to the code generation task, the out-of-domain test set refers to the mathematical reasoning task, and LCB is short for LiveCodeBench.

[0144] In terms of experimental model configuration, the teacher model used was GPT-OSS-120B, and the student model used was Qwen3-8B. DeepSeek-R1-0528 and Qwen3-235B were also introduced as alternative teacher models, and Qwen3-30B as an alternative student model, to verify the generality of the method.

[0145] For training dataset construction, the OpenThoughts and NVIDIA Nemotron open-source datasets were collected. Carefully designed prompts were used to guide the GPT-OSS-120B filter to select samples relevant to programming competition tasks. Original responses were discarded, retaining only the questions. Finally, 80,000 questions (including 37,000 deduplicated questions) were randomly sampled for generating responses and model training.

[0146] For the evaluation datasets, LiveCodeBench-V5 (August 2024 - February 2025), LiveCodeBench-V6 (February 2025 - May 2025), LiveCodeBench-Pro, and OJBench were used for in-domain evaluation (code generation tasks). For out-of-domain evaluation (auxiliary tasks), the Mathematical Task Evaluation Set (AIME-2024, AIME-2025, OlympiadBench) and the General Question Answering (GPQA) task were used. For multimodal data, only the plain text portion was used.

[0147] In the experiments, the maximum number of lexical units (k) per iteration was empirically set to 20. When the vocabulary of the student and teacher models differed, the last word was discarded to prevent semantic inconsistencies caused by subword mismatches. A vLLM-based implementation was used, with prefix caching enabled to support efficient model switching. Training was performed using the XTuner framework on 32 H200 GPUs. The batch size was set to 128, and the learning rate to 5e-5. To avoid underestimating the baseline model due to insufficient training, all models were trained for up to 9 epochs. For Qwen3-8B, the default maximum inference length of 40K and temperature of 0.6 were used.

[0148] The experiment compared the embodiments of the present invention with three types of SFT data synthesis settings: student-led, teacher-student collaborative, and teacher-led. The student-led setting includes: teacher-reference, where the teacher generates a reference answer, and the student generates the thought process and final answer based on it; and teacher-scoring, where the student generates five candidate options, and the teacher scores and selects the best option. The teacher-student collaborative setting includes: teacher-answer, where the student generates the thought content, and the teacher generates the final answer; and teacher-thinking, where the teacher generates the thought content, and the student generates the final answer. The teacher-led setting includes: teacher-only, where all training samples are entirely generated by the teacher; and teacher-mixed, where teacher-generated and student-generated samples are mixed in a 1:1 ratio.

[0149] As shown in Table 4, the experimental results indicate that, despite other synthesis methods leading to varying degrees of performance degradation, the embodiments of the present invention achieve consistent performance improvements across all four code datasets. Specifically, the embodiments of the present invention improve Qwen3-8B performance by 7.78%, 5.85%, 11.34%, and 6.68% on the three LiveCodeBench datasets and OJBench, respectively. In contrast, the teacher-scoring method provides only a slight improvement (0.71%), while the widely used teacher-only method results in a performance degradation of up to 10.02% on OJBench.

[0150] Experimental observations show that, generally, the higher the teacher involvement, the more significant the decline in student performance. However, the embodiments of this invention, despite using up to 77.65% teacher-generated words, did not cause any performance degradation; on the contrary, they steadily improved the model's performance.

[0151] Furthermore, embodiments of the present invention demonstrate excellent out-of-domain generalization ability. On the mathematics evaluation sets AIME-2024 and AIME-2025, the model trained using TESSY synthetic data improved performance by 3.75% and 0.93%, respectively, and maintained comparable performance on GPQA and OlympiadBench. Conversely, the Teacher-Only method, which relies on teacher data, resulted in performance degradation of 4.35%, 2.71%, and 2.05% on GPQA, AIME-2025, and OlympiadBench, respectively.

[0152] Therefore, it can be seen that using the embodiments of the present invention to synthesize training datasets for training student models significantly improves the performance of student models.

Claims

1. A method for synthesizing training datasets, characterized in that, Includes the following steps: S1: A plurality of lexical sequences are iteratively determined based on input, using one of a first model and a second model as the target model. The input includes a reasoning problem. The first model has stronger reasoning ability and / or a larger parameter scale than the second model. The first model generates lexicals associated with the task being performed, and the second model generates lexicals associated with style. The plurality of lexical sequences include lexicals generated by the target model based on the input and associated with the thinking portion of the reasoning problem. S1 includes performing the following steps for the current iteration: S11: The target model generates word fragments based on the current word sequence completed by the previous iteration. The current word sequence includes the reasoning question and all word sequences determined by the previous iteration. The word fragments include a predetermined number of words. S12: The truncation position in the word segment is determined by a boundary predictor associated with the target model, wherein the boundary predictor is trained to determine, based on a classification algorithm, whether there are non-target type words in the word segment that are different from the target type words associated with the target model, wherein the truncation position is determined based on the starting point of the non-target type words in the word segment. S13: Determine the lexical sequence associated with the current iteration based on the truncation position to obtain the current lexical sequence; S14: Based on whether the truncated position is a word in the word segment, select one of the first model and the second model as the target model for the next iteration, and select the boundary predictor associated with the selected model as the boundary predictor for the next iteration. The initial iteration is performed by the second model as the target model; S2: The second model generates answer lexical units associated with the final answer based on the current lexical sequence obtained in the last iteration; and S3: Generate a synthetic word sequence based on the current word sequence obtained in the last iteration and the answer word sequence, as the training dataset.

2. The method according to claim 1, characterized in that, S12 includes: S121: The boundary predictor calculates the probability that each word in the word segment is the starting point of the non-target type word; and S122: Determine the cutoff position based on the calculated probability.

3. The method according to claim 2, characterized in that, S122 includes: S1221: In response to determining that the non-target type word exists in the word segment, the position of the word with the highest probability is determined as the truncation position.

4. The method according to claim 2, characterized in that, S122 includes: S1222: In response to determining that the non-target type word does not exist in the word segment, the position of the predefined indicator indicating the end of the segment is determined as the truncation position.

5. The method according to claim 1, characterized in that, S13 includes: S131: Determine the lexical units preceding the truncation position as the lexical sequence associated with the current iteration; and S132: Update the current lexical sequence by adding the lexical sequence determined in the current iteration.

6. The method according to claim 1, characterized in that, S14 includes: S141: In response to determining that the non-target type lexical exists in the lexical fragment, select the model that is different from the target model in the first model and the second model as the target model for the next iteration, and determine the corresponding boundary predictor for the next iteration.

7. The method according to claim 1, characterized in that, S14 includes: S142: In response to determining that the non-target type lexical does not exist in the lexical fragment, the target model is retained as the target model for the next iteration, and the boundary predictor associated with the target model is retained as the boundary predictor for the next iteration.

8. The method according to claim 1, characterized in that, The S1 is completed in response to the identification of a predefined token in the current lexical sequence or the current lexical sequence reaching a predetermined length.

9. The method according to claim 1, characterized in that, The boundary predictor is trained by labeling samples randomly sampled from the first model and the second model using the first model, and using the labeled samples as training data to train the boundary predictor.

10. The method according to claim 1, characterized in that, The training dataset is used to train student models with inference capabilities and / or parameter sizes similar to the second model.

11. The method according to claim 10, characterized in that, The student model includes the second model.

12. The method according to claim 10, characterized in that, The student model is configured to perform at least one of a code generation task and a mathematical reasoning task.

13. The method according to claim 10, characterized in that, The student model is configured to perform mathematical reasoning tasks. Among them, the lexical units associated with the execution of tasks include descriptions involving formula derivation, and the lexical units associated with style include conjunctions.

14. A computing device, characterized in that, include: processor; as well as A memory having instructions stored thereon, which, when executed by the processor, implement the steps of the method as described in any one of claims 1-13.

15. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1-13.

16. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1-13.

Images