An error-driven training data set forming method, device and equipment

Abstract

The present application relates to the technical field of modeling data synthesis, and discloses a method, device and equipment for synthesizing training data based on error driving, which comprises the following steps: obtaining a question-standard model seed data pair, automatically modeling the question by using a target large language model, identifying and extracting error patterns of the model on variables, constraints and objective functions by comparing the generated model with the standard model; based on the error patterns, performing semantic evolution on the original question, synthesizing a new question containing specific semantic pitfalls, and generating a corresponding standard mathematical model; the synthesized data pair needs to pass the solver executability verification and bidirectional consistency verification, and the verified data pairs are collected to construct a high-value training data set directly aiming at model defects. The present application realizes the automatic and targeted synthesis of training data, can significantly improve the accuracy, robustness and training efficiency of the large language model in the optimization modeling task, and reduces the dependence on artificial labeling.

Description

Technical Field

[0001] This invention relates to the field of modeling data synthesis technology, and in particular to a method, apparatus, and device for synthesizing error-driven training datasets. Background Technology

[0002] Mathematical optimization modeling is central to operations research, management science, and industrial engineering. It abstracts real-world problems such as resource allocation, path planning, and production scheduling into mathematical models containing decision variables, objective functions, and constraints, and then seeks the optimal decision through an optimization solver. Traditionally, this modeling process has relied heavily on manual work by domain experts, which is time-consuming, labor-intensive, and has a very high barrier to entry, limiting the application of optimization techniques in a wider range of scenarios.

[0003] In recent years, large language models have demonstrated powerful capabilities in natural language understanding and generation, making automated modeling from natural language descriptions to optimization models a research hotspot. Automated optimization modeling aims to enable large language models to understand business problems described by users in natural language and automatically generate corresponding, executable mathematical model code (such as PuLP, Pyomo, or commercial solver formats), which is expected to greatly reduce the barrier to entry for optimization techniques.

[0004] However, training a large model capable of reliably performing automated optimization modeling faces a fundamental challenge: the scarcity of high-quality aligned data. Such data requires paired natural language problems and standard mathematical models. Existing methods for obtaining training data mainly fall into two categories:

[0005] (1) Forward data synthesis: Based on a small number of manually written seed problems, new problem descriptions are generated through semantic rewriting, synonym substitution, and scenario expansion, and then the standard model is annotated by experts or strong models. This method relies heavily on the quality and diversity of the initial seed data, and the data augmentation process may not be able to effectively cover various complex errors that the model may make in the future.

[0006] (2) Reverse data synthesis: Starting from existing mathematical models (such as textbook cases, open source problem libraries), a large language model is used to explain or describe the real-world problems corresponding to the model. This method can quickly expand the amount of data, but the generated problem descriptions are often too straightforward and have a single pattern. They lack the ambiguity, vagueness and incompleteness that are common in real user descriptions, making it difficult to train the model to handle error-prone scenarios.

[0007] A common drawback of existing technologies is that they are data-driven rather than problem-driven or error-driven. They fail to fully utilize the information-rich error signals generated by models in automated modeling tasks. Model errors are not random noise, but rather reflect structural flaws in their internal knowledge representation or reasoning logic. Ignoring these errors means that the training process is repeatedly teaching the model what it already knows, while undertraining its true weaknesses. This results in low sample efficiency, limited model generalization ability, and the need for massive amounts of data to achieve usable performance.

[0008] Therefore, there is an urgent need for a method that can automatically and efficiently generate highly targeted training data. This method should be able to accurately cover the error-prone points of the model in optimization modeling, thereby achieving better model performance with a smaller data scale and cost. Summary of the Invention

[0009] To overcome the shortcomings of the prior art, the present invention aims to provide an error-driven training dataset synthesis method, apparatus, and device, which aims to solve the problems of low training efficiency, poor generalization ability, and reliance on a large amount of manual annotation caused by the scarcity of high-quality, highly targeted training data and the inability of existing data synthesis methods to effectively cover the real weaknesses of the model when training large language models.

[0010] The first aspect of this invention provides an error-driven training dataset synthesis method, comprising the steps of: acquiring seed data pairs containing an original natural language problem and its corresponding original standard mathematical optimization model; automatically mathematically modeling the original natural language problem using a target large language model to obtain a generative model; comparing the generative model with the original standard mathematical optimization model to identify and extract structural error patterns generated by the target large language model during the modeling process; semantically evolving the description of the original natural language problem based on the structural error patterns to synthesize a new natural language problem, and simultaneously generating an updated standard mathematical optimization model corresponding to the new natural language problem to form a new synthesized data pair; performing solver executability verification on the updated standard mathematical optimization model in the new synthesized data pair, and employing a bidirectional consistency verification mechanism to verify the logical consistency from the new natural language problem to the updated standard mathematical optimization model, and then back from the updated standard mathematical optimization model; and collecting the new synthesized data pairs that have passed the executability verification and bidirectional consistency verification mechanisms to construct a training dataset for training a target large language model.

[0011] A second aspect of the present invention provides an error-driven training dataset synthesis apparatus, comprising: an acquisition module for acquiring seed data pairs containing an original natural language problem and its corresponding original standard mathematical optimization model; a modeling module for automatically mathematically modeling the original natural language problem using a target large language model to obtain a generated model; a comparison module for comparing the generated model with the original standard mathematical optimization model, identifying and extracting structural error patterns generated by the target large language model during the modeling process; an evolution module for semantically evolving the description of the original natural language problem based on the structural error patterns, synthesizing a new natural language problem, and simultaneously generating an updated standard mathematical optimization model corresponding to the new natural language problem to form a new synthesized data pair; a verification module for performing solver executability verification on the updated standard mathematical optimization model in the new synthesized data pair, and employing a bidirectional consistency verification mechanism to verify the logical consistency from the new natural language problem to the updated standard mathematical optimization model, and then back from the updated standard mathematical optimization model; and a collection module for collecting the new synthesized data pairs that have passed the executability verification and bidirectional consistency verification mechanisms to construct a training dataset for training a target large language model.

[0012] A third aspect of the present invention provides an electronic device comprising: a memory and at least one processor, the memory storing computer-readable instructions, the memory and the at least one processor being interconnected via a circuit; the at least one processor invokes the computer-readable instructions in the memory to cause the electronic device to perform the steps of the error-driven training dataset synthesis method described above.

[0013] A fourth aspect of the present invention provides a computer-readable storage medium storing computer-readable instructions that, when executed on a computer, cause the computer to perform the steps of the error-driven training dataset synthesis method described above.

[0014] Beneficial Effects: The error-driven training dataset synthesis method provided by this invention delivers significant multi-level and systematic technical effects by constructing an automated closed loop that identifies errors, generates data using errors, and rigorously verifies them. This fundamentally changes the way training data is prepared for large-scale optimization models. Specifically, traditional data augmentation methods (such as rewriting and back-translation) produce data that is random in difficulty and relevance, with many samples representing repetitions of already mastered knowledge for the model, resulting in sparse training signals. In contrast, this invention directly attacks the model's knowledge blind spots. Each synthesized sample precisely corresponds to one or more structural errors exposed by the model (such as omitted constraints or confused variable types). This maximizes the corrective effect of unit data on the model's shortcomings, achieving targeted treatment. Under the same data scale, training with datasets synthesized using this method results in faster model convergence and a higher performance ceiling on the test set, significantly improving data utilization and training efficiency.

[0015] Traditional methods for acquiring high-quality problem-model alignment data heavily rely on manual writing and verification by domain experts, which is costly, time-consuming, and limited in scale. In contrast, this invention transforms expert knowledge into an algorithmic process: error pattern recognition, operator-based semantic evolution, solver, and bidirectional verification. Once the process is established, massive amounts of high-quality data can be generated automatically and in batches based on a small amount of seed data, significantly reducing the human and time costs of data preparation. This makes it possible to build large-scale, professional optimization modeling datasets, breaking the data bottleneck in the development of this field.

[0016] Real-world optimization problems are often characterized by ambiguity, vagueness, and incompleteness, which is the root cause of errors in existing models. The semantic evolution steps of this invention (such as fuzzification, ambiguity reduction, and information reconstruction) systematically simulate these real-world text features, synthesizing complex samples rich in semantic traps. This allows the model to repeatedly learn, understand, and overcome these traps during training, essentially stress-testing its natural language understanding and symbolic reasoning abilities. Therefore, models trained on this dataset exhibit stronger robustness and stable modeling accuracy when faced with imprecise, redundant, or structurally complex real-world user problems, demonstrating significantly better generalization performance than models trained using traditional data.

[0017] The dataset generated by this method comes with rich metadata (error pattern types, evolution strategies, and validation confidence). This is not only a training set but also a teaching archive, which trainers can use to implement course learning (such as from single errors to multiple errors) or to perform reinforcement training on specific error types. At the same time, by analyzing the performance changes of the model on various error samples, its capability evolution and residual defects can be clearly diagnosed, providing clear guidance for model iteration and making the entire training process more transparent and controllable. The error-driven reverse synthesis paradigm established by this invention has universality in its core idea (using the structural errors of the model as signals to reverse construct targeted training samples). It can be seamlessly transferred to other structured generation tasks where the input is natural language and the output must follow strict syntax or logic, such as program code synthesis, text-to-SQL query, and automatic theorem proof. This provides an innovative and effective technical path to solve the common problem of scarcity of high-quality aligned data in more AI fields. Attached Figure Description

[0018] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0019] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, those skilled in the art can obtain other drawings based on these drawings without creative effort.

[0020] Figure 1 A flowchart illustrating an error-driven training dataset synthesis method provided in an embodiment of the present invention.

[0021] Figure 2 This is a schematic diagram of the error-driven training dataset synthesis device provided by the present invention.

[0022] Figure 3 This is a schematic diagram of the electronic device structure provided by the present invention. Detailed Implementation

[0023] This invention provides a method, apparatus, device, and storage medium for synthesizing error-driven training datasets. The terms "first," "second," "third," "fourth," etc. (if present)," in the specification, claims, and accompanying drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" or "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or device that includes a series of steps or units is not necessarily limited to those explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or devices.

[0024] Please see Figure 1 , Figure 1 A flowchart of an error-driven training dataset synthesis method provided by this invention is shown in the figure, which includes the following steps:

[0025] S10. Obtain seed data pairs containing natural language problem descriptions and their corresponding standard mathematical optimization models;

[0026] Specifically, the purpose of this step is to obtain initial, high-quality seed data pairs as the cornerstone of the entire method. These seed data pairs consist of a natural language problem description and its corresponding standard mathematical optimization model. In this embodiment, the sources of the seed data pairs can be diverse, including but not limited to:

[0027] Classic examples in textbooks and academic papers include production planning and transportation problems in linear programming, and knapsack and traveling salesman problems in integer programming. These problems are described in a relatively standardized manner, with standard mathematical models.

[0028] Open-source optimization problem libraries, such as MIPLIB, CBLIB, and QPLIB, typically provide mathematical model files, which require either writing or having experts generate corresponding natural language descriptions.

[0029] Internal historical accumulation: Records of optimization problems accumulated in the company's operations and corresponding solutions.

[0030] Artificially constructed small batches: A batch of problem pairs carefully compiled by domain experts, covering different types (linear, nonlinear, integer-constrained, etc.) and varying levels of difficulty.

[0031] The quality of seed data is crucial. Standard mathematical optimization models must undergo rigorous validation to ensure their accuracy and solvability by mainstream solvers (such as Gurobi, CPLEX, and SCIP). Meanwhile, the natural language problem description should be clear and unambiguous, but may include some business context to simulate real-world scenarios.

[0032] As an example, suppose we select a classic resource allocation problem from a textbook as the seed data pair, as shown below:

[0033] Natural Language Processing Problem Description (Seed): A factory produces two products, A and B. Producing each unit of product A requires 2 hours of labor and 1 kg of raw materials, yielding a profit of 30 yuan. Producing each unit of product B requires 1 hour of labor and 3 kg of raw materials, yielding a profit of 40 yuan. The factory has 100 hours of available labor and 90 kg of raw materials per week. Question: How should the factory allocate its weekly production of products A and B to maximize total profit?

[0034] Standard mathematical optimization model (seed):

[0035] text

[0036] Decision variables:

[0037] x_A: Production volume of product A (units / week), continuous, non-negative.

[0038] x_B: Production volume of product B (units / week), continuous, non-negative.

[0039] Objective function (maximization):

[0040] Max Profit = 30*x_A + 40*x_B

[0041] Constraints:

[0042] Manual constraint: 2*x_A + 1*x_B <= 100

[0043] Material constraint: 1*x_A + 3*x_B <= 90

[0044] Nonnegativity constraints: x_A>= 0, x_B>= 0.

[0045] S20. Utilize the target large language model to automatically perform mathematical modeling on the natural language problems in the seed data pairs to obtain a generative model;

[0046] In this embodiment, the natural language problem descriptions from all the seed data pairs obtained in step S10 are input one by one into the target large language model (i.e., the model that needs to be improved and trained). Special prompt words can be designed to require the model to output the complete mathematical model of the problem (e.g., in the form of Python PuLP library code, or structured mathematical expression text), thus obtaining the generated model.

[0047] S30. Compare the generated model with the standard mathematical optimization model to identify and extract the structural error patterns generated by the target large language model during the modeling process;

[0048] In this embodiment, the generative model generated by the target large language model is automatically compared with the standard mathematical optimization model provided in the seed data. This comparison is not a simple string comparison, but a semantically based structured comparison. It requires parsing the two models and extracting their respective sets of decision variables, objective function expressions, and constraint sets to obtain the structural error patterns generated by the target large language model during the modeling process. The specific steps include:

[0049] S31. Analyze the generative model and the standard mathematical optimization model respectively, and convert them into structured intermediate representations. The intermediate representations include at least: a set of decision variables, an objective function expression, and a set of constraints.

[0050] Specifically, the prerequisite for model comparison is to transform unstructured model descriptions (which may be a mixture of natural language, mathematical formulas, and text, or solver-specific code) into a unified, structured intermediate representation (IR), which is the key to achieving automated comparison.

[0051] For standard mathematical optimization models: Since the standard models in the seed data pairs are of high quality, their IRs can be pre-generated or obtained through a reliable resolver. IRs typically consist of three core components:

[0052] The set of decision variables, V_std, consists of a tuple containing the variable name, type (continuous, integer, 0-1, etc.), lower bound, and upper bound. Example: V_std = {('x_A', 'Continuous', 0, None), ('x_B', 'Integer', 0, 100)};

[0053] The constraint set C_std: Each constraint is a tuple containing the constraint name, a linear / nonlinear expression on the left-hand side, relational operators (<=,=,>=), and a constant on the right-hand side. Example: C_std = {('c1', '2*x_A + 1*x_B', '<=', 100),('c2', 'x_A + 3*x_B', '<=', 90)};

[0054] The objective function O_std is a tuple containing the optimization direction (min or max) and an expression. Example: O_std =('max', '30*x_A + 40*x_B').

[0055] For generative models: A model parser needs to be designed to process the output of the target large language model. This parser needs to be robust enough to handle non-standard formats, minor grammatical errors, or explanatory text that may exist in the model output. It may combine rule matching, lightweight parsing, and machine learning models. The goal is to extract the above IR components from the text as accurately as possible, denoted as V_gen, C_gen, and O_gen.

[0056] S32. Based on the intermediate representation, the generated model and the standard mathematical optimization model are compared item by item in three dimensions: variables, constraints, and objectives, and the difference features are calculated.

[0057] After obtaining IR_std and IR_gen, the system performs a fine-grained comparison across three dimensions to generate a list of differences in features, Diff_List.

[0058] The comparison of variable dimensions includes:

[0059] Missing / Redundant Variables: Check if a variable in V_std appears in V_gen (omission error) or vice versa (redundancy error). This is achieved through variable name matching (alias mapping needs to be considered).

[0060] Incorrect variable type: For common variables, compare their types. For example, if V_std contains integers (Integer) while V_gen contains continuous variables (Continuous), then record a difference feature (x_A, inconsistent types, Integer vsContinuous);

[0061] Variable boundary error: Compare whether the upper and lower bounds are consistent.

[0062] Constraint dimension comparison (this is a high-risk area for errors) includes:

[0063] Constraint Missing / Redundant: Similar to variable comparison, this checks the inclusion relationship of constraint sets. However, constraint comparison is more complex because even when describing the same constraint, expressions may be written in different but mathematically equivalent forms (e.g., 2x + y <= 10 vs. x + 0.5y <= 5). Therefore, normalization and equivalence checks are needed. First, the expression is normalized (e.g., merging like terms to a standard form). Then, symbolic equivalence is checked or numerical sampling is performed to determine whether two constraints are substantially the same. Only constraints that still cannot match after normalization are judged as missing or redundant. For a missing constraint c, the difference characteristic (constraint missing, c) is recorded.

[0064] Constraint expression error: For matching constraints (judged by name or equivalence), compare the coefficients, relation signs, and right-hand constants of the normalized left-hand expression in detail. Any inconsistencies are recorded as differences, such as (constraint c1, coefficient error, x_A coefficient should be 2 but is actually 1) or (constraint c2, direction error, should be <= but is actually >=).

[0065] A comparison of objective function dimensions includes:

[0066] Incorrect optimization direction: directly comparing O_std[0] with O_gen[0];

[0067] Target expression error: The normalized expression is compared and the coefficients are identified as incorrect or the structure is incorrect (such as missing terms).

[0068] Furthermore, to address more complex model differences, the system can employ more advanced comparison strategies. For example, each model can be viewed as a bipartite graph, with nodes divided into variable nodes and constraint nodes, and edges representing the occurrence and coefficients of variables within constraints. By calculating the graph edit distance or the maximum common subgraph between two graphs, differences can be comprehensively measured, and inconsistent substructural regions can be located. Another approach is to convert the model's IR (Inverse Relationship) into feature vectors (e.g., containing variable type distribution, number of constraints, average constraint density, frequency of specific keywords, etc.), and quickly filter potentially significantly different samples by calculating the cosine or Euclidean distance between vectors before performing the aforementioned refined comparison.

[0069] S33. According to the preset error classification rules, the difference features are mapped to specific structural error pattern types, and the context fragments in the original natural language question that trigger the structural error pattern are recorded.

[0070] This step maps technical differences to business-related error patterns and establishes a link between errors and problem text. Specifically, the system maintains a pre-defined error classification mapping table, which defines which error pattern each type of difference should belong to. For example:

[0071] The structural error pattern corresponding to the difference feature (inconsistent variable types, Integer vs Continuous) is: incorrect variable type;

[0072] The structural error pattern corresponding to the difference feature (missing constraints, c) is: missing constraints;

[0073] The structural error pattern corresponding to the difference feature (constraint c1, direction error) is: constraint direction error.

[0074] Furthermore, for subsequent reverse synthesis, it is necessary to know which descriptions in the original question caused the error. The system uses attention backtracking or keyword association techniques to record contextual fragments in the natural language question descriptions that trigger the structural error pattern.

[0075] The attention backtracking technique involves analyzing the attention distribution of the model towards each token in the input question when generating erroneous parts (such as omitted constraints), provided the target large language model supports output attention weights. Segments with low attention may be ignored by the model, and these segments are the triggers.

[0076] The keyword association technique (more general) establishes a connection between the question text and the model's information relationship (IR) elements. While parsing the original natural language question, simple semantic role labeling is performed simultaneously to identify phrases describing resources (e.g., 100 hours of human time) and objectives (e.g., maximizing profit). When a structural error pattern (e.g., constraint omission) is determined to be associated with a specific element in the IR (e.g., constraint c1: 2*x_A + x_B<= 100), the system searches for the text fragment in the original question associated with that element (e.g., 100 hours of human time). This fragment is extracted as the trigger text fragment. For example, it might be recorded as: Structural Error Pattern: Constraint Omission; Associated Model Element: c1; Trigger Text Fragment: 100 Hours of Human Time; Error Confidence: High.

[0077] All identified instances of structural error patterns, along with their triggering text fragments, original problem IDs, and details of differential characteristics, are structurally stored in an error pattern library that supports querying and analysis. For example, it's possible to statistically analyze which words (such as "at least" or "no more than") in the problem description most frequently co-occur with "constraint direction error." Through aggregated analysis of historical errors, a confidence level or frequency can be calculated for each error pattern. This helps prioritize common and high-confidence structural error patterns during subsequent reverse synthesis, improving the efficiency and relevance of data synthesis.

[0078] Through the detailed steps S31 to S33 above, this invention achieves accurate, automated, and structured identification and extraction of model errors, laying a solid data foundation for subsequent error-driven data synthesis. This process transforms messy modeling errors into system-understandable and operable knowledge, which is the core technical aspect that distinguishes this method from random data augmentation.

[0079] As an example, if we input the natural language problem description from the seed data pair into the target large language model, and assume the model returns the following incorrect generative model:

[0080] Decision variables:

[0081] x_A: Production volume of product A, continuous.

[0082] x_B: Production volume of product B, continuous.

[0083] Objective function (maximization):

[0084] Max Profit = 30*x_A + 40*x_B

[0085] Constraints:

[0086] Manual constraint: 2*x_A + 1*x_B <= 100

[0087] # Note: The model is missing raw material constraints!

[0088] Nonnegativity constraints: x_A>= 0, x_B>= 0.

[0089] By automatically comparing the constraint set {manual constraints, material constraints, non-negative constraints} in the standard model with the constraint set {manual constraints, non-negative constraints} in the generated model, the system identified a constraint omission error. Specifically, the "material constraint: 1*x_A + 3*x_B <= 90" was omitted. At this point, the system records an error in the error mode database.

[0090] Error type: Missing constraint;

[0091] Related fragment of the original question: The raw material weighs 90 kilograms;

[0092] Possible triggers: The model may misinterpret the raw material weight of 90 kg as background information rather than a constraint, or its attention mechanism may fail to associate this information with the constraint.

[0093] S40. Based on the structural error pattern, the description of the original natural language problem is semantically evolved to synthesize a new natural language problem. At the same time, an updated standard mathematical optimization model corresponding to the new natural language problem is generated to form a new synthetic data pair.

[0094] In this embodiment, step S40 is the core manifestation of the error-driven concept of the present invention. Its goal is not to randomly generate new questions, but to purposefully construct new natural language questions that can accurately trigger specific error patterns. Specifically, it includes the following steps:

[0095] S41. Select one or more structural error patterns from the error pattern library and obtain their corresponding causal text fragments and original natural language questions;

[0096] S42. Based on the type and complexity of the selected structural error mode, select one or more semantic evolution operators from the preset evolution strategy library;

[0097] S43. Taking the inducing text fragment and the complete description of the original natural language question as input, apply the selected semantic evolution operator to generate one or more candidate new natural language questions;

[0098] S44. Analyze the new natural language problem and determine whether its mathematical structure has undergone substantial changes compared to the original natural language problem;

[0099] S45. If the new natural language problem does not undergo substantial changes in its mathematical structure, then the standard mathematical optimization model is directly copied or slightly adjusted as the updated standard mathematical optimization model.

[0100] S46. If the new natural language problem undergoes substantial changes in its mathematical structure, then based on the evolved semantics, an updated standard mathematical optimization model is derived through preset rules or an auxiliary large language model.

[0101] S47. Combine the new natural language problem with the corresponding updated standard mathematical optimization model to form a new synthetic data pair.

[0102] Specifically, in step S41, the system selects structural error patterns from the error pattern library. The selection strategy can be: frequency-based selection (prioritizing samples corresponding to high-frequency errors); difficulty-based selection (selecting entries with high model error confidence but complex structural error patterns); or combination-based selection (selecting multiple logically coexisting structural error patterns to prepare for multi-error synthesis). In this embodiment, after selecting a structural error pattern instance, the system reads its associated natural language problem ID and triggering text fragment. For example, the new structural error pattern "constraint omission" is associated with natural language problem P001, and its triggering text fragment is "raw material is 90 kg".

[0103] In step S42, the system maintains a semantic evolution strategy library containing various semantic evolution operators. Each operator is a text transformation rule designed to increase the deceptiveness of the problem description in a specific way. In this embodiment, the semantic evolution operators include, but are not limited to, the following types: Ambiguity operators: adding ambiguous qualifiers such as approximately, close to, or around before precise numbers or definite descriptions; Ambiguity operators: replacing explicit words with polysemous terms, or implying constraints in complex sentence structures or background information; Information reconstruction operators: changing the position, order, or sentence structure of key constraint information in the text, including moving it to the beginning or end of paragraphs or embedding it in clauses of long sentences; Redundant information injection operators: adding background descriptions, details, or comments unrelated to the core mathematical structure to distract attention; Negation and double negation operators: using negative sentence structures to express constraints, or using double negations to increase the complexity of understanding; Terminology substitution operators: replacing common vocabulary with more specialized, obscure, or domain-specific terms.

[0104] Based on the type and complexity of the selected structural error pattern, this embodiment automatically matches or combines multiple semantic evolution operators. For example, for errors that ignore precise numerical constraints, fuzzification operators (such as changing 90 kg to approximately 90 kg) and information reconstruction operators (such as moving the constraint to the end of the paragraph) are suitable; for errors that confuse variable types, ambiguity operators (such as changing "must be an integer" to "must be in batches") and term substitution operators (such as replacing "product" with "SKU") are suitable; for errors in the direction of the objective function, negation and double negation operators are suitable (such as expressing maximizing profit as avoiding excessively low profits).

[0105] In step S43, the selected semantic evolution operator is applied to the original natural language problem description. There are two main implementation methods:

[0106] One approach is template-based filling (high controllability): This involves predefining text transformation templates for each type of "structural error pattern + semantic evolution operator" combination. The templates retain the skeleton of the original natural language problem and mark the slots that need modification. The system extracts key entities (such as numbers, resource names, and action verbs) from the triggering text fragments, then modifies and fills them according to the template rules.

[0107] As an example, here is a template for the fuzzification + information reconstruction operator for constraint omission:

[0108] Original sentence: [ORIGINAL_SENTENCE] (e.g., "The raw material is 90 kilograms.")

[0109] The revised sentence is: "In addition, regarding materials, the available quantity of [RESOURCE] is approximately [NUMBER] units."

[0110] The system fills in RESOURCE="raw materials" and NUMBER="90" to generate a new natural language question.

[0111] Another approach is the free generation method based on large language models (high diversity): constructing sophisticated prompt words to guide a generative large language model (which can be a dedicated text generation model or the target large language model itself) to create content.

[0112] As an example, the prompt is as follows: You are an operations research problem editing expert. Please modify the given optimization problem description based on the following instructions:

[0113] [Original Question]: {Original Natural Language Question Text}

[0114] [Revision Requirements]: Please make the description of this problem more likely to lead to "omitting constraints regarding the 'quantity of raw materials'" during AI modeling. Please use a more vague expression and place this information in an inconspicuous location. Ensure that the revised problem is mathematically equivalent to the original natural language problem, i.e., the core decision variables, constraints, and objective function remain unchanged.

[0115] [Revised Question]: {New Natural Language Problem Text}

[0116] Based on this prompt, the model generates diverse new descriptions. This method can produce more natural and diverse text, but the controllability and logical consistency may be slightly worse, requiring further verification.

[0117] Specifically, since evolution may introduce unintended semantic distortions, it is necessary to verify whether the new natural language problem remains equivalent to the original natural language problem. A lightweight logical consistency checker is employed here. This can be a rule-based system that checks whether key entities (variables, resources, goals) are preserved; or it can be a small, high-precision large language model for verification, asked: "Do the following two problem descriptions express the exact same model at the mathematical optimization modeling level? Problem A: [Original Problem], Problem B: [New Problem]". Only new natural language problem descriptions judged as "yes" or "highly likely to be the same" will proceed to the next stage. This step ensures the targeting of evolution; that is, this embodiment only changes the way the expression is presented to induce errors, without changing the essence of the problem.

[0118] As an example, if "maximizing profit" is rewritten as "making profit reach its highest level", the verification will consider the new problem to be just a "repackaged" problem of the original problem. The essence of the problem remains unchanged and the core mathematical structure is unchanged. It can be entered into the next stage as a new natural language problem and marked as passing the verification.

[0119] For example, if "one raw material" is broken down into "two sub-raw materials" and a clear consumption coefficient is given, the verification will consider that the new problem, while retaining the core logic of the original problem (such as limited resources and demand satisfaction), has reasonably added details or changed the form of constraint expression. Moreover, these changes are clear and unambiguous, that is, the essence of the problem remains unchanged but the core mathematical structure has changed. It can also be regarded as a new natural language problem and enter the next stage and be marked as passing the verification.

[0120] For example, the original natural language problem fragment, "...the goal is to maximize total profit," is inappropriately evolved using "negation and double negation operators" to generate a new natural language problem fragment: "...our goal is to avoid profits not being minimum, that is, we do not want to minimize profits." Validation reveals an inherent logical conflict in this description. "Avoiding profits not being minimum" is a double negation, its direct meaning being "we want profits to be minimum" (i.e., minimize). However, this directly contradicts the latter part, "we do not want profits to be minimized." The entire sentence makes it impossible to determine whether the ultimate intention is maximization or minimization; for an optimization modeler, it's impossible to uniquely determine from this description whether the objective function should be max or min. Therefore, a definitive "standard mathematical model" cannot be generated. This evolution changes the essence of the problem because it fails to clearly and unambiguously convey the core mathematical intention of "maximizing profit," instead introducing irresolvable logical confusion. If such a sample enters the training set, it will send contradictory signals to the model, impairing learning performance. Therefore, this new natural language problem is rejected from the next stage and marked as rejected for validation.

[0121] The verification process safeguards the essential mapping between the problem description and the definable mathematical model. Any evolution that disrupts this mapping, no matter how misleading its initial intention, will be filtered out, thus ensuring that every sample in the final synthesized training dataset is both a challenging question and a valid question with a standard answer.

[0122] Steps S44-S47 are crucial for ensuring the correctness of the answers in the new synthetic data pairs. Generating the updated standard mathematical optimization model corresponding to the new natural language problem involves two cases:

[0123] Scenario A: Semantic Equivalence Evolution (most common cases): After verification, the new natural language problem is mathematically equivalent to the original natural language problem, meaning there is no substantial change in its mathematical structure. In this case, the updated standard mathematical optimization model corresponding to the new natural language problem can be directly copied from the standard mathematical optimization model corresponding to the original natural language problem without modification. This is because the evolution goal of this embodiment is that the representation changes while the model remains unchanged.

[0124] For example, the original natural language problem is "maximizing profit," while the new natural language problem is expressed as "avoiding excessively low profits," and its standard model is still the objective function Max Profit.

[0125] Scenario B: Semantic Enhancement Evolution (Rare Cases): Sometimes, to increase difficulty or realism, evolution introduces new constraints, changes variable types, or adjusts target coefficients. In such cases, the new natural language problem is deemed to have undergone a substantial change in mathematical structure. For example, "90 kg of raw materials" is clarified as "Raw material A has 50 kg, and raw material B has 40 kg." The total remains 90, but the constraints become two. In this situation, the updated standard mathematical optimization model corresponding to the new natural language problem needs to be precisely derived based on the original standard mathematical optimization model. The derivation method is as follows:

[0126] a) Based on a pre-defined natural language-mathematical model mapping rule base, the system parses and generates updated standard mathematical optimization models: The system has a built-in set of parsing rules from natural language to mathematical models. When specific patterns such as splitting, merging, percentage, and proportion are detected, the corresponding model transformation rules are triggered. For example, if "raw materials A and B are 50 and 40 kg respectively" are detected, the rule will split the original constraint 1*x_A + 3*x_B<= 90 into two constraints 1*x_A<= 50 and 3*x_B<= 40 (Note: This is just a possible rule; the actual business logic may be different; this is just an example). The rule base needs to be built by domain experts to ensure correctness.

[0127] (b) A trained and highly reliable auxiliary large language model is invoked. Taking the new natural language problem as input, the generated candidate standard model is pre-validated by the solver to ensure correctness, and then used as the updated standard mathematical optimization model. An independent, rigorously validated auxiliary large language model (which can be a highly accurate small model or a general large model refined with extensive SFT) is used as input to generate a candidate standard model. The key here is that this auxiliary large language model must be independent of the trained target large language model, and its reliability must be significantly higher than the target large language model to avoid error cycles. The generated candidate standard model must be validated by the solver and / or cross-validated with the model derived through rule derivation to ensure absolute accuracy.

[0128] Whether obtained through copying or derivation, standard mathematical models need to be converted into a uniform format required for downstream training and validation processes. This could be the code for a specific solver (such as PuLP or Pyomo), a standard mathematical programming language (such as LP or MPS format), or a structured intermediate representation (such as JSON format, containing explicit fields for variables, constraints, and objectives). Formatting ensures the standardization and usability of the dataset.

[0129] Through the detailed steps of S41 to S47 above, the present invention realizes the automated and controllable synthesis of structural error patterns into targeted training samples. Each newly synthesized data pair accurately contains one or more semantic traps, and its corresponding standard answer is clear and unambiguous, thus constituting a high-quality data pair for efficient training of the target large language model.

[0130] S50. Perform solver executability verification on the updated standard mathematical optimization model in the new synthetic data pair, and adopt a two-way consistency verification mechanism to verify the logical consistency from the new natural language problem to the updated standard mathematical optimization model, and then back from the updated standard mathematical optimization model.

[0131] Specifically, the automatically synthesized new data may contain logical errors or inconsistent expressions. This step establishes two firewalls to ensure the quality of the final dataset.

[0132] The first level is solver executability verification, a fundamental and mandatory check to ensure the validity of the synthesized data. Its core objective is to confirm that the standard answer prepared for the synthesized problem—that is, the generated updated standard mathematical optimization model itself—is well-defined, executable, and has meaningful solutions. If a mathematical model itself is unsolvable or flawed, then using it as a standard answer for training samples will be meaningless and may even mislead model training. Its specific steps include:

[0133] S51. Use a mathematical optimization solver to solve the updated standard mathematical optimization model in the synthetic data pair, and verify whether the model has a feasible solution and whether the solution process is error-free.

[0134] S52. If the model is infeasible, unbounded, or the solution reports an error, then the synthesized data pair is determined to be invalid.

[0135] Specifically, it automatically invokes one or more mathematical optimization solvers (such as Gurobi, CPLEX, SCIP, OR-Tools, etc.) to instantiate and solve a given updated standard mathematical optimization model, and comprehensively checks the status codes and output information returned by the solver.

[0136] As an example, the system will update the standard mathematical optimization model and input it according to the format required by the selected solver (such as .lp file, .mps file, or model built directly in memory via solver API). Then, it will automatically execute the solver command or call the solver function.

[0137] After the solution is completed, the system analyzes the solver's solution status. Key states include: Optimal: The solver has found the optimal solution; Feasible (for some solvers or settings): A feasible solution has been found, but it may not be proven to be optimal (e.g., when the time limit is reached); Infeasible: The model is infeasible, i.e., there is no set of decision variable values ​​that can simultaneously satisfy all constraints; Unbounded: The model is unbounded. For maximization problems, the objective function value can tend to positive infinity; for minimization problems, it can tend to negative infinity, which usually means that necessary constraints are missing; Error: The solution process failed due to model syntax errors, numerical problems (such as numerical overflow), license problems, or internal errors.

[0138] Based on the solution status, the system executes automated decision-making logic:

[0139] A solution is considered valid if and only if the solution state is Optimal or Feasible (depending on the preset acceptance criteria). This means that the model is not only syntactically correct, but also has at least one feasible solution and usually has a definite, finite optimal objective value. The data pair passes this round of verification and proceeds to the next round (bidirectional consistency verification).

[0140] If the solution status is Infeasible, Unbounded, or Error, the synthesized data pair is immediately deemed invalid and discarded without further processing. This is because: Infeasible: The standard answer is contradictory and cannot be used as the correct answer to any question; Unbounded: The standard answer is incomplete, lacking the constraints that should exist in real-world problems, and would transmit incorrect knowledge as a training sample; Error: The standard answer has technical flaws, cannot be reliably solved, and its correctness cannot be verified.

[0141] As an example, continuing to use the resource allocation problem as a basis, its correct standard mathematical model (denoted as Model_Correct) is:

[0142] Maximize Profit = 30*x_A + 40*x_B

[0143] Subject to:

[0144] 2*x_A + 1*x_B <= 100 (human resource constraint)

[0145] 1*x_A + 3*x_B <= 90 (Material constraint)

[0146] x_A>= 0, x_B>= 0

[0147] Solver verification of the model returns the state Optimal and yields the optimal solution x_A=30, x_B=20, Profit=1700. Therefore, the data pair containing Model_Correct will pass the verification in steps S51-S52.

[0148] The following are three situations that would lead to an "invalid" determination:

[0149] Example 1: Invalid due to infeasibility, incorrect standard mathematical model (Model_Infeasible): Based on Model_Correct, an incorrect constraint that contradicts other constraints is added.

[0150] Maximize Profit = 30*x_A + 40*x_B

[0151] Subject to:

[0152] 2x_A + 1x_B <= 100

[0153] 1*x_A + 3*x_B <= 90

[0154] x_A>= 40# New constraint: Requires A to produce at least 40 units.

[0155] x_B>= 30# New constraint: Requires B to produce at least 30 units.

[0156] x_A>= 0, x_B>= 0

[0157] S51 Solution Process: The solver attempts to find x_A and x_B that simultaneously satisfy all conditions. However, substituting the constraints x_A>=40 and x_B>=30 into the material constraint 1*x_A + 3*x_B<= 90, this single condition alone requires at least 40 + 90 = 130>90, which is impossible to satisfy.

[0158] S52 Decision: The solver quickly returns the status "Infeasible". The system determines that the data pair is invalid and discards it directly. This indicates that a logical error occurred when the previously generated standard mathematical optimization model was updated, leading to a contradiction in the answer itself.

[0159] Example 2: Invalid due to unboundedness, incorrect standard mathematical model (Model_Unbounded): Based on Model_Correct, key constraints are incorrectly omitted.

[0160] Maximize Profit = 30*x_A + 40*x_B

[0161] Subject to:

[0162] # Human constraints have been overlooked!

[0163] 1*x_A + 3*x_B <= 90# Only material constraints remain.

[0164] x_A>= 0, x_B>= 0

[0165] S51 Solution Process: The solver discovers that, under the only material constraints, as long as x_A becomes very large, although x_B will be limited, the total profit 30*x_A + 40*x_B can increase infinitely as x_A increases.

[0166] S52 Decision: The solver returned the status "Unbounded". The system determined that the data pair was invalid and discarded it directly. This indicates that the previously generated updated standard mathematical optimization model was incomplete and failed to correctly reflect all the constraints in the problem description.

[0167] Example 3: The model is deemed invalid due to a solution error. The incorrect standard mathematical model (Model_Error) is one that contains syntax errors or illegal constructions.

[0168] Maximize Profit = 30*x_A + 40*x_B

[0169] Subject to:

[0170] 2x_A + 1x_B <= 100

[0171] 1*x_A + 3*x_B <= 90

[0172] sqrt(x_A) + x_B <= 10# In standard linear programming solvers, sqrt is a non-linear term, and failing to enable this feature will result in an error.

[0173] x_A>= 0, x_B>= 0

[0174] Alternatively, the variable name may contain illegal characters (such as xA), or there may be a division by zero error.

[0175] S51 Solving Process: When the solver detects an error while parsing or preprocessing the model, it immediately terminates and returns an Error status, usually accompanied by an error message (such as "Nonlinear term not supported").

[0176] S52 Decision: The system detects an Error state, determines the data pair to be invalid, and discards it directly. This indicates a technical error occurred during model formatting or derivation.

[0177] The solver executability verification (S51-S52) acts as a mathematical correctness filter, ensuring from a purely computational mathematical perspective that the answers in the synthetic data pairs are consistent, complete, and achievable. This is an indispensable first step in building a high-quality training dataset, filtering out bad samples that are inevitably generated during the automatic generation process, thus ensuring that the data subsequently used to train the model is mathematically sound and reliable.

[0178] After passing the first-level solver executability verification, the standard mathematical model in the synthetic data pair has been proven to be correct and solvable. However, this only guarantees the correctness of the answer and does not verify whether our carefully designed test question, namely the new natural language problem, can stably and accurately guide a model to arrive at the preset answer. If the description of the new natural language problem itself contains hidden ambiguity, misleading information, or ambiguity, causing different models (or even the same model at different times) to produce multiple reasonable but different interpretations, then the value of the problem as a training sample will be greatly reduced, and may even introduce noise. Therefore, this embodiment also designs a bidirectional consistency verification mechanism to test the robustness and consistency of the logical mapping between the problem description and the standard answer. This mechanism simulates the role of an external referee to conduct a higher-level evaluation of the quality of the synthetic data, and its specific steps include:

[0179] S53. Input the new natural language problem in the new synthetic data pair into a large language model for validation, obtain the validation model generated therefrom, and calculate the similarity between the validation model and the updated standard mathematical optimization model in the new synthetic data pair.

[0180] S54. Use the solver to calculate the optimal objective function values ​​of the verification model and the updated standard mathematical optimization model under the same instance parameters, and compare whether the two optimal objective function values ​​are consistent within the preset error range;

[0181] S55. If the similarity is higher than a preset threshold and the two optimal objective function values ​​are consistent, then the newly synthesized data pair is determined to have passed the bidirectional consistency verification.

[0182] Step S53 aims to evaluate whether an unfamiliar model (the validation large language model) can reproduce an interpretation highly consistent with the updated standard mathematical optimization model after reading a new problem. In this step, the new natural language problem from the synthetic data pair is submitted as input to an independent validation large language model. This model should be designed to be independent of the target large language model to avoid self-verification loops. It can be another model with the same architecture, a more powerful general-purpose model, or a specially tuned validation model. The system uses standardized prompts (e.g., "Please transform the following optimization problem into a mathematical model") to obtain its generated validation model.

[0183] Then, the system automatically calculates the structured differences between the verification model and the updated standard mathematical optimization model. Specific methods may include: IR-based comparison: as described in step S32, the two models are parsed into structured IRs (variable set, constraint set, objective function), the Jaccard similarity coefficient, edit distance, or graph-based similarity is calculated, and a quantified similarity score Sim_Score is output.

[0184] Step S54 aims to perform ultimate verification at the level of the final mathematical results: even if the two models have slightly different expressions, if they essentially characterize the same optimization problem, then their optimal solutions should be the same. To ensure a fair comparison, the models need to be filled with the same instance parameters, which usually means that if the problem description contains abstract symbols (such as unit profits of 30, 40), then these specific values ​​are used directly; the system inputs the verification model and the updated standard mathematical optimization model into the solver for solving; the optimal objective function values ​​after solving the two models are obtained, denoted as Obj_Std and Obj_Ver; since numerical calculations may have small floating-point errors, or the model generated by the verification model may have non-essential equivalent transformations (such as multiplying both sides of the constraint by a positive number), directly performing an absolute equivalence comparison may be too strict. Therefore, the system uses relative error judgment: whether they are consistent = (|Obj_Std - Obj_Ver|) / (max(|Obj_Std|, |Obj_Ver|,1))<ε, where ε is a preset small threshold (such as 0.001 or 1%), which ensures that the two models are equivalent from the perspective of optimization results.

[0185] Step S55 is the final decision-making step, which comprehensively considers both form (model structure) and substance (optimization result), and is achieved by simultaneously satisfying the following conditions:

[0186] (a) Similarity condition: Sim_Score is higher than the preset threshold T_sim (e.g., T_sim = 0.85), and (b) Target value consistency condition: Obj_Std and Obj_Ver are consistent within the error ε.

[0187] Newly synthesized data pairs that meet the above conditions are considered to have passed the most stringent quality control. This means that the new problem we synthesize can be understood relatively stably and correctly by an independent AI system, and this understanding is mathematically consistent with our pre-defined standard answer. Such data pairs are considered high-quality and are retained in the final dataset. If any condition is not met, the data pair is deemed to have failed the two-way validation and is discarded.

[0188] S60. The new synthetic data pairs, which have passed the executability verification and bidirectional consistency verification mechanisms, are aggregated to construct a training dataset for training the target large language model.

[0189] This step is the final output of the method flow of this invention. Its core task is to systematically collect, post-process, and format all synthetic data pairs that have passed the aforementioned rigorous quality verification, thereby constructing a high-quality error-driven training dataset that can be directly used to train or fine-tune large-scale mathematical optimization modeling models. This dataset is not a simple data accumulation, but a carefully designed machine learning asset with clear teaching objectives and an inherent structure.

[0190] In summary, existing technologies (whether forward or backward synthesis) are essentially data-driven, aiming to expand the quantity or diversity of data, but they fail to establish a direct connection with the intrinsic needs of model learning. The information-rich signals—where and why the model makes mistakes—are completely ignored. This invention achieves a fundamental paradigm shift: treating errors generated by large language models in specific tasks (optimization modeling) as valuable signals and roadmaps guiding data synthesis. Its core logic is: if a model makes a mistake at point A, it indicates a deficiency in its understanding of patterns related to A; therefore, more data specifically targeting pattern A should be generated to reinforce learning. This approach shifts the goal of data synthesis from covering a broader data space to precisely targeting the model's weaknesses, resulting in a qualitative leap in the educational value of the final synthesized data pairs. This error-driven approach is significantly original in the field of automated training data generation.

[0191] The structured intermediate representation (IR) comparison method proposed in this invention parses the model into standard components such as variable sets, constraint sets, and objective functions, and performs multi-dimensional, refined comparisons (e.g., missing variables / type errors, omitted constraints / direction errors), transforming ambiguous errors into structured difference features. Furthermore, through inducement text fragment extraction technology, the identified error patterns are associated with specific text fragments in the original problem description, providing precise targets for subsequent reverse evolution. This automated error diagnosis and attribution mechanism is the prerequisite and cornerstone for error-driven implementation. This invention innovatively designs a semantic evolution operator library (such as fuzzification, ambiguity, and information reconstruction), which is an algorithmic summary of human experts' experience in creating misleading descriptions. By applying these operators to inducement text fragments, the system can automatically synthesize new problem descriptions containing specific semantic traps. More innovative is the synchronous answer generation mechanism: while the problem evolves, the system intelligently chooses to replicate the original model or derive a new model through rules / auxiliary models, depending on whether the evolution changes the mathematical essence. This ensures that each problem has an indisputable standard answer, solving the two core challenges of how to create problems and how to guarantee the correctness of answers in targeted data synthesis. This invention also designs two progressive, automated firewalls. The first is solver executability verification, which filters out bad answers that are mathematically contradictory, unsolvable, or technically flawed. More importantly, it performs bidirectional consistency verification: it assigns the synthesized new problem to an independent verification model for interpretation, requiring that the model generated by the verification model not only be structurally highly similar to the standard answer (forward verification) but also that its optimal objective value is consistent (reverse verification). This mechanism creatively simulates the idea of ​​multiple grading and result reverse deduction, effectively filtering out bad problems with seriously ambiguous expressions that lead to inconsistent interpretations by different models, ensuring high standards of understandability and objective consistency in the synthesized data. This verification system is the key guarantee for the high-quality, highly reliable dataset produced by this invention.

[0192] Compared to existing methods mentioned in the background (forward synthesis relies on seeds, and reverse synthesis lacks specificity), the technological advancements brought by this invention are significant: it reduces the reliance on large amounts of high-quality seed data and expert annotations; it improves the matching degree between synthesized data and the actual needs of the model (specificity); it ensures the mathematical correctness and logical consistency of the synthesized data (high quality); and ultimately, it achieves the core objective of training a more accurate and robust optimization model with higher sample efficiency.

[0193] The error-driven training dataset synthesis method in the embodiments of the present invention has been described above. The error-driven training dataset synthesis apparatus in the embodiments of the present invention is described below. Please refer to [link / reference]. Figure 2One embodiment of the error-driven training dataset synthesis device of the present invention includes:

[0194] The acquisition module 10 is used to acquire seed data pairs containing the original natural language problem and its corresponding original standard mathematical optimization model;

[0195] Modeling module 20 is used to automatically perform mathematical modeling on the original natural language problem using the target large language model to obtain a generative model;

[0196] The comparison module 30 is used to compare the generated model with the original standard mathematical optimization model, identify and extract the structural error patterns generated by the target large language model during the modeling process;

[0197] Evolution module 40 is used to semantically evolve the description of the original natural language problem based on the structural error pattern, synthesize a new natural language problem, and at the same time generate an updated standard mathematical optimization model corresponding to the new natural language problem to form a new synthetic data pair;

[0198] The verification module 50 is used to verify the solver executability of the updated standard mathematical optimization model in the new synthetic data pair, and to use a two-way consistency verification mechanism to verify the logical consistency from the new natural language problem to the updated standard mathematical optimization model and then back from the updated standard mathematical optimization model.

[0199] The aggregation module 60 is used to aggregate new synthetic data pairs that have passed the executability verification and bidirectional consistency verification mechanisms to construct a training dataset for training the target large language model.

[0200] Figure 2 The error-driven training dataset synthesis device in this embodiment of the invention will be described in detail from the perspective of modular functional entities. The error-driven training dataset synthesis device in this embodiment of the invention will be described in detail from the perspective of hardware processing.

[0201] Figure 3This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. The electronic device 100 can vary significantly due to differences in configuration or performance. It may include one or more central processing units (CPUs) 111 (e.g., one or more processors) and a memory 121, and one or more storage media 130 (e.g., one or more mass storage devices) for storing application programs 133 or data 132. The memory 121 and storage media 130 may be temporary or persistent storage. The program stored in the storage media 130 may include one or more modules (not shown in the diagram), each module including a series of instruction operations on the electronic device 100. Furthermore, the processor 111 may be configured to communicate with the storage media 130 and execute the series of instruction operations in the storage media 130 on the electronic device 100.

[0202] Electronic device 100 may also include one or more power supplies 141, one or more wired or wireless network interfaces 151, one or more input / output interfaces 161, and / or one or more operating systems 131, such as Windows Server, Mac OS X, Unix, Linux, FreeBSD, etc. Those skilled in the art will understand that... Figure 3 The device structure shown does not constitute a limitation on the electronic device 100, and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0203] The present invention also provides a computer-readable storage medium, which can be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium, wherein the computer-readable storage medium stores instructions that, when executed on a computer, cause the computer to perform the steps of an error-driven training dataset synthesis method.

[0204] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the system, device, or unit described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0205] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0206] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for synthesizing error-driven training datasets, characterized in that, Including the following steps: Obtain seed data pairs containing the original natural language problem and its corresponding original standard mathematical optimization model; The original natural language problem is automatically mathematically modeled using a target large language model to obtain a generative model; The generative model and the original standard mathematical optimization model are analyzed and converted into structured intermediate representations. These intermediate representations include at least a set of decision variables, an objective function expression, and a set of constraints. Based on these intermediate representations, the generative model and the original standard mathematical optimization model are compared item by item across the three dimensions of variables, constraints, and objective, and difference features are calculated. According to preset error classification rules, these difference features are mapped to specific structural error patterns, and the triggering text fragments in the original natural language questions that trigger these structural error patterns are recorded. The structural error patterns, the original natural language questions, and the triggering text fragments constitute an error pattern library. One or more structural error patterns are selected from the error pattern library, and their corresponding triggering text fragments and original natural language questions are obtained. Based on the type and complexity of the selected structural error patterns, one or more semantic evolution operators are selected from a preset semantic evolution strategy library. Using the complete descriptions of the triggering text fragments and the original natural language questions as input, the selected semantic evolution operators are applied to generate one or more candidate new natural language questions. The new natural language questions are analyzed, and it is determined whether their mathematical structure has undergone substantial changes compared to the original natural language questions. If the new natural language questions have not undergone substantial changes in mathematical structure, the standard mathematical optimization model is directly copied or slightly adjusted as an updated standard mathematical optimization model. If the new natural language questions have undergone substantial changes in mathematical structure, an updated standard mathematical optimization model is derived based on the evolved semantics through preset rules or an auxiliary large language model. The new natural language questions and the corresponding updated standard mathematical optimization models form a new synthetic data pair. The solver executability of the updated standard mathematical optimization model in the new synthetic data pair is verified, and a two-way consistency verification mechanism is adopted to verify the logical consistency from the new natural language problem to the updated standard mathematical optimization model and then back from the updated standard mathematical optimization model. New synthetic data pairs, validated through executability and bidirectional consistency mechanisms, will be aggregated to construct a training dataset for training the target large language model.

2. The error-driven training dataset synthesis method according to claim 1, characterized in that, The semantic evolution operators include: fuzzification operator, ambiguous operator, information reconstruction operator, redundant information injection operator, negation and double negation operators, and term substitution operator.

3. The error-driven training dataset synthesis method according to claim 1, characterized in that, If the new natural language problem undergoes substantial changes in its mathematical structure, then based on the evolved semantics, an updated standard mathematical optimization model is derived using pre-defined rules or an auxiliary large language model, including the following steps: If the evolution of the new natural language problem introduces new constraints, changes the variable types, or adjusts the target coefficients, then the new natural language problem is judged to have undergone substantial changes in its mathematical structure. Based on a pre-defined natural language-mathematical model mapping rule base, the original standard mathematical optimization model is parsed to generate an updated standard mathematical optimization model, or a trained and highly reliable auxiliary large language model is called. With a new natural language problem as input, the generated candidate standard model is pre-verified by the solver to ensure its correctness before being used as the updated standard mathematical optimization model.

4. The error-driven training dataset synthesis method according to claim 1, characterized in that, The solver executability verification of the updated standard mathematical optimization model in the new synthetic data pair includes the following steps: The updated standard mathematical optimization model in the synthetic data pair is solved using a mathematical optimization solver to verify whether the model has a feasible solution and whether the solution process is error-free. If the model is infeasible, unbounded, or the solution reports an error, the synthesized data pair is deemed invalid.

5. The error-driven training dataset synthesis method according to claim 4, characterized in that, A bidirectional consistency verification mechanism is employed to verify the logical consistency from the new natural language problem to the updated standard mathematical optimization model, and then back to the solution from the updated standard mathematical optimization model. This includes the following steps: The new natural language problem in the new synthetic data pair is input into a large language model for validation to obtain the generated validation model, and the similarity between the validation model and the updated standard mathematical optimization model in the new synthetic data pair is calculated. The optimal objective function values ​​of the verification model and the updated standard mathematical optimization model under the same instance parameters are calculated using the solver, and the two optimal objective function values ​​are compared to see if they are consistent within a preset error range. If the similarity is higher than a preset threshold and the two optimal objective function values ​​are consistent, then the newly synthesized data pair is determined to have passed the bidirectional consistency verification.

6. A training dataset synthesis device based on error-driven methods, characterized in that, include: The acquisition module is used to acquire seed data pairs containing the original natural language problem and its corresponding original standard mathematical optimization model; The modeling module is used to automatically perform mathematical modeling on the original natural language problem using the target large language model to obtain a generative model; The comparison module is used to parse the generative model and the original standard mathematical optimization model respectively, and convert them into structured intermediate representations. The intermediate representations include at least: a set of decision variables, an objective function expression, and a set of constraints. Based on the intermediate representations, the generative model and the original standard mathematical optimization model are compared item by item in three dimensions: variables, constraints, and objective, and the difference features are calculated. According to the preset error classification rules, the difference features are mapped to specific structural error patterns, and the triggering text fragments in the original natural language questions that trigger the structural error patterns are recorded. The structural error patterns, the original natural language questions, and the triggering text fragments constitute an error pattern library. The evolution module is used to select one or more structural error patterns from the error pattern library and obtain their corresponding triggering text fragments and original natural language questions; based on the type and complexity of the selected structural error patterns, it selects one or more semantic evolution operators from a preset semantic evolution strategy library; taking the triggering text fragments and the complete description of the original natural language question as input, it applies the selected semantic evolution operators to generate one or more candidate new natural language questions; it analyzes the new natural language questions and determines whether their mathematical structure has undergone substantial changes compared to the original natural language question; if the new natural language question has not undergone substantial changes in mathematical structure, it directly copies or fine-tunes the standard mathematical optimization model as an updated standard mathematical optimization model; if the new natural language question has undergone substantial changes in mathematical structure, it derives an updated standard mathematical optimization model based on the evolved semantics through preset rules or an auxiliary large language model; and it forms a new synthetic data pair with the new natural language question and the corresponding updated standard mathematical optimization model. The verification module is used to verify the solver executability of the updated standard mathematical optimization model in the new synthetic data pair, and to use a two-way consistency verification mechanism to verify the logical consistency from the new natural language problem to the updated standard mathematical optimization model and then back from the updated standard mathematical optimization model. The aggregation module is used to aggregate new synthetic data pairs that have passed executability verification and bidirectional consistency verification mechanisms to construct a training dataset for training the target large language model.

7. An electronic device, characterized in that, It includes a memory and at least one processor, wherein the memory stores computer-readable instructions; The at least one processor invokes the computer-readable instructions in the memory to perform the steps of the error-driven training dataset synthesis method as described in any one of claims 1-5.

8. A computer-readable storage medium storing computer-readable instructions thereon, characterized in that, When the computer-readable instructions are executed by a processor, they implement the steps of the error-driven training dataset synthesis method as described in any one of claims 1-5.

Images