An automatic modeling and solving method and system for constraint optimization problems based on LLM
By adopting an LLM-based automated modeling and solving method, the entire process from natural language problem description to executable code solution is automated. This solves the shortcomings of existing solvers in modeling and algorithm adaptability, improves solving efficiency and flexibility, and is suitable for optimization problems in scientific research, education, and industry.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- BEIJING UNIV OF POSTS & TELECOMM
- Filing Date
- 2026-04-24
- Publication Date
- 2026-07-31
AI Technical Summary
Existing solvers face challenges such as high modeling thresholds, poor algorithm adaptability, and insufficient system flexibility when dealing with complex industrial scenarios. They struggle to directly understand the natural language descriptions of business personnel, have poor adaptability to non-smooth functions and dynamic constraints, and are difficult to flexibly adapt and expand their functions according to the characteristics of different industries.
An automatic modeling and solving method based on LLM is adopted. Through natural language processing and code generation capabilities, it automatically parses the natural language question text input by the user, generates a mathematical model and solves it. It uses a global knowledge base and algorithm library to automatically solve the problem and output the solution results.
It significantly lowers the technical threshold for mathematical modeling, improves solution efficiency, stability and scalability, can adapt to complex problems and provide flexible solution schemes, and is suitable for scientific research, education and industry.
Smart Images
Figure CN122489698A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the fields of artificial intelligence and operations research, and in particular to an automatic modeling and solving method and system for constrained optimization problems based on LLM. Background Technology
[0002] Constrained optimization problems are mathematical problems that seek the best decision under specific constraints. They require maximizing or minimizing a specific objective function while ensuring that several variables satisfy a series of equality or inequality rules.
[0003] Constrained optimization problems are widespread in engineering and scientific computing, such as network topology optimization, resource allocation and planning, power system dispatching, and hyperparameter tuning in machine learning. To solve these complex mathematical problems, specialized software tools—solvers—have been developed. The core function of a solver is to automatically find the optimal or feasible solution to a constrained optimization problem. Users simply need to clearly express the mathematical model of the problem using a modeling language or application programming interface (API), and the solver will invoke its internally integrated advanced algorithms. After a series of steps including preprocessing and iterative computation, it will finally output a high-quality solution. Based on their problem-solving capabilities, solvers are categorized into linear programming solvers, mixed-integer programming solvers, and nonlinear programming solvers, among others.
[0004] Existing solvers face three major challenges when dealing with complex industrial scenarios: First, at the modeling level, they heavily rely on the mathematical modeling capabilities of experts and cannot directly understand the natural language descriptions of business personnel, becoming a barrier to technology application. Second, at the algorithm level, traditional algorithms rely on precise mathematical models and derivative information, making them poorly adaptable to problems such as non-smooth functions and dynamic constraints, often getting stuck in local optima or consuming excessive computation time. Although existing technologies have introduced metaheuristic algorithms (such as genetic algorithms and sparrow search algorithms) to enhance global search capabilities, they still suffer from premature convergence or weak local search capabilities, and are mostly limited to local improvements within the algorithm itself. Third, at the system architecture level, existing solvers are mostly single, rigid, and monolithic designs, making it difficult to flexibly adapt and expand their functionality according to the specific knowledge and workflows of different industries such as finance, energy, and logistics.
[0005] Therefore, there is an urgent need to provide an automatic modeling and solving method and system for constrained optimization problems based on LLM to solve the above-mentioned technical problems. Summary of the Invention
[0006] To address the problems existing in the prior art, this invention provides an automatic modeling and solving method and system for constrained optimization problems based on LLM. By utilizing the natural language processing and code generation capabilities of LLM, it achieves automatic parsing of natural language, mathematical modeling, code generation, and automated solving, ultimately significantly reducing the modeling threshold and improving the efficiency, stability, robustness, and scalability of solving complex problems.
[0007] The technical solution of this invention is implemented as follows: An automated modeling and solution method for constrained optimization problems based on LLM includes the following steps: S1. Obtain the user-input question text containing natural language; parse and process the question text to generate structured question features; the question features include the solution objective, constraints, and solution process settings; S2. Using a preset global knowledge base, the problem features are retrieved and matched to obtain mathematical model templates, standard form cases and domain prior knowledge, and a temporary knowledge base is constructed. Based on the temporary knowledge base, comprehensive reasoning is performed to classify and model the problem text mathematically, generating a mathematical model framework. Each entry in the global knowledge base is pre-converted into a high-dimensional semantic vector using an embedding model and stored in a vector index structure. For key semantic information in the problem features, the same embedding model is used to convert it into a query vector. The similarity between the query vector and the semantic vector in the global knowledge base is calculated, and several entries with similarity higher than a preset threshold or with the highest ranking are retrieved, thus forming a temporary knowledge base.
[0008] S3. Match the mathematical model framework with the preset algorithm library to generate an executable code file; S4. Call the algorithm interface provided by the algorithm library, use the code interpreter to execute the code file, perform numerical calculations and solve the problem, and obtain the solution result that meets the solution objective; S5. Analyze the solution results and summarize the entire process from the problem text to the solution results, and output the summary results.
[0009] As a further optimization of the above scheme, in S1, the information input by the user includes plain text and / or files; The plain text may be part or all of the question text; The file includes one or any combination of document, table, or image files; Use a file parsing tool to extract part or all of the problem text from the file.
[0010] When an uploaded file is detected, the corresponding tool is dispatched based on the file type to process it, read the relevant data, and clean and transform it. Specifically, if the document's file type is a Word-related file format, such as docx, a Word parser is constructed using Python to parse the document's text content and paragraph structure. In addition to extracting the text content, elements in the document, including mathematical formulas such as objective functions and constraint equations, are structurally broken down, type labels are added to the formula content, and text blocks are associated with corresponding mathematical elements.
[0011] Similarly, the table corresponds to the file format associated with Excel. An Excel parser is constructed to parse and extract the table data, column structure, and numerical content. After reading the table, the data type (decision variable matrix, constraint coefficient matrix, objective function coefficient vector) is first identified, then basic statistical characteristics such as mean, variance, and extreme values are calculated, and preliminary characterization is performed based on data sparsity. A compressed storage format is used for sparse matrices, while the original dimensional information is preserved for dense matrices. Finally, the table header, matrix / vector type, dimension parameters, statistical characteristics, and storage format are saved.
[0012] In practice, you can use parsing libraries provided by Python, such as "python-docx" and "openpyxl" which are respectively used to parse docx and xlsx formats.
[0013] For image files, an OCR processor is used to extract textual elements from the image, and a visual understanding agent is used to extract non-textual elements. The visual understanding agent relies on a multimodal large language model to recognize formula structures, chart types (including line charts, bar charts, and network diagrams) and modeling schematic logic in the image.
[0014] As a further optimization of the above solution, the problem features are as follows: The solution objective includes a preliminary extracted mathematical expression; the mathematical expression includes one or any combination of fundamental functions, trigonometric functions, logarithmic functions, exponential functions, norms, differentials, integrals, and indicator functions; The constraints are restrictive conditions related to the solution objective, including equality constraints, inequality constraints, and variable boundaries; The solution process is set by the user to specific settings, including parameter configuration, number of iterations, convergence accuracy, and output format; In step S4, the code interpreter executes the code file according to the solution process settings.
[0015] As a further optimization of the above scheme, in S2, the problem is classified into mathematical problem types based on the problem to be solved; the mathematical problem types include linear programming, nonlinear programming, integer programming, convex optimization, and non-convex optimization. The mathematical model framework includes a standardized objective function, a set of constraints, and a variable space. The objective function is obtained by transforming the mathematical expression. The constraint set is obtained based on the constraints; the constraint set is one or any combination of a linear equality constraint set, a linear inequality constraint set, a nonlinear equality constraint set, and a nonlinear inequality constraint set. The variable space includes the dimension, boundary, and variable type of the variables; the variable type includes continuous and discrete.
[0016] Furthermore, consistency verification is performed on the mathematical model framework, such as checking whether the variables are correctly defined in the constraints.
[0017] As a further optimization of the above solution, in step S3, a target algorithm is matched in the algorithm library according to the type of mathematical problem; and the code file is generated according to the target algorithm and the mathematical model framework. The code file includes a problem code and any number of data preprocessing codes; any number includes zero; the data preprocessing codes are used to clean, transform, and perform feature engineering on the data provided in the problem text; the problem code is used to solve the objective function.
[0018] Data cleaning removes invalid or dirty data, including handling missing values, outliers, and duplicates. Data conversion involves processing data in a uniform format, classifying data of the same type, normalizing, and binning. Binning divides continuous values into discrete intervals, such as dividing [0,100] into multiple age groups like [0,18], [19,60], and [60,100].
[0019] Feature engineering involves extracting or constructing new features from existing data that are more informative and meaningful for the prediction target (label). For example, in the e-commerce field, a new feature called "average order value" can be created from "purchase amount" and "purchase frequency".
[0020] As a further optimization of the above scheme, in step S4, the code interpreter first executes the data preprocessing code, and then executes the problem code.
[0021] As a further optimization of the above solution, the format of the code file is associated with C++; the generation of the code file includes: The information contained in the mathematical model framework is identified and preprocessed using preset keywords; the preprocessing includes removing whitespace characters and segmenting the mathematical expressions; preset keywords include "OBJECTIVE:", "EQUAL_CONSTRAINTS:", etc.
[0022] The pre-processed mathematical expression is tokenized by a lexical analyzer, and an abstract syntax tree is constructed by a parser based on operator precedence and associativity to structurally express the objective function and the constraints. The abstract syntax tree is rendered as a C++ code string, and a header file is automatically generated. Specifically, during the rendering process, variables are mapped to array access format, mathematical functions are converted to standard library functions, and exponentiation is implemented using `std::pow`. In the header file, the `GeneratedProblem` class, which inherits from the optimization problem base class, implements key methods such as objective function calculation, numerical gradient calculation, and constraint evaluation, allowing the output code to be directly compiled and integrated into the main solver.
[0023] As a further optimization of the above scheme, the algorithms in the algorithm library include sequential quadratic programming algorithm, augmented Lagrange method, gradient projection method and exterior point penalty function method.
[0024] Sequential Quadratic Programming (SQP) is an algorithm used to handle nonlinear constrained optimization problems. SQP converges to the optimal solution of the original nonlinear problem by iteratively solving a series of quadratic programming subproblems through local quadratic approximation of the original problem.
[0025] The Augmented Lagrangian Method (ALM) is used to handle nonlinear optimization problems with equality and inequality constraints. The ALM algorithm transforms the original constrained problem into a series of unconstrained subproblems by introducing slack variables and penalty functions, gradually approximating the optimal solution.
[0026] The Gradient Projection Method (GPM) is used to handle nonlinear optimization problems with box constraints (i.e., upper and lower bound constraints on variables). The GPM algorithm ensures that the constraints are always satisfied during the iterative process by projecting the unconstrained gradient descent direction into the feasible region, thereby achieving a stable solution to the constrained problem.
[0027] The exterior penalty method is used to handle nonlinear optimization problems with equality and inequality constraints (atypical constraints). This method transforms the original constrained problem into a series of unconstrained subproblems by introducing the constraint violation into the objective function in the form of a penalty function, and forces the iteration points to converge toward the feasible region by gradually increasing the penalty factor.
[0028] As a further optimization of the above solution, in step S5, the problem text, problem features, mathematical model framework, code file, and solution results are obtained, summarized, and the summary results are generated; wherein, the solution results are subjected to structured parsing and semantic extraction to generate readable natural language text. The summary results are output directly in natural language or in the form of a report; wherein, the code interpreter is invoked to generate the report using a script; the report includes charts and text content.
[0029] The normal output summary includes, but is not limited to: the optimal solution and the corresponding value of the objective function, the number of iterations, the algorithm convergence curve, and the AI analysis results. Furthermore, if the solution process for S4 terminates abnormally, such as failing to converge after reaching the maximum number of iterations or inconsistencies between different data files, the exception type and possible solutions will also be output.
[0030] This invention also provides an automatic modeling and solving system for LLM-based constrained optimization problems, which applies the above-mentioned automatic modeling and solving method for LLM-based constrained optimization problems, and includes the following modules: The problem decomposition module is used to parse and process the problem text to generate the problem features; A mathematical modeling module is used to construct the mathematical model framework based on the characteristics of the problem. The intelligent solver module is used to generate and execute the code file according to the mathematical model framework to obtain the solution result. The intelligent solver module has a built-in dedicated conversion module whose core function is to automatically process the mathematical expression of the optimization problem output by the large model and convert it into a compilable C++ header file, so as to seamlessly integrate it into the main solver. This realizes end-to-end automation from problem description to solution code, which significantly improves the solver's ease of use, accessibility, automation level and efficiency.
[0031] The dedicated transformation module supports a wide range of mathematical expressions, including nonlinear functions, multivariables, and complex operators, and handles gradient calculations numerically, making it suitable for non-smooth or black-box functions and ensuring numerical stability.
[0032] By directly converting natural language or mathematical formulas into executable code, the need for manual coding is eliminated, human error is reduced, and the process of solving optimization problems is effectively accelerated.
[0033] Furthermore, the dedicated conversion module is highly compatible with AI-driven optimization scenarios, capable of parsing optimization problems described in natural language, and is suitable for education, scientific research, and industrial fields, accelerating the development of customized solvers.
[0034] The results summary module is used to summarize and output the summary results.
[0035] Furthermore, it includes a file processing module for parsing user-uploaded attachments, such as text, tables, PDFs, and images, extracting data and semantic information relevant to the optimization problem, and converting them into structured data. This module can call an OCR processor to recognize text and formulas in images, parse table content, align the extracted results with the problem context, and output them to the problem decomposition module or directly integrate them into the "problem features".
[0036] Furthermore, it also includes an optimized feature recognition module, which automatically analyzes the features of structured problems to identify their inherent mathematical and computational characteristics and generate key attribute labels to guide the selection of solution strategies.
[0037] Based on the objective function type, constraints, and problem background requirements provided by the mathematical modeling module, the optimization feature recognition module provides corresponding solution methods and suggestions such as GPM, ADMM, SQP, and ALM, along with causal analysis. The intelligent solution module then extracts algorithms from the algorithm library based on the optimization feature recognition module.
[0038] Compared with the prior art, the present invention achieves the following beneficial effects: (1) This invention provides a fully automated mathematical modeling problem-solving method that goes from natural language problem description to executable code solution and output summary. Based on a large language model, users only need to input the problem in natural language, and the system can automatically complete the problem feature analysis, mathematical model construction, code generation, numerical calculation and solution, and summarize and output the entire process. Users do not need to have professional mathematical modeling knowledge or programming skills, which significantly reduces the technical threshold for solving mathematical problems and improves the automation and efficiency from problem understanding to result acquisition.
[0039] (2) The Retrieval Enhancement Generation (RAG) framework is deeply integrated with the professional optimization knowledge base to provide intelligent recommendations for problem decomposition, algorithm selection and parameter configuration. This effectively overcomes the limitations of large language models in deep mathematics and operations research expertise, ensures the scientificity and reliability of the solution, and avoids solution failures caused by model misuse or improper algorithms.
[0040] (3) The present invention adopts a "description and execution separation" architecture, which divides the intelligent solution process into two stages: first, the mathematical model is converted into an executable code file by combining the matching algorithm, and then the code is executed by the code interpreter to complete the solution. This design not only ensures the flexibility of problem description, but also makes full use of the computing power of existing high-performance solvers, thereby achieving a balance between the convenience of mathematical modeling and the efficiency of numerical solution.
[0041] (4) This invention is applicable to a wide range of optimization problems in scientific research, education and industry. Its optimization objectives can cover linear programming, nonlinear programming and more complex mathematical programming problems. The system can not only output the optimal solution and solution status, but also diagnose and provide feedback on solution anomalies, guiding users to adjust the model or parameters. It has significant practical application value and promotion prospects. Attached Figure Description
[0042] Figure 1 This is a flowchart illustrating an automatic modeling and solving method for constrained optimization problems based on LLM, provided in an embodiment of the present invention. Figure 2 This is a schematic diagram of the structured file internal structure of the problem features provided in the embodiments of the present invention; Figure 3 This is a schematic diagram of the processing flow within the intelligent solver module provided in this embodiment of the invention; Figure 4 This is a schematic diagram of the processing flow within the result summarization module provided in this embodiment of the invention. Detailed Implementation
[0043] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions in the embodiments of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this invention, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.
[0044] like Figures 1 to 4 As shown in the figure, this embodiment provides an automatic modeling and solving system for constrained optimization problems based on LLM, including a problem decomposition module, a file processing module, a mathematical modeling module, an optimization feature recognition module, an intelligent solving module, and a result summarization module.
[0045] The modules in the system of the present invention interact with a rag system, which contains a global knowledge base related to solving and mathematical modeling.
[0046] Regarding the construction of the global knowledge base, a text segmentation strategy adapted to mathematical modeling scenarios is adopted. Uploaded PDF, DOC, and other documents are structurally decomposed according to mathematical modeling requirements. Text blocks are divided according to mathematical model type (linear programming, integer programming, multi-objective optimization, stochastic optimization, etc.), problem-solving paradigm, constraint hierarchy, and objective function structure. The text blocks are then labeled with their types to ensure that the segmented text blocks fully retain the closed-loop logic of "problem abstraction - model construction - algorithm solution - result verification," rather than simply segmenting based on the number of tokens. Subsequently, using a dedicated mathematical modeling embedding model, the text blocks are encoded into high-dimensional feature vectors that integrate "objective function expression features, constraint mathematical representations, algorithm iteration logic, and cross-domain modeling transfer experience," and stored in a vector database, laying the foundation for subsequent accurate retrieval.
[0047] Regarding knowledge sources, the global knowledge base includes core monographs on optimization theory, covering subfields such as convex optimization, combinatorial optimization, and robust optimization. It focuses on top-tier interdisciplinary journal papers with impact factors exceeding 100 from the past five years in areas such as multi-objective modeling in bioinformatics, constraint optimization of engineering systems, numerical modeling of physical processes, optimal scheduling modeling of power systems, and path optimization of transportation networks, as well as collections of outstanding papers from mathematical modeling competitions. The base extracts modeling ideas, algorithm selection logic, and problem adaptation experience from these papers. At the same time, it integrates classic mathematical modeling textbooks from China and abroad, emphasizing the standardization of modeling processes, constraint decomposition methods, and objective function construction techniques. It constructs an interdisciplinary optimization problem case library and a specialized mathematical modeling question bank to achieve the classification of knowledge base content and the cross-referencing function of text blocks within each knowledge base.
[0048] The Rag system calculates the relevance between the input text and the knowledge base text using embedding and rerank models. The embedding model obtains the vector value of the input text, which the Rag system compares with the vector values of the knowledge base text blocks, returning the higher-ranked blocks. The rerank model directly prioritizes text blocks based on the input question. Therefore, the Rag system can recall corresponding text blocks according to the agent's requirements.
[0049] The document processing module focuses on knowledge extraction and mathematical representation in mathematical modeling scenarios. The core components and processing logic are as follows: First, the document processing agent confirms the processing plan based on the built-in "mathematical information extraction prompts". The plan needs to clarify the priority and representation method of mathematical elements in the document (such as formulas, tabular data, and modeling constraints). Then, it calls three types of mathematical processing tools as needed: Word parser, Excel processor, visual understanding agent and OCR tool.
[0050] This system employs an automated modeling and solution method for constrained optimization problems based on LLM, including the following steps: S1. Obtain the user-inputted question text containing natural language; specifically, the user input includes plain text and files; the plain text is part of the question text; the files include Excel spreadsheets containing the numerical data required to solve the problem. The numerical data is also part of the question text.
[0051] In this embodiment, the file processing module uses a file parsing tool, namely an Excel parser, to extract numerical data from an Excel spreadsheet, aligns the extraction results with the problem context, and outputs them to the problem decomposition module or directly integrates them into the "problem features".
[0052] Users submit questions to the system by inputting natural language or files (doc documents, Excel spreadsheets, images). If the user only inputs natural language, the question decomposition module processes it directly. If the user uploads files such as spreadsheets, the file and question are first sent to the file processing module. The question decomposition module then asynchronously waits for the file processing module's result, which is stored as a temporary file. If the file is small, it is sent directly to the question decomposition module in text format. The question decomposition module combines the processed file with the user's natural language prompts, and provides analysis results in a formatted and standardized manner, including question classification, suggested methods, question restatement, and formula decomposition, before sending it to the mathematical modeling module for further processing.
[0053] In addition, after the problem decomposition module processes the entire problem and generates the corresponding analysis results, the analysis results need to be confirmed by the user. If the user approves, the analysis results are sent to the mathematical modeling module; if the user does not approve, the module will seek further improvement directions and supplementary information from the user.
[0054] Finally, the problem features will be sent to the mathematical modeling module in the form of a structured file (such as output.json). The output.json file is designed with key-value pairs containing the filename and its content, facilitating recognition and reading by other agents. This results in a mathematical intermediate result containing structured mathematical matrices, formula category labels, variable dimension parameters, etc., providing accurate input for the subsequent modeling element matching and logical decomposition in the problem decomposition module.
[0055] The problem decomposition module parses and processes the problem text to generate structured problem features. Specifically, the problem decomposition module has a problem decomposition agent that can decompose the problem text into several prompt words according to the built-in prompt word function, and then call tools including the rag system.
[0056] Problem characteristics include the solution objective, constraints, and solution process settings.
[0057] In this embodiment, the objective to be solved includes the initially extracted mathematical expression.
[0058] Mathematical expressions include one or any combination of fundamental functions, trigonometric functions, logarithmic functions, exponential functions, norms, differentials, integrals, and indicator functions.
[0059] Constraints are restrictive conditions related to the solution objective, including equality constraints, inequality constraints, and variable boundaries.
[0060] The solution process is set by the user to specific settings, including parameter configuration, number of iterations, convergence accuracy, output format, etc. S2: Using a preset global knowledge base, the problem features are retrieved and matched to obtain mathematical model templates, standard form cases, and domain prior knowledge, and a temporary knowledge base is constructed.
[0061] The mathematical modeling module performs comprehensive reasoning based on a temporary knowledge base, classifies the problem text mathematically, constructs models, and generates a mathematical model framework.
[0062] In this embodiment, each entry in the global knowledge base is pre-converted into a high-dimensional semantic vector using an embedding model and stored in a vector index structure. For key semantic information in the problem features, the same embedding model is used to convert it into a query vector. The similarity between the query vector and the semantic vector in the global knowledge base is calculated, and several entries with similarity higher than a preset threshold or ranked higher are retrieved, thus forming a temporary knowledge base.
[0063] In this embodiment, the problem is classified into mathematical problem types based on the problem to be solved; the mathematical problem types include linear programming, nonlinear programming, integer programming, convex optimization, and nonconvex optimization; The mathematical model framework includes a standardized objective function, a constraint set, and a variable space. The objective function is obtained by transforming the mathematical expression; the constraint set is obtained by constraining the constraints; the constraint set can be one or any combination of linear equality constraint set, linear inequality constraint set, nonlinear equality constraint set, and nonlinear inequality constraint set; the variable space includes the dimension, boundary, and variable type of the variables; the variable types include continuous and discrete.
[0064] The mathematical modeling module includes a mathematical modeling agent. With the assistance of built-in prompts and a temporary knowledge base, the agent first categorizes problems according to the following rules: It first determines whether the user input is a valid mathematical optimization problem, requiring a clearly defined objective function and constraints. Valid problems are then categorized based on their background: first, they are classified as traditional problems and problems in the field of artificial intelligence. For problems in the field of artificial intelligence, they are further categorized according to the task objective, such as pagination, clustering, neural networks, and generative adversarial networks. For traditional problems, they can be further divided into integer programming problems and continuous optimization problems based on variable type and constraint structure. Integer programming problems include linear integer programming and nonlinear integer programming, suitable for scenarios where variables need to take integer values, such as combinatorial optimization, scheduling, and location selection. Continuous optimization problems cover unconstrained optimization, bounded optimization, nonlinear programming, nondifferentiable optimization, game theory problems, dynamic programming, quadratic programming, bilevel programming, and probabilistic optimization models, suitable for various engineering, economic, and scientific computing problems where variables are real numbers and the objective function or constraints have different mathematical properties. After classification, the mathematical modeling module performs comprehensive reasoning based on the temporary knowledge base to establish a mathematical model, which includes: (1) the previously identified problem type; (2) the mathematical model framework, including the logical relationship between possible sub-problems, the specific meaning of mathematical symbols, the number of variables, etc.; (3) the standardized objective function, which transforms the initially extracted objective expression into a standard mathematical form; (4) the standardized constraint set, which organizes all identified constraints into a set of linear equality constraints, a set of linear inequality constraints, a set of nonlinear equality constraints, and a set of nonlinear inequality constraints; and (5) the variable space definition, i.e. the dimension, type (continuous, discrete), and boundary of the decision variables. When the problem is more complex, it may involve multiple objective functions and constraints. The mathematical modeling intelligence will decompose the problem into sub-problems that are easier to solve.
[0065] In addition, consistency verification is performed on the mathematical model framework, such as checking whether the variables are correctly defined in the constraints.
[0066] Subsequently, the optimized feature recognition module automatically analyzes the structured problem features to identify their inherent mathematical and computational characteristics and generates key attribute labels to guide the selection of solution strategies.
[0067] S3, the intelligent solution module matches the mathematical model framework with a preset algorithm library. In particular, based on the type of mathematical problem, it matches the target algorithm in the algorithm library to generate an executable code file.
[0068] Specifically, in this embodiment, the algorithms in the algorithm library include sequential quadratic programming algorithm, augmented Lagrange method, gradient projection method and exterior point penalty function method.
[0069] Sequential Quadratic Programming (SQP) is an algorithm used to handle nonlinear constrained optimization problems. SQP converges to the optimal solution of the original nonlinear problem by iteratively solving a series of quadratic programming subproblems through local quadratic approximation of the original problem.
[0070] The Augmented Lagrangian Method (ALM) is used to handle nonlinear optimization problems with equality and inequality constraints. The ALM algorithm transforms the original constrained problem into a series of unconstrained subproblems by introducing slack variables and penalty functions, gradually approximating the optimal solution.
[0071] The Gradient Projection Method (GPM) is used to handle nonlinear optimization problems with box constraints (i.e., upper and lower bound constraints on variables). The GPM algorithm ensures that the constraints are always satisfied during the iterative process by projecting the unconstrained gradient descent direction into the feasible region, thereby achieving a stable solution to the constrained problem.
[0072] The exterior penalty method is used to handle nonlinear optimization problems with equality and inequality constraints (atypical constraints). This method transforms the original constrained problem into a series of unconstrained subproblems by introducing the constraint violation into the objective function in the form of a penalty function, and forces the iteration points to converge toward the feasible region by gradually increasing the penalty factor.
[0073] In this embodiment, the intelligent solution module performs scientific scheduling based on the difficulty of the problem.
[0074] If the problem is relatively simple and the user has not uploaded any files, the intelligent solving module will directly convert the extracted formulas into the necessary solution file, call the existing solver to solve the problem, and obtain the result. If the user has uploaded files, or the problem is more complex, the intelligent solving module will further process the uploaded table file, and then call the appropriate solver through the code interpreter to solve the problem and obtain the result.
[0075] The intelligent solution module includes a tool calling agent, a data processing agent, and an algorithm calling agent.
[0076] The tool invokes an intelligent agent responsible for determining the problem difficulty based on the problem decomposition results: When the problem is relatively easy and the user has not uploaded any spreadsheet files, the tool will call the intelligent system to select and send the formula extracted from the problem decomposition results to the formula conversion program. The formula conversion program will then convert the formula into the formula file needed by the solver, and directly call the corresponding solver to solve the problem and output the solution result.
[0077] In cases of complex problems, inability of a single solver to solve the problem, or when the user has uploaded a table file, the tool's AI invokes the data processing AI. Based on built-in prompts and basic table information in `output.json`, the data processing AI generates a Python script for data processing. This script is then executed by a code interpreter to clean the data, performing necessary tasks such as removing invalid items and merging cells, resulting in a new table file. This process may repeat multiple times, as the AI dynamically adjusts the script based on the interpreter's performance until reasonably cleaned data is obtained. If serious problems are encountered or the number of execution rounds is excessive, the AI throws an exception, handing it over to the user for handling. After data cleaning, the AI receives a brief data processing report (to be revised) from the data processing AI, providing advance knowledge of the cleaned data's table structure. Based on the problem decomposition module's results, the data processing AI's report, and the built-in prompts, the AI writes the algorithm's Python script or C++ header file (i.e., the code file). In existing technologies, the Python script can also call, load, and execute predefined C++ library files.
[0078] In this embodiment, based on the objective function type, constraints, and problem background requirements provided by the mathematical modeling module, the optimization feature recognition module provides corresponding solution methods and suggestions such as GPM, ADMM, SQP, and ALM, along with causal analysis. Therefore, the intelligent solution module extracts algorithms from the algorithm library based on the optimization feature recognition module's input.
[0079] In this embodiment, the code file includes a problem code and any number of data preprocessing codes; any number here includes zero; the data preprocessing codes are used to clean, transform, and feature-engineer the data provided in the problem text; the problem code is used to solve the objective function.
[0080] Data cleaning removes invalid or dirty data, including handling missing values, outliers, and duplicates. Data conversion involves processing data in a uniform format, classifying data of the same type, normalizing, and binning. Binning divides continuous values into discrete intervals, such as dividing [0,100] into multiple age groups like [0,18], [19,60], and [60,100].
[0081] Feature engineering involves extracting or constructing new features from existing data that are more informative and meaningful for the prediction target (label). For example, in e-commerce, a new feature called "average order value" can be created from "purchase amount" and "purchase frequency"; in mechanical fault diagnosis, a new feature called "peak energy ratio" can be constructed from the two original signals of "vibration frequency" and "amplitude" to more sensitively indicate the degree of bearing wear.
[0082] In this embodiment, the intelligent solver module has a built-in dedicated conversion module. Its core function is to automatically process the mathematical expressions of the optimization problem output by the large model and convert them into compilable C++ header files, thereby seamlessly integrating them into the main solver. This achieves end-to-end automation from problem description to solution code, significantly improving the solver's ease of use, accessibility, automation level, and efficiency.
[0083] Specifically, the code file generation process includes: The information contained in the mathematical model framework is identified and pre-processed using preset keywords; the pre-processing includes removing whitespace characters and splitting mathematical expressions; preset keywords include “OBJECTIVE:”, “EQUAL_CONSTRAINTS:”, etc.
[0084] The pre-processed mathematical expression is tokenized by a lexical analyzer, and an abstract syntax tree is constructed by a parser based on operator precedence and associativity to express the objective function and constraints in a structured way. The abstract syntax tree is rendered as a C++ code string, and header files are automatically generated. Specifically, during rendering, variables are mapped to array access, mathematical functions are converted to standard library functions, and exponentiation is implemented using `std::pow`. In the header file, the `GeneratedProblem` class inherits from the optimization problem base class and implements key methods such as objective function calculation, numerical gradient calculation, and constraint evaluation, allowing the output code to be directly compiled and integrated into the main solver.
[0085] The dedicated transformation module supports a wide range of mathematical expressions, including nonlinear functions, multivariables, and complex operators, and handles gradient calculations numerically, making it suitable for non-smooth or black-box functions and ensuring numerical stability.
[0086] By directly converting natural language or mathematical formulas into executable code, the need for manual coding is eliminated, human error is reduced, and the process of solving optimization problems is effectively accelerated.
[0087] Furthermore, the dedicated conversion module is highly compatible with AI-driven optimization scenarios, capable of parsing optimization problems described in natural language, and is suitable for education, scientific research, and industrial fields, accelerating the development of customized solvers.
[0088] S4. The intelligent solution module calls the algorithm interface provided by the algorithm library, uses the code interpreter to execute the code file according to the solution process settings, performs numerical calculations and completes the problem solution, and obtains the solution result that meets the solution objective; among them, the code interpreter executes the data preprocessing code first, and then executes the problem code.
[0089] The code interpreter reads the cleaned table file from the code file, calls the formula conversion program, and invokes existing algorithms to solve the entire problem, ultimately outputting the solution. This process may continue multiple times because the AI dynamically adjusts the script based on the code interpreter's output and errors until the correct result is obtained. If a serious problem is encountered or the number of execution rounds is too high, the AI will throw an exception, which will be handled by the user. Finally, the intelligent solving module sends the processed results to the result aggregation module.
[0090] S5, the Result Summary module parses the solution results and summarizes the entire process from the problem text to the solution result, outputting a summary result. Specifically, it obtains the problem text, problem features, mathematical model framework, code file, and solution results, that is, it summarizes the output results of each module and the user's problem to generate a summary result; among them, it performs structured parsing and semantic extraction on the solution results to generate a rigorous and readable solution result that conforms to the original user problem context.
[0091] Specifically, the result aggregation program within the result aggregation module outputs a temporary file of aggregated results based on the outputs of each module and the user's questions, and then sends this file to the result aggregation agent. This output can be directly sent to the user in natural language; alternatively, a result aggregation script can be constructed, which calls a code interpreter to generate a report (such as a doc file) and output it to the user. This report can include natural language text, graphs, tables, and other content.
[0092] The normal output summary includes, but is not limited to: the optimal solution and the corresponding value of the objective function, the number of iterations, the algorithm convergence curve, and the AI analysis results. Furthermore, if the solution process for S4 terminates abnormally, such as failing to converge after reaching the maximum number of iterations or inconsistencies between different data files, the exception type and possible solutions will also be output.
[0093] Here are some examples of simple questions: (1) A color TV manufacturer plans to launch two new products: a 19-inch LCD flat-screen TV with a suggested retail price of 3390 yuan and a 21-inch LCD flat-screen TV with a retail price of 3990 yuan. The company's costs are 1950 yuan per 19-inch TV and 2250 yuan per 21-inch TV, plus fixed costs of 4,000,000 yuan. According to statistics, for each type of color TV, the average selling price will decrease by 0.1 yuan for every additional TV sold. Moreover, the sales of 19-inch TVs will affect the sales of 21-inch TVs, and vice versa. It is estimated that for every 21-inch TV sold, the average selling price of 19-inch TVs will decrease by 0.03 yuan, and for every 19-inch TV sold, the average selling price of 21-inch TVs will decrease by 0.04 yuan. How many of each type of TV should be produced?
[0094] Based on this problem, the problem decomposition module extracts its objective function. Constraints and The problem is categorized into quadratic objective functions with linear constraints, and the SQP method is selected. The decomposition results are then converted into JSON format and sent to the intelligent solver module. Based on the problem's complexity, the intelligent solver module directly calls the formula converter to generate a temporary file, converting the objective function and constraints into a C++ header file according to a specified format. Finally, it solves the problem using the SQP solver to obtain the desired solution. , Finally, the results summary module reports the solution to the user in natural language: "To maximize profits, the optimal production plan is obtained by solving the quadratic programming problem using the SQP method: 4,735 19-inch LCD flat-panel TVs and 7,043 21-inch LCD flat-panel TVs. This result is based on the cost and selling price linkage rules and the objective function optimization calculation."
[0095] (2) Suppose you only have 10 yuan and want to buy apples and bananas. Apples cost 2 yuan each and bananas cost 1 yuan each. You need a maximum of 6 fruits in total. You want to maximize the total number of fruits and the budget, but not exceed the 10 yuan budget. Give a method.
[0096] Based on this problem, the problem decomposition module extracts its objective function as follows: Constraints include budget constraints. Quantity constraints Non-negativity constraints and integer constraints The problem was then categorized as an integer programming problem with a linear objective function and linear constraints. Alternating Direction Multiplier (ALM) combined with an integer constraint handling strategy was chosen for solving the problem. The decomposition results were then converted to JSON format and sent to the intelligent solution module. The intelligent solution module, based on the ALM solution logic, called a formula converter to generate a C++ header file containing the objective function and constraints. An integer solution rounding and verification mechanism was embedded within the ALM iteration framework: first, an approximate solution was obtained by solving the continuous relaxation problem using ALM; then, the result was corrected for integer values and constraint satisfaction was verified; finally, the solution was obtained. , .
[0097] (3) A reservoir needs to supply water downstream while also generating electricity. The reservoir has a usable water volume of 1 million cubic meters, which needs to be allocated to two sectors: agricultural irrigation and urban water supply. Let the agricultural water consumption be... (10,000 cubic meters), urban water consumption is (10,000 cubic meters). To ensure downstream ecological flow, the total water supply shall not be less than 800,000 cubic meters. Meanwhile, agricultural water consumption shall not exceed 600,000 cubic meters, and urban water consumption shall not exceed 700,000 cubic meters. The goal is to balance the water supply ratio between the two sectors as much as possible, that is, to minimize the absolute value of the difference in water consumption between the two.
[0098] The problem decomposition module extracts the objective function as follows: This is equivalent to introducing an auxiliary variable. , represented as The constraints are , , , , , , This problem involves a linear objective function with linear constraints. The ALM method is chosen to solve it, yielding the optimal solution. , The results summary module translates the solution into natural language: "To ensure fairness in water use for both agriculture and urban areas, after optimization using the alternating direction multiplier method, it is recommended that each of the two areas supply 400,000 cubic meters of water. At this point, the water consumption of both areas is completely equal, and the downstream ecological requirement of a total water supply of not less than 800,000 cubic meters is met." Based on the disclosure and teachings of the foregoing specification, those skilled in the art can make changes and modifications to the above embodiments. Therefore, the present invention is not limited to the specific embodiments disclosed and described above, and some modifications and changes to the present invention should also fall within the protection scope of the claims of the present invention. Furthermore, although some specific terms are used in this specification, these terms are only for convenience of explanation and do not constitute any limitation on the present invention.
Claims
1. An automatic modeling and solution method for constrained optimization problems based on LLM, characterized in that, Includes the following steps: S1. Obtain the user-input question text containing natural language; parse and process the question text to generate structured question features; the question features include the solution objective, constraints, and solution process settings; S2. Using a preset global knowledge base, the problem features are retrieved and matched to obtain mathematical model templates, standard form cases and domain prior knowledge, and a temporary knowledge base is constructed. Based on the temporary knowledge base, comprehensive reasoning is performed to classify and model the problem text mathematically, generating a mathematical model framework. S3. Match the mathematical model framework with the preset algorithm library to generate an executable code file; S4. Call the algorithm interface provided by the algorithm library, use the code interpreter to execute the code file, perform numerical calculations and solve the problem, and obtain the solution result that meets the solution objective; S5. Analyze the solution results and summarize the entire process from the problem text to the solution results, and output the summary results.
2. The automatic modeling and solution method for constrained optimization problems based on LLM according to claim 1, characterized in that, In S1, the information input by the user includes plain text and / or files; The plain text may be part or all of the question text; The file includes one or any combination of document, table, or image files; Use a file parsing tool to extract part or all of the problem text from the file.
3. The automatic modeling and solution method for constrained optimization problems based on LLM according to claim 1, characterized in that, Among the characteristics of the problem: The solution objective includes a preliminary extracted mathematical expression; the mathematical expression includes one or any combination of fundamental functions, trigonometric functions, logarithmic functions, exponential functions, norms, differentials, integrals, and indicator functions; The constraints are restrictive conditions related to the solution objective, including equality constraints, inequality constraints, and variable boundaries; The solution process settings include parameter configuration, number of iterations, convergence accuracy, and output format; In step S4, the code interpreter executes the code file according to the solution process settings.
4. The automatic modeling and solution method for constrained optimization problems based on LLM according to claim 3, characterized in that, In step S2, the problem classification is the division of mathematical problem types into the problem to be solved; the mathematical problem types include linear programming, nonlinear programming, integer programming, convex optimization, and nonconvex optimization. The mathematical model framework includes a standardized objective function, a set of constraints, and a variable space. The objective function is obtained by transforming the mathematical expression. The constraint set is obtained based on the constraints; the constraint set is one or any combination of a linear equality constraint set, a linear inequality constraint set, a nonlinear equality constraint set, and a nonlinear inequality constraint set. The variable space includes the dimension, boundary, and variable type of the variables; the variable type includes continuous and discrete.
5. The automatic modeling and solution method for constrained optimization problems based on LLM according to claim 4, characterized in that, In step S3, a target algorithm is matched in the algorithm library according to the type of mathematical problem; and the code file is generated according to the target algorithm and the mathematical model framework. The code file includes a problem code and any number of data preprocessing codes; The data preprocessing code is used to clean, transform, and perform feature engineering on the data provided in the problem text; the problem code is used to solve the objective function.
6. The automatic modeling and solution method for constrained optimization problems based on LLM according to claim 5, characterized in that, In step S4, the code interpreter executes the data preprocessing code first, and then executes the problem code.
7. The automatic modeling and solution method for constrained optimization problems based on LLM according to claim 4, characterized in that, The format of the code file is associated with C++; The generation of the code file includes: The information contained in the mathematical model framework is identified and preprocessed using preset keywords; the preprocessing includes removing whitespace characters and segmenting the mathematical expressions; The pre-processed mathematical expression is tokenized by a lexical analyzer, and an abstract syntax tree is constructed by a parser based on operator precedence and associativity to structurally express the objective function and the constraints. The abstract syntax tree is rendered as a C++ code string, and header files are automatically generated.
8. The automatic modeling and solution method for constrained optimization problems based on LLM according to claim 1, characterized in that, The algorithms in the algorithm library include sequential quadratic programming, augmented Lagrange method, gradient projection method, and exterior point penalty function method.
9. The automatic modeling and solution method for constrained optimization problems based on LLM according to claim 1, characterized in that, In step S5, the problem text, problem features, mathematical model framework, code file, and solution results are obtained, summarized, and the summary result is generated; wherein, the solution results are subjected to structured parsing and semantic extraction to generate readable natural language text; The summary results are output directly in natural language or in the form of a report; wherein, the code interpreter is invoked to generate the report using a script; the report includes charts and text content.
10. An automatic modeling and solving system for LLM-based constrained optimization problems, employing the automatic modeling and solving method for LLM-based constrained optimization problems as described in any one of claims 1 to 9, characterized in that, Includes the following modules: The problem decomposition module is used to parse and process the problem text to generate the problem features; A mathematical modeling module is used to construct the mathematical model framework based on the characteristics of the problem. The intelligent solution module is used to generate and execute the code file according to the mathematical model framework to obtain the solution result; The results summary module is used to summarize and output the summary results.