Problem generation method and system fusing large language model and symbolic reasoning engine

Abstract

The application discloses a kind of fusion big language model and symbolic reasoning engine problem generation method and system, problem generation method includes receiving input question described in natural language, input question is converted into structured intermediate language representation, then intermediate language representation is converted into formalized logic predicate, after detecting that formalized logic predicate meets grammar specification, output;Formalized logic predicate is constructed by logic geometry, new geometric point is added using geometric reasoning engine, new geometric point is selected by geometric point biased selection algorithm to construct new geometric problem, and candidate question is obtained by verifying and screening new geometric problem;Difficulty coefficient and difference score are calculated to candidate question, and generated question of different difficulty and structural difference is screened out.The application does not need to rely on artificial, by introducing symbolic reasoning engine as the constraint of generation process, eliminate wrong question generation risk, reduce the proportion of invalid question generation, realize question difficulty stratification and large-scale high-quality output.

Description

Technical Field

[0001] This invention belongs to the field of computer technology, specifically relating to a problem generation method and system that integrates a large language model and a symbolic reasoning engine. Background Technology

[0002] Geometry problems mainly study the shape, size, positional relationships and properties in space, covering two major fields: plane geometry and solid geometry. The core of geometry problems is logical reasoning and spatial imagination.

[0003] Currently, existing methods for generating geometric problems are mainly divided into three paradigms: template-based, enumeration-based, and manual-based.

[0004] (1) Template-based generation: In this type of method, experts typically abstract common geometric configurations into a series of structured templates in the early stages and set placeholders for key elements. When the system generates questions, it only needs to inject random or controlled parameters as needed to quickly produce "parallel questions" in batches. Template-based generation has a low implementation threshold, the correctness of questions is easy to guarantee, and the difficulty of questions can be coarsely adjusted by limiting the range of parameters. However, this type of method has significant limitations in terms of scale and diversity. The generated questions are almost structurally similar to the original templates, have poor scalability, are difficult to generate innovative question forms, and rely too much on the quality of template design, making it difficult to cover the entire problem space in the geometric domain.

[0005] (2) Enumeration-based generation: This type of method attempts to exhaustively explore all possible combinations of geometric problems. The algorithm first places several basic elements on the geometric plane, then enumerates their positional relationships or constraints, and automatically derives verifiable conclusions by combining classical theorem libraries, finally selecting propositions that meet teaching value. Theoretically, this method can generate a massive number of problems with rich configurations, providing a challenging dataset for research-level symbolic reasoning models. However, as the number of elements increases, the combination space often expands exponentially, leading to a sharp increase in search and verification costs; at the same time, the unscreened original output often contains degenerate graphics (such as three points collinear leading to triangle degeneration), self-contradictory premises, or "noise problems" that contribute little to the learning objective.

[0006] (3) Manual generation: This method relies on the experience of experts to manually select, adapt, or derive questions from textbooks, past exam papers, competition papers, and question bank websites. This approach can fully guarantee the quality and authority of the questions, especially in innovative question types and comprehensive application questions. However, its output speed is always limited by manual input and cannot keep up with the pace of large-scale, personalized teaching; in addition, the different styles and preferences of experts make it difficult to standardize the question bank, and the difficulty level and knowledge point coverage lack a unified standard, resulting in insufficient scalability and maintainability.

[0007] With the accelerating trend of personalized and online education, the traditional method of relying on teachers to manually create questions can no longer meet the modern teaching needs of "large scale, comprehensive gradient, and fast feedback". At the same time, the research on next-generation artificial intelligence models also urgently needs a large-scale, highly diverse, and logically consistent geometric question bank to drive breakthroughs in advanced mathematical reasoning. Summary of the Invention

[0008] The purpose of this invention is to address the problems in the prior art by providing a question generation method and system that integrates a large language model and a symbolic reasoning engine. This method eliminates the need for manual intervention by introducing a symbolic reasoning engine as a constraint on the generation process, thereby reducing the risk of generating incorrect questions, lowering the proportion of invalid questions, and achieving high-quality output with tiered question difficulty and on a large scale.

[0009] To achieve the above objectives, the present invention provides the following technical solution: Firstly, a question generation method integrating large language models and symbolic reasoning engines is provided, including: It receives an input question described in natural language, transforms the input question into a structured intermediate language representation, then transforms the intermediate language representation into a formal logical predicate, and outputs the result after checking that the formal logical predicate conforms to the syntax rules. A logical geometry graph is constructed using formal logical predicates. New geometric points are added using a geometric inference engine. A biased selection algorithm is used to select geometric points to construct new geometric problems. Candidate problems are obtained by verifying and filtering the new geometric problems. The difficulty coefficient and difference score of the candidate questions are calculated, and the generated questions with different difficulties and structural differences are selected.

[0010] As a preferred approach, the structured intermediate language representation conforms to the expression habits of natural language in its form, enabling large language models to understand and generate it; and its underlying structure is constructed with reference to the logical framework of geometric predicates, allowing it to be mapped to the target formal language.

[0011] As a preferred approach, the step of detecting whether the formal logical predicate conforms to the grammatical rules and outputting the result is as follows: if the formal logical predicate does not conform to the grammatical rules, the process backtracks to the initial step and re-converts; if it still does not conform after a predetermined number of iterations, the corresponding text conversion process is determined to have failed.

[0012] As a preferred approach, the geometric inference engine employs the AlphaGeometry model, which enhances and expands the logical geometric graph by adding new geometric points. The AlphaGeometry model learns and internalizes the inherent structure of the geometric problem during training by performing the next lexical prediction task on the serialized text, which follows the pattern of "<preconditions><target conclusion><proof process>". In the process of capturing specific geometric configurations, specific types of auxiliary points are introduced to generalize new geometric constructions.

[0013] As a preferred approach, the AlphaGeometry model uses a beam search algorithm to select new geometric constructions during inference, choosing several with scores higher than a set value from all possible candidate solutions while keeping the beam width constant. After constructing new geometric points, a geometric point dependency graph is built for each formally expressed geometric problem. The geometric point dependency graph is a directed acyclic graph. By analyzing the topological characteristics of the geometric point dependency graph, when geometric points with more dependencies than a set value are selected as candidate proof targets, the generated problem is associated with multiple geometric elements, reducing the proportion of meaningless problems generated and improving generation efficiency.

[0014] As a preferred approach, the method of selecting geometric points and constructing a new geometric problem using a biased geometric point selection algorithm includes the following steps: Identify the geometric primitives of the problem and initialize the corresponding scores to 1; Read each subsequent formalized logical predicate and calculate the score of the geometric point according to the following formula:

[0015] In the formula, Let be the geometric point for which the score is to be calculated. geometric point Dependence on geometric points, Scoring of geometric points; Based on the topological features of the geometric point dependency graph, geometric points with higher scores have more dependencies; the geometric points are sorted according to their scores, and points with higher scores are selected first, and then combined with candidate logical predicates to randomly generate the proof target. A candidate predicate list is established. By adjusting the range of candidate predicates in the candidate predicate list, a geometric problem is constructed for a specific proof objective, making the generated geometric problem personalized. The generated candidate proof objective is merged with the reasoning cache of the original problem, and this result is combined with the stem of the original problem to construct a brand new candidate problem.

[0016] As a preferred embodiment, the step of verifying and screening new geometric problems to obtain candidate problems includes: The maximum inference closure of the problem stem is obtained by running the geometric inference engine. The proof objective of the candidate problem is checked based on the inference closure. Unsolvable candidate problems are discarded, and only solvable candidate problems are kept. For the selected candidate problems, a backtracking simplification algorithm is used, with the proof target as the root, to backtrack and obtain the simplest subproblem. The backtracking simplification algorithm starts with the proof target predicate and adds the geometric points in the predicate to the set of used geometric points. Then, it traverses the geometric predicates of the problem in reverse. If the geometric points constructed by the geometric predicate are in the set of used geometric points, then all geometric points involved in the corresponding predicate are added to the set of used geometric points. If the geometric points constructed by the geometric predicate are not in the set of used geometric points, then the index of the corresponding predicate is recorded. After the traversal is completed, the candidate problem deletes the corresponding predicate according to the recorded deleted predicate index, and obtains a solvable and simplest new subproblem.

[0017] As a preferred approach, the difficulty coefficient of the candidate problems is calculated according to the following expression, and generated problems of different difficulties are selected:

[0018] In the formula, Indicates the number of proof steps for the problem; Indicates the number of geometric points included; This indicates the maximum reasoning depth reached by the geometric reasoning engine when solving the corresponding problem; , , The model selects the corresponding parameters of the original baseline problem as the normalization reference; , , To adjust the parameters to meet the requirements Setting parameters Avoid difficulty level It was set to zero.

[0019] As a preferred approach, the difference scores of candidate questions are calculated according to the following expression to filter out the generating questions with different structural differences:

[0020] In the formula, , , The relative importance of the three indicators used to control structural differences must be met. ; This is a difficulty penalty factor used to adjust the sensitivity to deviations in difficulty.

[0021] Secondly, a question generation system integrating a large language model and a symbolic reasoning engine is provided, including: The logical predicate conversion module is used to receive input questions described in natural language, convert the input questions into structured intermediate language representations, convert the intermediate language representations into formal logical predicates, and output them after checking that the formal logical predicates conform to the syntax rules. The geometry problem extension module is used to construct a logical geometry graph through formal logical predicates, add new geometric points using a geometric inference engine, select geometric points to construct new geometric problems using a biased selection algorithm, and verify and filter the new geometric problems to obtain candidate problems. The problem generation and filtering module is used to calculate the difficulty coefficient and difference score of candidate problems and filter out generated problems with different difficulties and structural differences.

[0022] Compared with the prior art, the present invention has at least the following beneficial effects: To address the urgent need for generating large-scale, high-quality geometric problems, this invention proposes a problem generation method that integrates a large language model and a symbolic reasoning engine. Compared to traditional methods that rely on manual problem creation or simply large language models, this invention receives input problems described in natural language, transforms them into structured intermediate language representations, and then converts these intermediate language representations into formal logical predicates. This structured intermediate language representation bridges the semantic gap between natural and formal languages. This invention constructs a logical geometric graph using formal logical predicates, adds new geometric points using a geometric reasoning engine, and selects geometric points using a biased selection algorithm to construct new geometric problems. During this process, geometric points are scored and sorted by score to obtain those with more direct or indirect dependencies for proving the target. This invention combines a symbolic reasoning engine and a large language model to generate new geometric problems on a large scale. Simultaneously, the biased selection algorithm balances the generation speed and quality. This invention verifies and filters new geometric problems to ensure that the generated geometric problems are necessarily solvable and that the problem stems are kept as simple as possible. Finally, by calculating the difficulty coefficient and difference score of candidate questions, generated questions with different difficulties and structural differences are selected, providing a set of questions with reasonable difficulty gradients and diverse structures. The geometric problem generation method proposed in this invention effectively overcomes the bottlenecks of low efficiency and difficulty in scaling by manual question generation, as well as the technical defects of large language models that are prone to generating incorrect questions and illusions due to the lack of logical verification mechanisms. By introducing a symbolic reasoning engine as the core constraint of the generation process, the method of this invention constructs a complete technical closed loop from semantic understanding and structural derivation to correctness verification, completely eliminating the risk of generating incorrect questions due to AI illusions or enumeration noise, and achieving the goal of high-quality generation that still ensures solvable questions, correct answers, and concise question stems under large-scale output.

[0023] Furthermore, the structured intermediate language representation of this invention conforms to the expression habits of natural language in its form, enabling large language models to understand and generate it. At its underlying structure, it is constructed with reference to the logical framework of geometric predicates, allowing it to be accurately and unambiguously mapped to the target formal language. The intermediate language representation unifies the conversion path from natural language to formal logical predicates, significantly reducing semantic ambiguity and conversion errors, laying the foundation for subsequent parallel problem generation. Combining auxiliary point generation and bundle search strategies, and achieving biased target generation and inference closure reuse based on geometric point dependency graphs, significantly improves search efficiency, enabling the efficient and batch generation of hundreds of new problems with different structures, all of which are solvable, from a single parent problem.

[0024] Furthermore, the symbolic reasoning engine not only provides formalized and verifiable proofs of question correctness, but also uses a backtracking simplification algorithm to backtrack from the proof target to obtain the simplest subproblems, automatically eliminating redundant conditions in the question stem to ensure the simplicity and effectiveness of the generated questions. In addition, it introduces "difference scores" and multi-dimensional difficulty coefficients to jointly filter questions from the perspectives of structural changes and difficulty proximity, effectively avoiding question homogenization, and supporting fine-grained difficulty stratification and personalized recommendations.

[0025] Furthermore, the method of this invention receives input questions described in natural language, which are then preprocessed and translated by a large language model that has undergone prompt word engineering optimization and context learning, and transformed into a structured intermediate language representation. This intermediate language representation is then converted into formal logical predicates. A strict error detection mechanism is implemented during the conversion process; only predicates that fully conform to grammatical rules are considered successfully converted. Otherwise, the process backtracks to the initial step and re-converts. If errors still exist in the logical predicates after a predetermined number of iterations, the corresponding text conversion process is ultimately deemed a failure, and subsequent processing is terminated. By introducing a strong constraint mapping mechanism in the natural language to intermediate language conversion stage, the conversion accuracy is significantly improved (experimental accuracy reaches 67.65%, an improvement of 17.65% and 9.32% compared to directly using DeepSeek-V3 and ChatGPT-4o, respectively), fundamentally reducing key errors such as parameter order, number, and predicate confusion, and lowering the proportion of invalid questions generated. Attached Figure Description

[0026] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention. For those skilled in the art, other related drawings can be obtained from these drawings without creative effort.

[0027] Figure 1 Flowchart of the problem generation method integrating large language model and symbolic reasoning engine in this invention embodiment; Figure 2 This invention provides a geometric predicate diagram obtained through intermediate language representation conversion. Figure 3 A flowchart illustrating the formal logical predicate transformation of the input problem according to an embodiment of the present invention; Figure 4 This invention constructs a new geometric problem flowchart using a logical geometric graph; Figure 5 Example diagram of constructing a geometric point dependency graph according to an embodiment of the present invention; Figure 6 An example diagram of the input geometry problem for generating geometry problems on the JGEX dataset in this embodiment of the invention; Figure 7 Example diagram of a simple geometric problem generated by embodiments of the present invention; Figure 8 Example diagram of complex geometric problems generated by embodiments of the present invention; Figure 9 A text conversion interface diagram in the automated geometric problem generation system of this invention; Figure 10 A problem generation interface diagram of the automated geometric problem generation system of this invention; Figure 11 A detailed information interface diagram of the generated problem in the automated geometric problem generation system of this invention. Detailed Implementation

[0028] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, those skilled in the art can obtain other embodiments without creative effort.

[0029] Please see Figure 1 This invention proposes a problem generation method that integrates a large language model and a symbolic reasoning engine. First, it performs a translation from natural language to formal geometric predicates, using an intermediate language to bridge the semantic gap and ensure the accuracy of text conversion. Next, it generates new problems based on logical geometric graphs and a geometric reasoning engine, introducing a biased construction algorithm to balance the generation speed and quality. Finally, it comprehensively evaluates the generated problems by calculating difficulty coefficients and difference scores, achieving classification and filtering of the generated problems. The problem generation method of this invention mainly includes the following steps: S1. Receive the input question described in natural language, transform the input question into a structured intermediate language representation, then transform the intermediate language representation into a formal logical predicate, and output the formal logical predicate after checking that it conforms to the syntax rules. S2. Construct a logical geometry graph using formal logical predicates, add new geometric points using a geometric reasoning engine, select geometric points using a biased selection algorithm to construct new geometric problems, and verify and filter the new geometric problems to obtain candidate problems. S3. Calculate the difficulty coefficient and difference score for candidate questions, and filter out generated questions with different difficulties and structural differences.

[0030] In the research and application of geometric problems, a common phenomenon is that their initial expressions often rely on natural language, possessing extremely high expressive flexibility. However, these natural language expressions are often accompanied by significant ambiguity and implicitly contain a large amount of unexpressed background knowledge, posing a significant obstacle to understanding for computer systems lacking corresponding cognitive abilities. Therefore, it is necessary to translate them into a formal language that computers can recognize. Using a large language model, this invention introduces a carefully designed intermediate language to bridge the semantic gap in translation, such as... Figure 2 As shown, the structured intermediate language representation in this embodiment conforms to the expression habits of natural language in terms of expression form, enabling large language models to understand and generate it; at the underlying structure level, it is constructed with reference to the logical framework of geometric predicates, ensuring that it can be accurately and unambiguously mapped to the target formal language.

[0031] Please see Figure 3 In this embodiment of the invention, step S1 employs a phased processing strategy. First, it receives a question input described in natural language. Then, a large language model, optimized through prompt word engineering and context learning, preprocesses and translates the input, transforming it into a structured intermediate language representation. Next, this intermediate language is converted into formalized logical predicates. It is worth noting that during the conversion process, the system executes a strict error detection mechanism. Only predicates that fully conform to grammatical rules are considered successfully converted; otherwise, the algorithm will backtrack to the initial step and retry the translation. If, after a predetermined number of iterations, the logical predicates still contain errors, the system will ultimately determine that the text conversion process has failed and terminate subsequent processing.

[0032] Please see Figure 4 In one possible implementation, step S2 completes the construction of a large number of candidate problems. The initial stage of problem construction realizes the extraction of logical relations of geometric problems, that is, constructing a logical geometric graph through formal predicate information. This process transforms the two-point relations and geometric primitives described by the formal language into the program's internal representation and uses a pre-built knowledge base to discover potential new relations or conflicts. In addition, the program establishes a series of caching structures in this process to accelerate the reasoning and computation of the subsequent inference engine. This process aims to accurately convert the formal geometric problem description into a structured internal representation that is easy for the machine to perform symbolic deduction for subsequent iteration and retrieval.

[0033] After the initial construction of the logical geometry graph, a geometric construction generation model—AlphaGeometry—is introduced. This model enhances and expands the logical geometry graph by intelligently adding new geometric points. AlphaGeometry is essentially a language model, trained from scratch on a massive synthetic dataset containing millions, even hundreds of millions, of geometric theorems and their corresponding machine-readable proofs. The generation and organization of this training data are meticulously designed and serialized into a structured text string. This serialization format strictly follows the pattern of "<preconditions><target conclusion><proof process>". By performing a standard next-word prediction task on this large-scale serialized text, the AlphaGeometry language model gradually learns and internalizes the inherent structure of geometric problems during training. This learning mechanism enables the model to identify common patterns in geometric theorems, typical auxiliary construction strategies, and complex interrelationships between different geometric construction steps. The model can capture specific types of auxiliary points, such as midpoints, perpendicular feet, and angle bisector intersections, within specific geometric configurations. More importantly, by learning from massive proof processes, the language model can generalize to predict useful new constructions in unseen geometric problems, which is the core advantage of its "adding entirely new geometric points" function.

[0034] To enhance the diversity of geometric point generation, the AlphaGeometry model employs a beam search algorithm during inference to select the most valuable new geometric constructions. The core of this algorithm lies in maintaining a fixed-width "bundle" containing the most promising partial construction schemes. In each inference step, the model generates multiple possible continuations for each candidate scheme, then selects the top-scoring options from all possible new candidates, keeping the bundle width constant. This strategy allows the algorithm to explore multiple potential paths, avoiding the possibility of missing out on better constructions due to premature single-choice selection. The pseudocode for the beam search algorithm is as follows:

[0035] Please see Figure 5After constructing entirely new geometric points, a pruning algorithm based on a construction dependency graph is employed to generate the biased proof target. Formalized geometric problems exhibit explicit construction dependencies. These problems typically begin with geometric primitive predicates, and subsequent predicates necessarily rely on previously defined geometric elements for construction description, without referencing points that appear later. Based on this inherent dependency characteristic, this embodiment constructs a precise geometric point dependency graph structure for each formalized problem. A geometric point dependency graph is a directed acyclic graph. Analyzing its topological characteristics reveals that nodes closer to the right side typically have more direct or indirect dependencies, while nodes closer to the left have relatively fewer dependencies. Therefore, when selecting geometric points with more dependencies as candidate proof targets, the generated problem will naturally be associated with more geometric elements, effectively reducing the proportion of meaningless problems and improving overall generation efficiency. Accordingly, this embodiment proposes a biased geometric point selection algorithm. By scoring and sorting geometric points, the algorithm can obtain geometric points with more direct or indirect dependencies for proof target generation.

[0036] The pseudocode for the biased selection algorithm for geometric points in this embodiment is as follows:

[0037] The process of selecting geometric points and constructing a new geometric problem using a biased geometric point selection algorithm includes the following steps: Identify the geometric primitives of the problem and initialize the corresponding scores to 1; Read each subsequent formalized logical predicate and calculate the score of the geometric point according to the following formula:

[0038] In the formula, Let be the geometric point for which the score is to be calculated. geometric point Dependence on geometric points, Scoring of geometric points; Based on the topological features of the geometric point dependency graph, geometric points with higher scores have more dependencies; the geometric points are sorted according to their scores, and points with higher scores are selected first, and then combined with candidate logical predicates to randomly generate the proof target. A candidate predicate list is established. By adjusting the range of candidate predicates in the list, geometric problems are constructed for specific proof objectives, making the generated geometric problems more personalized. The generated candidate proof objectives are merged with the inference cache of the original problem, and this result is combined with the stem of the original problem to construct entirely new candidate problems. These candidate problems are not necessarily solvable, and their stems may contain redundant conditions. Therefore, these candidate problems need to be verified to identify and extract the truly solvable subproblems with the most concise conditions.

[0039] In one possible implementation, step S2 verifies and filters the new geometric problem to obtain candidate problems, including: The problem verification section ensures that the generated problems are necessarily solvable, and guarantees the simplicity of the problem stem by performing forward backtracking on the problems. Since candidate problems share the same problem stem, firstly, the maximum inference closure of the problem stem is obtained by running the geometric inference engine. Then, the proof objective of the candidate problems is checked based on the inference closure, and unsolvable candidate problems are discarded, retaining only solvable candidate problems. For the shortlisted candidate problems, a backtracking simplification algorithm is used, with the goal as the root, to backtrack and obtain the simplest subproblems. Similar to the biased selection algorithm based on geometric points, the backtracking algorithm also relies on the constructive dependencies of points to obtain the simplest conditions of the problem. The pseudocode for the backtracking simplification algorithm is as follows:

[0040] The backtracking simplification algorithm begins by proving the target predicate and adds the geometric points in the predicate to the set of used geometric points. Then, it iterates backward through the geometric predicates of the problem. If the geometric points constructed by the geometric predicate are in the set of used geometric points, all geometric points involved in the corresponding predicate are added to the set of used geometric points. If the geometric points constructed by the geometric predicate are not in the set of used geometric points, the index of the corresponding predicate is recorded. After the traversal is completed, the candidate problem deletes the corresponding predicate according to the recorded index of the deleted predicate, resulting in a solvable and simplest new subproblem.

[0041] In one possible implementation, step S3 scientifically evaluates and classifies these generated questions, calculates the difficulty coefficient of the candidate questions according to the following expression, and filters out generated questions of different difficulties:

[0042] In the formula, Indicates the number of proof steps for the problem; Indicates the number of geometric points included; This indicates the maximum reasoning depth reached by the geometric reasoning engine when solving the corresponding problem; , , The model selects the corresponding parameters of the original baseline problem as the normalization reference; , , To adjust the parameters to meet the requirements Setting parameters Avoid difficulty level It was set to zero.

[0043] In practice, problems are typically set up during generation. To reflect the different weights of the number of proof steps, the number of geometric points, and the depth of reasoning on the overall difficulty; set To avoid difficulty level The values ​​are set to zero. This normalization and weighting model allows for a more objective quantification of the difficulty of each generated problem relative to the original problem.

[0044] Meanwhile, to ensure that the generated problems have a reasonable difficulty distribution while covering a wider range of structural variations, problems are also screened based on a "difference score." The difference score calculation considers not only the relative changes in structural elements between the generated and original problems but also the similarity in difficulty between the two. Specifically, the difference score is calculated for candidate problems according to the following expression to filter out generated problems with different structural differences:

[0045] In the formula, , , The relative importance of the three indicators used to control structural differences must be met. ; This is a difficulty penalty factor used to adjust the sensitivity to deviations in difficulty. Generally, it is set... , In this way, questions of similar difficulty and those with structural variations are prioritized for selection, striving to maximize the diversity and exploratory significance of generated questions while preserving the core educational and reasoning values ​​of the original questions.

[0046] Finally, the generated question bank was scientifically stratified and screened by combining the difficulty coefficient and the difference score. On the one hand, questions with a D < 1 / 2, i.e., those that were too simple and lacked challenge, were filtered out; on the other hand, new questions with rich structural variations and similar difficulty levels to the original questions were prioritized from the remaining candidate questions. This comprehensive evaluation ensured that the final selected question set maintained its relevance to the source questions while providing variations of questions with different difficulties and characteristics.

[0047] The following experiments use the JGEX dataset as experimental data to test the problem generation method of this invention, which integrates a large language model and a symbolic reasoning engine. This dataset contains 204 standard plane geometry problems. These problems are formally described using JGEX and their difficulty level ranges from junior high to high school geometry problems. To construct an effective test corpus, all problems in the dataset were professionally translated by humans, creating a set of parallel data pairs between natural language and corresponding formal language, providing a reliable foundation for subsequent experimental evaluation.

[0048] In this experiment, the text conversion algorithm used Deepseek-V3 as the base model, and the translation test was conducted using the algorithm flow of the text conversion module. Each geometry problem was only attempted once. The testing of Deepseek-V3 and ChatGPT-4o utilized the openai SDK to communicate with the model API, and learned translation rules from natural language to formal language using prompt words. Finally, regular expressions were used to extract the text. <answer>< / answer> The text within the label serves as the model's final response.

[0049] For the test results of three different translation methods, this experiment used manual methods for detection and evaluation. The experimental results are shown in Tables 1, 2, and 3. The experimental results show that the text conversion algorithm proposed in this embodiment significantly outperforms the baseline methods in terms of the number of correctly translated questions. The text conversion algorithm correctly translated 138 questions, accounting for 67.65% of the total questions, which is a significant improvement compared to Deepseek-V3's 50.00% and ChatGPT-4o's 58.33%. This indicates that by introducing an intermediate language as a bridge and combining it with the context learning capabilities of a large language model, the accuracy of converting geometric problems from natural language to formal language can be effectively improved.

[0050] Table 1. Comparison of Correct Translation Results of Different Translation Methods

[0051] Table 2 Number of Translation Error Types by Translation Method (Part 1)

[0052] Table 3 Number of Translation Error Types by Translation Method (Part 2)

[0053] Analyzing the distribution of various error types in the table reveals that the text conversion algorithm has a significant advantage in reducing parameter order errors, parameter quantity errors, and predicate fusion errors, effectively eliminating these three types of errors during the translation process. This is mainly attributed to the design of the intermediate language and the strict matching mechanism of the mapping translation program, which ensures the accuracy of the generated formal representation throughout its structure. Regarding predicate confusion errors, the text conversion algorithm also shows improvement over the baseline method, but 15 error cases still exist, indicating that even with intermediate language conversion, there is still room for improvement in the semantic understanding of natural language problems.

[0054] The following description, in conjunction with the accompanying drawings, illustrates some of the results of problem generation on the JGEX dataset according to an embodiment of the present invention.

[0055] Please see Figure 6 ,from Figure 6 Starting from the parent problem, the geometry problem generation algorithm successfully generated 323 new solvable subproblems, including 284 isoangular problems, 19 parallel problems, 7 perpendicular problems, 7 similarity problems, 3 congruent problems, and 3 concyclic problems. In the problem generation process, the algorithm retains the core geometric idea of ​​the circumcircle of the triangle from the parent problem, while creating new problems by adding or subtracting geometric elements and transforming the proof objective. Both the simple and complex problems generated demonstrate good logical consistency and have complete and feasible proof paths.

[0056] Please see Figure 7 , Figure 7 The simpler problem retains the basic triangle and circumcenter conditions from the parent problem, expanding the geometric elements by introducing points on the perpendicular bisector and midpoint relationships. The proof is clearly structured, relying primarily on midpoint properties, parallel line relationships, and basic angle relationships for reasoning; the steps are relatively concise and intuitive. While the problem's complexity is similar to the parent problem, it retains the core ideas of exploring circumcenter properties and angle relationships, making it suitable as a basic practice problem.

[0057] Please see Figure 8 , Figure 8 The complexity of the problem significantly increases the difficulty, constructing a more intricate network of geometric relationships by introducing the concept of circumcenter and additional auxiliary points. The proof process comprises 21 steps, forming multiple interwoven chains of reasoning, requiring the comprehensive application of advanced geometric techniques such as parallel line properties, similar triangle criteria, and angle tracking. Steps 17 and 18, in particular, require integrating multiple previously established angular relationships for reasoning, highlighting the problem's complexity and challenge, making it suitable as an advanced training exercise.

[0058] Overall, these two examples fully demonstrate the effectiveness of the problem generation algorithm in generating geometric problems. The algorithm not only creates new problems with diverse structures based on a parent problem, but also reasonably controls the difficulty of the problems, providing suitable practice materials for learners of different levels. This ability to generate problems according to difficulty gradients has practical application value in geometry teaching, helping students understand and master relevant geometric concepts from different perspectives and improve their geometric thinking skills.

[0059] Another embodiment of the present invention proposes a question generation system that integrates a large language model and a symbolic reasoning engine, comprising: The logical predicate conversion module is used to receive input questions described in natural language, convert the input questions into structured intermediate language representations, convert the intermediate language representations into formal logical predicates, and output them after checking that the formal logical predicates conform to the syntax rules. The geometry problem extension module is used to construct a logical geometry graph through formal logical predicates, add new geometric points using a geometric inference engine, select geometric points to construct new geometric problems using a biased selection algorithm, and verify and filter the new geometric problems to obtain candidate problems. The problem generation and filtering module is used to calculate the difficulty coefficient and difference score of candidate problems and filter out generated problems with different difficulties and structural differences.

[0060] Please see Figure 9 In this embodiment of the invention, the problem generation system that integrates a large language model and a symbolic reasoning engine first enters the text of the geometric problem described in natural language into the text box. After clicking the "Analyze Problem" button, the system starts to call the text conversion module and the geometric drawing program to translate the natural language into formal predicates and draw the visualization image.

[0061] After successful text conversion, the webpage displays a notification indicating successful parsing, and shows the geometric problem described by the formal predicate in a second text box. Users can choose to use the formal description directly or make slight modifications. Afterwards, users can choose to generate a problem based on this description, such as... Figure 10 As shown.

[0062] After clicking the "Generate Question" button, the system will invoke the question generation module and difficulty filtering module, and display the generated questions in the question card. Furthermore, users can use the webpage's category filtering buttons, difficulty range slider, and sorting method buttons to filter and sort the questions displayed on the right, selecting personalized questions that suit their needs.

[0063] Finally, when users wish to see detailed information about the generated question, they can click on the corresponding question card. Detailed information, including the question stem, image, and solution process, will then be displayed in a pop-up window. Figure 11As shown.

[0064] Another embodiment of the present invention also provides an electronic device comprising: A memory for storing at least one instruction; and a processor for executing the instructions stored in the memory to implement the question generation method that integrates a large language model and a symbolic reasoning engine.

[0065] Another embodiment of the present invention also proposes a computer-readable storage medium storing at least one instruction, which is executed by a processor in an electronic device to implement the question generation method that integrates a large language model and a symbolic reasoning engine.

[0066] The computer program includes computer program code, which can be in the form of source code, object code, executable file, or some intermediate form. The computer-readable storage medium can include any entity or device capable of carrying the computer program code, a medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory, a random access memory, an electrical carrier signal, a telecommunication signal, and a software distribution medium, etc. It should be noted that the content included in the computer-readable medium can be appropriately added or removed according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable medium does not include electrical carrier signals and telecommunication signals. For ease of explanation, the above content only shows the parts related to the embodiments of the present invention; for specific technical details not disclosed, please refer to the method section of the embodiments of the present invention. This computer-readable storage medium is non-transitory and can be stored in storage devices formed by various electronic devices, enabling the execution process described in the method of the embodiments of the present invention.

[0067] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0068] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0069] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0070] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0071] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the specific implementation of the present invention. Any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A question generation method integrating a large language model and a symbolic reasoning engine, characterized in that, include: It receives an input question described in natural language, transforms the input question into a structured intermediate language representation, then transforms the intermediate language representation into a formal logical predicate, and outputs the result after checking that the formal logical predicate conforms to the syntax rules. A logical geometry graph is constructed using formal logical predicates. New geometric points are added using a geometric inference engine. A biased selection algorithm is used to select geometric points to construct new geometric problems. Candidate problems are obtained by verifying and filtering the new geometric problems. The difficulty coefficient and difference score of the candidate questions are calculated, and the generated questions with different difficulties and structural differences are selected.

2. The question generation method integrating a large language model and a symbolic reasoning engine according to claim 1, characterized in that, The structured intermediate language representation conforms to the expression habits of natural language in its form, enabling large language models to understand and generate it; at the underlying structure level, it is constructed with reference to the logical framework of geometric predicates, and can be mapped to the target formal language.

3. The question generation method integrating a large language model and a symbolic reasoning engine according to claim 1, characterized in that, The step of detecting whether a formal logical predicate conforms to the grammatical rules and outputting the result is as follows: if the formal logical predicate does not conform to the grammatical rules, the process backtracks to the initial step and re-converts; if it still does not conform after a predetermined number of iterations, the corresponding text conversion process is determined to have failed.

4. The question generation method integrating a large language model and a symbolic reasoning engine according to claim 1, characterized in that, The geometric inference engine employs the AlphaGeometry model, which enhances and expands the logical geometric graph by adding new geometric points. The AlphaGeometry model learns and internalizes the inherent structure of the geometric problem during training by performing the next lexical prediction task on the serialized text, which follows the pattern of "<preconditions><target conclusion> <proof process>". In the process of capturing specific geometric configurations, specific types of auxiliary points are introduced to generalize new geometric constructions.

5. The question generation method integrating a large language model and a symbolic reasoning engine according to claim 4, characterized in that, The AlphaGeometry model uses a beam search algorithm to select new geometric structures during inference, choosing several with scores higher than a set value from all possible candidate schemes while keeping the beam width constant. After constructing new geometric points, a geometric point dependency graph is built for each formally expressed geometric problem. The geometric point dependency graph is a directed acyclic graph. By analyzing the topological characteristics of the geometric point dependency graph, when geometric points with more dependencies than a set value are selected as candidate proof targets, the generated problem is associated with multiple geometric elements, reducing the proportion of meaningless problems generated and improving generation efficiency.

6. The question generation method integrating a large language model and a symbolic reasoning engine according to claim 5, characterized in that, The method of selecting geometric points and constructing a new geometric problem using a biased geometric point selection algorithm includes the following steps: Identify the geometric primitives of the problem and initialize the corresponding scores to 1; Read each subsequent formalized logical predicate and calculate the score of the geometric point according to the following formula: In the formula, Let be the geometric point for which the score is to be calculated. geometric point Dependence on geometric points, Scoring of geometric points; Based on the topological features of the geometric point dependency graph, geometric points with higher scores have more dependencies; the geometric points are sorted according to their scores, and points with higher scores are selected first, and then combined with candidate logical predicates to randomly generate the proof target. A candidate predicate list is established. By adjusting the range of candidate predicates in the candidate predicate list, a geometric problem is constructed for a specific proof objective, making the generated geometric problem personalized. The generated candidate proof objective is merged with the reasoning cache of the original problem, and this result is combined with the stem of the original problem to construct a brand new candidate problem.

7. The question generation method integrating a large language model and a symbolic reasoning engine according to claim 6, characterized in that, The steps for verifying and filtering new geometric problems to obtain candidate problems include: The maximum inference closure of the problem stem is obtained by running the geometric inference engine. The proof objective of the candidate problem is checked based on the inference closure. Unsolvable candidate problems are discarded, and only solvable candidate problems are kept. For the selected candidate problems, a backtracking simplification algorithm is used, with the proof target as the root, to backtrack and obtain the simplest subproblem. The backtracking simplification algorithm starts with the proof target predicate and adds the geometric points in the predicate to the set of used geometric points. Then, it traverses the geometric predicates of the problem in reverse. If the geometric points constructed by the geometric predicate are in the set of used geometric points, then all geometric points involved in the corresponding predicate are added to the set of used geometric points. If the geometric points constructed by the geometric predicate are not in the set of used geometric points, then the index of the corresponding predicate is recorded. After the traversal is completed, the candidate problem deletes the corresponding predicate according to the recorded deleted predicate index, and obtains a solvable and simplest new subproblem.

8. The question generation method integrating a large language model and a symbolic reasoning engine according to claim 1, characterized in that, The difficulty coefficient of the candidate questions is calculated according to the following expression, and generated questions of different difficulties are selected: In the formula, Indicates the number of proof steps for the problem; Indicates the number of geometric points included; This indicates the maximum reasoning depth reached by the geometric reasoning engine when solving the corresponding problem; , , The model selects the corresponding parameters of the original baseline problem as the normalization reference; , , To adjust the parameters to meet the requirements Setting parameters Avoid difficulty level It was set to zero.

9. The question generation method integrating a large language model and a symbolic reasoning engine according to claim 8, characterized in that, The candidate questions are scored using the following expression to filter out the generated questions with different structural differences: In the formula, , , The relative importance of the three indicators used to control structural differences must be met. ; This is a difficulty penalty factor used to adjust the sensitivity to deviations in difficulty.

10. A question generation system integrating a large language model and a symbolic reasoning engine, characterized in that, include: The logical predicate conversion module is used to receive input questions described in natural language, convert the input questions into structured intermediate language representations, convert the intermediate language representations into formal logical predicates, and output them after checking that the formal logical predicates conform to the syntax rules. The geometry problem extension module is used to construct a logical geometry graph through formal logical predicates, add new geometric points using a geometric inference engine, select geometric points to construct new geometric problems using a biased selection algorithm, and verify and filter the new geometric problems to obtain candidate problems. The problem generation and filtering module is used to calculate the difficulty coefficient and difference score of candidate problems and filter out generated problems with different difficulties and structural differences.

Images