A difficulty controllable question generating method and system based on bloom cognitive hierarchy

CN122594511BActive Publication Date: 2026-10-27ANHUI UNIV
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202611096701.9
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2026-07-23
Publication Date
2026-10-27
Estimated Expiration
2046-07-23

AI Technical Summary

Technical Problem

[0005]本发明所要解决的技术问题在于现有自动出题系统缺乏对题目认知层级的显式建模,导致题目难度不可控、难以匹配学生能力水平的问题

Benefits of technology

本发明面向智慧教育与分层教学中的自动出题应用场景。首先,提出参考Bloom认知分类法的K1至K6知识单元认知层级分类体系,将其转化为考点知识图谱节点的核心属性进行显式标注以区分不同认知阶段;其次,通过大语言模型驱动的P1至P5分轮标注机制,结合消融验证与语义聚类去重策略,从真实题目中自动抽取并构建带有认知层级的知识单元及其依赖关系,实现考点知识图谱的大规模自动化构建;接着,引入教学语言层、锚点层与底层知识节点层构成的三层映射结构,实现从自然教学语言到底层认知知识子图的自动映射,有效降低系统的使用门槛;然后,建立认知层级与知识子图间的显式对应关系,在出题前通过知识子图检索进行节点约束,并在评估时引入一致性验证机制,形成“出题约束—题目生成—结果验证”的认知层级控制闭环;最后,依托多模型协作的“生成—评估—反思”迭代流程,对题目的知识覆盖度、认知准确度等多维度进行综合评估与自动优化,并预留接口支持教师反馈以实现人机协作,保障生成题目的质量稳定性与教学可用性。

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN122594511B_ABST
    Figure CN122594511B_ABST
Patent Text Reader

Abstract

The application discloses a difficulty controllable question setting method and system based on Bloom cognitive hierarchy, and belongs to the technical field of intelligent education, solves the problem that the existing automatic question setting system lacks explicit modeling of question cognitive hierarchy, leading to uncontrollable question difficulty and difficulty in matching student ability level; the application defines K1 to K6 six types of knowledge units corresponding to Bloom classification method, constructs a subject examination point knowledge graph with cognitive hierarchy annotation through five rounds of hierarchical annotation driven by a large language model, ablation verification and semantic clustering deduplication, and establishes a mapping from a natural language anchor point to a knowledge node; cognitive constraints are determined through anchor point projection and knowledge subgraph retrieval during question setting, high-quality questions are output through candidate question generation, multi-dimensional evaluation and reflection iteration, human-computer cooperation closed loop supporting teacher feedback is supported, and the effect of the automatic question setting system in cognitive matching, difficulty controllability and personalized teaching support is effectively improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the field of smart education technology, and in particular to a method and system for generating questions with controllable difficulty based on Bloom's hierarchy of knowledge. Background Technology

[0002] Against the backdrop of rapid development in artificial intelligence technology, intelligent question generation and intelligent learning assistance have become important research directions in the field of smart education. In particular, generative artificial intelligence technology, represented by large language models, is widely used in mathematics teaching platforms due to its strong text understanding and generation capabilities. It is used to automatically generate practice questions, analyze problem-solving processes, and provide learning feedback, which is of great significance for improving teaching efficiency and learning experience. In most subject teaching scenarios, knowledge content typically exhibits structured characteristics, with significant differences in cognitive requirements and difficulty levels among different knowledge points. Therefore, reasonable question generation should not only cover relevant knowledge points but also match the teaching objectives and learners' ability levels in terms of cognitive level and complexity. This places higher demands on the knowledge modeling and question generation control capabilities of intelligent question generation systems.

[0003] Existing intelligent math question generation technologies typically combine question bank retrieval, knowledge graphs, or generative models to achieve automatic question generation. Some solutions use knowledge graphs to structurally model knowledge points and generate or recommend questions based on the relationships between these points; others directly utilize large language models to generate question text, improving question diversity and generation efficiency. However, existing technologies generally have several shortcomings: First, current technologies focus primarily on knowledge relationships in modeling mathematical knowledge points, lacking a systematic characterization of cognitive levels. This makes it difficult to distinguish the differences in cognitive requirements among different questions, leading to uncontrollable question difficulty and cognitive levels. Second, existing knowledge graphs typically lack cognitive level annotation mechanisms, making it difficult for the graph structure to directly support tiered question generation or fine-tuning based on learner abilities. Third, question generation methods based on large language models lack effective structural constraints, relying mainly on the model's implicit capabilities, which can easily lead to unstable cognitive levels in questions and unclear teaching objectives, failing to meet the controllability and consistency requirements of math teaching.

[0004] Therefore, there is an urgent need for an intelligent question generation method and system that can introduce cognitive hierarchy annotations at the graph level and realize hierarchical and controllable question generation based on this structure, so as to improve the effectiveness of intelligent question generation in terms of cognitive matching, difficulty stability and personalized teaching support. Summary of the Invention

[0005] The technical problem that this invention aims to solve is that existing automatic question generation systems lack explicit modeling of the cognitive levels of questions, resulting in uncontrollable question difficulty and difficulty in matching students' ability levels.

[0006] The present invention solves the above-mentioned technical problems through the following technical solutions: A method for controlling the difficulty of questions based on Bloom's hierarchy of cognitive levels includes the following steps: Step 1, Knowledge System Construction: Define a classification system of six cognitive level knowledge units corresponding to the six cognitive levels of Bloom's Taxonomy; perform multi-round hierarchical annotation on the original questions and their reasoning trajectories based on a large language model, and automatically extract knowledge units at each cognitive level; perform ablation verification on the extracted knowledge units, verifying one by one whether removing each knowledge unit would cause the corresponding question to fail, retaining the necessary knowledge units that cause the question to fail and filtering out unnecessary annotations; use semantic embedding vectors and hierarchical clustering algorithms to deduplicate and merge the knowledge units; use the deduplicated knowledge units as nodes to construct a knowledge graph of test points with cognitive level annotations; Step 2, Construction of a three-layer mapping anchor system: Teaching phrases are automatically extracted from the question processing results, and the teaching phrases are aggregated through semantic clustering to generate anchor candidates; the anchor candidates are aligned with the knowledge unit node set by embedding similarity calculation and large language model selection; a three-layer mapping structure is established, consisting of a teaching language layer, an independent anchor layer, and a bottom knowledge unit node layer, wherein the independent anchor layer serves as an intermediate mapping layer independent of the test point knowledge graph; Step 3, Layered Question Generation: Receive the subject topic and target Bloom's cognitive level; project the subject topic to the corresponding set of underlying knowledge unit nodes via the three-layer mapping structure through an independent anchor layer; determine a knowledge subgraph containing all dependent nodes based on the target cognitive level; generate structured question prompts based on the knowledge subgraph; call a large language model to generate candidate questions; comprehensively score the candidate questions using preset multi-dimensional indicators; through multiple rounds of generation-evaluation-correction iterative cycles, output questions that meet the target cognitive level.

[0007] Furthermore, the six cognitive-level knowledge unit classification system includes: K1 conceptual knowledge corresponding to the memory level, K2 formula or theorem knowledge corresponding to the understanding level, K3 operational knowledge corresponding to the application level, K4 structural triggering knowledge corresponding to the analysis level, K5 reasoning chain knowledge corresponding to the evaluation level, and K6 combination knowledge corresponding to the creation level. Among them, K4 is path routing knowledge, representing structural triggering and path selection in problem-solving; K5 is execution trajectory knowledge, representing reasoning steps and execution processes in problem-solving; K4 and K5 have strict engineering boundary definitions and cannot be substituted for each other.

[0008] Furthermore, the multi-round hierarchical annotation includes five rounds of hierarchical annotation from P1 to P5. The five rounds of hierarchical annotation are respectively aimed at different cognitive level attributes of knowledge units, and the same question and its reasoning trajectory are annotated in a progressive manner according to the round. The output result of each round serves as the reference input for the next round of annotation.

[0009] Furthermore, the ablation verification includes the following operations: for the set of knowledge units obtained through multiple rounds of hierarchical annotation, one knowledge unit is extracted one by one to form a verification set for the missing knowledge unit; the verification set is substituted into the problem-solving process of the corresponding question for verification; if the problem is solved successfully, the knowledge unit is determined to be an unnecessary annotation and is filtered out; if the problem is solved unsuccessfully, the knowledge unit is determined to be a necessary node and is retained in the test point knowledge graph.

[0010] Furthermore, the graph edges in the test point knowledge graph include four types: dependency, activation, combination, and prerequisite. Among them, dependency means that the dependent of node A is a necessary condition for node B to be dependent on; activation means that when a test point node is triggered, it can activate other related nodes to participate in problem-solving reasoning; combination means that multiple independent test point nodes can construct a new composite test point; prerequisite means that the learning sequence constraint set according to the teaching syllabus and cognitive laws requires that the learning arrangement of node A be earlier than that of node B.

[0011] Furthermore, in the three-layer mapping structure: the mapping from the teaching language layer to the independent anchor layer is determined by semantically encoding the input subject topic and calculating the embedding similarity with each anchor point; the mapping from the independent anchor layer to the underlying knowledge unit node layer is pre-calculated and stored through the anchor alignment stage; the semantic expression of the subject topic completes a two-stage mapping from natural teaching language to the underlying knowledge unit node set through the independent anchor layer.

[0012] Furthermore, the determination of the knowledge subgraph based on the target cognitive level includes: the higher the target cognitive level, the more cross-cognitive level dependent nodes are included in the determined knowledge subgraph; lower-level targets correspond to smaller-scale local knowledge subgraphs; higher-level targets need to include all lower-level preceding dependent nodes to form a complete knowledge subgraph constraint covering multiple cognitive levels.

[0013] Furthermore, the multi-round generation-evaluation-correction iterative cycle includes: when the comprehensive score of the candidate question is lower than the preset threshold, the scoring results and the analysis of deficiencies in each dimension are re-inputted into the large language model as reflection information to drive the large language model to make targeted corrections to the candidate questions; the corrected questions are scored again; the above process is repeated until the comprehensive score reaches the threshold requirement or the preset maximum number of iterations is reached.

[0014] Furthermore, the tiered question generation step also includes: receiving textual feedback from teachers on the output questions; parsing the textual feedback into modification instructions for specific issues in the questions; inputting the modification instructions into a large language model to drive the large language model to make targeted modifications to the questions; and outputting the modified questions for teachers to further confirm, thereby realizing human-computer collaborative question optimization based on teacher feedback.

[0015] This invention also provides a difficulty-controlled question generation system based on Bloom's hierarchy of cognitive levels, comprising: The knowledge system construction module defines a classification system of six cognitive level knowledge units corresponding to the six cognitive levels of Bloom's Taxonomy. Based on a large language model, it performs multi-round hierarchical annotation on the original questions and their reasoning trajectories, automatically extracting knowledge units at each cognitive level. The extracted knowledge units undergo ablation verification, retaining necessary units and filtering out unnecessary annotations. Semantic embedding vectors and hierarchical clustering algorithms are used to deduplicate and merge the knowledge units. The deduplicated knowledge units are then used as nodes to construct a knowledge graph of test points with cognitive level annotations. The three-layer mapping anchor system construction module is used to automatically extract teaching phrases from the question processing results, generate anchor candidates through semantic clustering, complete the alignment of anchors with knowledge unit node sets through embedding similarity calculation and large language model selection, and establish a three-layer mapping structure of teaching language layer, independent anchor layer and bottom knowledge unit node layer, wherein the independent anchor layer is a mapping intermediate layer that exists independently of the test point knowledge graph. The hierarchical question generation engine module receives the subject topic and the target Bloom cognitive level, projects the subject topic to the set of underlying knowledge unit nodes through the three-layer mapping structure via an independent anchor layer, and determines a knowledge subgraph containing all dependent nodes; generates candidate questions based on the knowledge subgraph; performs a comprehensive scoring of the candidate questions using preset multi-dimensional indicators; and outputs questions that meet the target cognitive level after multiple rounds of correction and iteration.

[0016] The advantages of this invention are: This invention addresses the application scenario of automated question generation in smart education and differentiated instruction. First, it proposes a K1-K6 knowledge unit cognitive hierarchy classification system, referencing Bloom's cognitive taxonomy, and converts it into explicit annotation of the core attributes of knowledge graph nodes to distinguish different cognitive stages. Second, through a P1-P5 round-by-round annotation mechanism driven by a large language model, combined with ablation verification and semantic clustering deduplication strategies, it automatically extracts and constructs knowledge units with cognitive hierarchy and their dependencies from real questions, achieving large-scale automated construction of the knowledge graph. Third, it introduces a three-layer mapping structure consisting of a teaching language layer, an anchor layer, and a bottom-level knowledge node layer, realizing a mapping from natural teaching language to the bottom layer. The automatic mapping of cognitive knowledge subgraphs effectively lowers the barrier to entry for the system. Then, an explicit correspondence is established between cognitive levels and knowledge subgraphs. Before generating questions, node constraints are performed through knowledge subgraph retrieval, and a consistency verification mechanism is introduced during evaluation, forming a cognitive level control closed loop of "question constraints - question generation - result verification". Finally, relying on the iterative process of "generation - evaluation - reflection" through multi-model collaboration, the knowledge coverage, cognitive accuracy, and other dimensions of the questions are comprehensively evaluated and automatically optimized. An interface is reserved to support teacher feedback to achieve human-computer collaboration, ensuring the quality stability and teaching usability of the generated questions.

[0017] This invention achieves a deep integration of cognitive hierarchy theory and automatic question generation technology, effectively solving the problem that existing question generation systems are unable to accurately match teaching objectives and learners' abilities. It overcomes the shortcomings of traditional systems, such as high dependence on manual annotation, high technical operation threshold, and uncontrollable cognitive depth of questions, and significantly improves the ease of use, scalability of knowledge graph construction, and accuracy of knowledge assessment of the automatic question generation system in real teaching scenarios. Attached Figure Description

[0018] Figure 1 This is a schematic diagram of the structure of the subject test point knowledge graph construction and hierarchical question generation system based on cognitive hierarchy annotation disclosed in the embodiments of the present invention; Figure 2 This is a flowchart of a subject test point knowledge graph construction and hierarchical question generation system based on cognitive hierarchy annotation disclosed in an embodiment of the present invention; Detailed Implementation To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the embodiments and accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0019] The following combination Figure 1 The system structure shown is as follows: Figure 2 The two-stage execution flow shown provides a detailed explanation of the specific implementation of the present invention's method and system for controllable question generation based on Bloom's hierarchy of knowledge. This embodiment uses high school mathematics as an example, with the topic of "maximum and minimum values ​​of quadratic functions" as a recurring example, to illustrate the workflow of each module.

[0020] I. Core Technology Description (a) Large Language Model Large language models, trained on massive amounts of text data, possess high-quality text generation, natural language understanding, and powerful logical reasoning capabilities. This invention deploys multiple large language models in different modules, fully leveraging the differentiated advantages of each model in logical reasoning, knowledge annotation, and creative generation.

[0021] In the knowledge system construction module, two large language models, GLM-4.7 and GLM-5.1, are used: GLM-4.7 has strong logical reasoning ability and is used for problem-solving trajectory generation and K4 / K5 consistency review verification; GLM-5.1 has strong structured annotation ability and is used for five rounds of structured annotation from P1 to P5, ablation verification and anchor point alignment.

[0022] In the hierarchical question generation engine module, the deepseek-v4-pro large language model is used for candidate question generation, which has strong creative text generation capabilities; GLM-5.1 is used for question quality assessment and reflection optimization, and its assessment and judgment capabilities are stable and reliable.

[0023] (II) Knowledge Map of Examination Points The exam knowledge graph is a directed graph structure composed of nodes and edges, depicting subject knowledge units and their interrelationships. This invention uses NetworkX directed graphs as the in-memory storage and computation carrier for the exam knowledge graph, and persists it in JSON format. Graph nodes correspond to knowledge units of types K1 to K6, and edges correspond to four types of relationships: dependency, activation, combination, and preconditions. Semantic embedding uses the BGE-M3 model to generate vector representations, and clustering uses the scikit-learn hierarchical clustering algorithm to complete the deduplication and merging of knowledge units.

[0024] II. Overall System Structure like Figure 1 As shown, the system of the present invention includes a knowledge unit definition module, a knowledge system construction module, and a hierarchical question generation engine module; wherein the knowledge system construction module includes a question analysis submodule, a knowledge unit extraction submodule, a cognitive level test point knowledge graph construction submodule, and an anchor point system construction submodule.

[0025] like Figure 2As shown, the system executes in two main phases: the knowledge system construction phase and the question generation engine phase. The knowledge system construction phase transforms the original questions into five major databases: a structured knowledge graph of test points, a question bank, a knowledge graph database of test points, a reasoning chain database, an anchor point database, and an experience database. These databases are repeatedly accessed by the question generation engine phase. The question generation engine phase generates high-quality questions that meet the target cognitive level based on the knowledge graph of test points. The knowledge system construction phase only needs to run once; after processing, the data is stored in the databases. The hierarchical question generation engine module can repeatedly access these databases, retrieving knowledge from the database each time questions are generated.

[0026] III. Knowledge System Construction Stage (a) Knowledge Unit Definition Module The knowledge unit definition module is used to define six types of knowledge units from K1 to K6 based on the Bloom hierarchy, establish the correspondence between subject knowledge and cognitive hierarchy theory, and provide a unified knowledge classification standard for all subsequent modules.

[0027] In this embodiment, taking the "maximum and minimum values ​​of quadratic functions" from high school mathematics as an example, the definitions and examples of K1 to K6 are as follows: K1 represents conceptual knowledge, corresponding to the memory level in Bloom's taxonomy, referring to the understanding of "what" a subject object is. For example, "the definition of a quadratic function: a function of the form f(x) = ax² + bx + c (a ≠ 0)" is a K1 node.

[0028] K2 represents formula / theorem knowledge, corresponding to the understanding level in Bloom's taxonomy. It refers to the mastery of a static mathematical conclusion and does not include the judgment of "when to use it". For example, "Vertex formula: The vertex coordinates of the quadratic function are (-b / 2a, f(-b / 2a))" is a K2 node.

[0029] K3 represents operational knowledge, corresponding to the application level in Bloom taxonomy. It refers to the ability to perform a local computational step without involving direction selection. For example, the "Complete operation: transform ax² + bx + c into the form a(x + h)² + k" is a K3 node.

[0030] K4 represents structure-triggered knowledge, corresponding to the analysis level in Bloom taxonomy. It refers to the ability to identify a specific structure in a problem and determine which problem-solving method should be activated. It is only responsible for path selection and does not perform reasoning. The boundary between K4 and K5 is defined as follows: K4 is path routing, and K5 is the execution trajectory; the two strictly do not overlap. For example, "identifying the structure 'finding the maximum and minimum values ​​of a quadratic function on a closed interval' and triggering the method of 'comparing the function values ​​of the vertices and endpoints'" is a K4 node.

[0031] K5 represents the reasoning chain knowledge, corresponding to the evaluation level in Bloom taxonomy. It refers to the ability to execute the reasoning sequence completely after the path is determined, including the annotation of breakpoints at each step. For example, the "chain for solving the maximum and minimum values ​​of a quadratic function in an interval: ① Calculate the vertex coordinates (breakpoint: incorrect vertex formula symbol) → ② Determine if the vertex is within the interval (breakpoint: forgetting to determine the interval range) → ③ Calculate the endpoint function values ​​(breakpoint: incorrect calculation of endpoint substitution) → ④ Compare and obtain the maximum and minimum values ​​(breakpoint: missing a candidate value)" is a K5 node.

[0032] K6 represents combinatorial knowledge, corresponding to the creative level in Bloom's taxonomy. It refers to a cross-knowledge point collaborative application pattern that recurs stably in different problems. For example, "the function extremum interval constraint pattern: the collaborative application of quadratic function + vertex formula + interval endpoints + function value calculation, suitable for finding the extremum of a function on a closed interval" is a K6 node.

[0033] (II) Problem Analysis Submodule The question parsing submodule is responsible for preprocessing the original questions and generating solution paths, while also extracting teaching phrases to provide structured input for subsequent annotation.

[0034] Data preprocessing: Import the MATH500 and AIME2024 datasets into the question bank as the original questions, clean and standardize the question text, and output standardized question JSON records.

[0035] Problem-solving trajectory generation: Using GLM-4.7, perform unhinted chained reasoning on each problem to generate a structured problem-solving trajectory containing the step-by-step derivation process. The output includes a list of reasoning steps, the final answer, and the confidence score. This step provides no hints and requires the model to derive the solution completely autonomously, ensuring that subsequent annotations are based on the actual reasoning process.

[0036] Teaching phrase extraction: For each question, a large language model is invoked to extract topic-level phrases that teachers or textbooks would naturally use when explaining the question. The knowledge points, problem-solving methods, common mistakes, or key concepts involved in these phrases serve as candidate inputs for natural language description of teaching, which are then used for subsequent anchor point clustering. For example, for quadratic function extremum problems, teaching phrases such as "quadratic function extremum," "completing the square method," and "interval endpoints" are extracted.

[0037] (III) Knowledge Unit Extraction Submodule The knowledge unit extraction submodule is responsible for extracting various knowledge units from questions and their solution trajectories, from K1 to K6. After ablation verification, experience card generation, and clustering deduplication, a stable set of Ki nodes is formed.

[0038] 1. Stratified extraction steps A round-based hint strategy is adopted, calling GLM-5.1 to perform five rounds of depth annotation (P1 to P5) on each question and its solution trajectory, and verifying the boundary between K4 and K5 through review consistency verification. Taking "finding the minimum value of f(x) = x² - 4x + 3 on [0, 3]" as a typical example, the annotation content of each round is as follows: P1 is a problem structure annotation: extracting the known conditions, solution objective, implicit constraints, and common mistakes in the problem. P1 output: The known conditions are "f(x)=x²-4x+3" and "interval [0,3]", the solution objective is "find the minimum value", the implicit constraint is "endpoint values ​​need to be considered", and the common mistake is "directly assuming that the vertex is the minimum value and ignoring endpoint checks".

[0039] P2 is used to annotate knowledge units K1 / K2 / K3: extracting the concept definitions (K1), formulas and theorems (K2), and operation steps (K3) involved in the problem. K2 only records the formula itself and does not include the conditions for its use. P2 outputs the K1 node "Definition of Quadratic Function", the K2 node "Vertex Formula x=-b / 2a" and the K3 node "Complete the Square Operation".

[0040] P3 is annotated for the K4 structure trigger rules: extracting the pattern of "identifying the type of problem structure → activating the type of problem-solving method", and annotating the trigger structure, activation method, forbidden operation, and typical error when not identified. P3 output: the trigger structure is "finding the maximum and minimum values ​​of a quadratic function on a closed interval", the activation method is "comparing the function values ​​of the vertices and endpoints", the forbidden operation is "not looking at only the vertices", and the typical error when not identified is "incorrectly assuming that the vertices are the maximum and minimum values".

[0041] P4 is used to annotate the K5 inference chain: extract the complete inference sequence, and annotate each step with a step description and corresponding breakpoint. P4 outputs a four-step inference chain: Step 1 calculates the vertex coordinates (breakpoint: the vertex formula is remembered incorrectly, or the signs are reversed); Step 2 determines whether the vertex is within a given interval (breakpoint: the interval range was not determined); Step 3 calculates the function values ​​of the interval endpoints (breakpoint: the endpoints were substituted incorrectly in the calculation); Step 4 compares to obtain the maximum and minimum values ​​(breakpoint: a value was missed during the comparison).

[0042] P5 is labeled with the K6 combination pattern: extracting multi-knowledge unit collaborative application patterns that stably recur in different questions, recording the applicable conditions, the knowledge unit combinations involved, and the combination description. P5 outputs "Function Extremum Interval Constraint Pattern", the applicable conditions are "finding the extremum of a function on a closed interval and the function is continuous within the interval", and the knowledge unit combination is "definition of quadratic function + vertex formula + interval endpoint check + function value calculation".

[0043] Review is for K4 / K5 consistency verification: call GLM-4.7 to verify the following three items: whether K4 has mixed in execution steps, whether each step of K5 contains breakpoint annotations, and whether the starting point of K5 corresponds to the activation method of K4. Output the verification results, warning messages and modification suggestions.

[0044] 2. Ablation Verification Procedure Ablation tests are performed on each knowledge unit labeled in the hierarchical extraction step. Specifically, for each knowledge unit, the large language model is invoked to attempt to solve the problem under the condition of removing the knowledge unit, and it is determined whether this leads to the failure of solving the problem; the ablation result and reason explanation for each knowledge unit are output. Knowledge units with true ablation results are marked as key knowledge units and assigned higher node weights in subsequent test point knowledge graphs; unnecessary labels with false ablation results are filtered out.

[0045] Taking the above question as an example: After removing the K4 node "maximum value structure trigger", the large language model directly outputs the vertex coordinates as the minimum value, causing the solution to fail. The ablation result is true, and this node is marked as a key knowledge unit. After removing the K1 node "definition of quadratic function", the large language model can still complete the solution based on the function expression. The ablation result is false, and this node is filtered in this question.

[0046] 3. Steps for generating experience cards Based on the common erroneous approaches marked in step P1 and the reasoning breakpoints marked in step P4, experience cards are generated and written into the experience database for reference in subsequent question prompts.

[0047] 4. Clustering and deduplication steps For all questions, K1, K2, and K3 knowledge units are generated using the BGE-M3 semantic embedding model to generate vector representations. Semantically similar nodes are merged using the scikit-learn hierarchical clustering algorithm. The most representative description of each cluster is selected as the canonical name to form a stable set of basic Ki nodes.

[0048] For the K4 triggering rule, semantic embedding similarity clustering based on the triggering structure text is used, with a more stringent similarity threshold than K1 to K3 to preserve fine-grained differences in triggering conditions. For the K5 inference chain, clustering is based on the similarity between the inference step sequence and breakpoint positions. For the K6 combination pattern, clustering is based on the combination features of the knowledge unit types involved. After clustering, the K-type mapping labeled for each cluster corresponds to the Bloom level, forming a stable set of deduplicated Ki nodes. The K5 inference chain and the breakpoints of each step are written into the inference chain library, and the Ki node set is written into the test point knowledge graph library.

[0049] (iv) Sub-module for constructing a knowledge graph of test points at the cognitive level The cognitive level test point knowledge graph construction submodule is responsible for constructing a complete test point knowledge graph and completing manual sampling inspection.

[0050] The steps for constructing the knowledge graph of test points are as follows: Using Ki nodes as graph nodes and the dependencies, activations, combinations, and prerequisites between knowledge units as directed graph edges, a directed knowledge graph of test points G=(V,E) with cognitive level annotations is constructed using NetworkX and persistently stored in JSON format.

[0051] Each node in the node set V corresponds to a knowledge unit, containing the following attributes: node identifier (unique ID), knowledge level (values ​​from K1 to K6), knowledge description (text), subject, and corresponding Bloom cognitive level. Each edge in the edge set E consists of a starting node identifier, a target node identifier, a relation type, and a relation weight; the relation type can be one of four types: "dependency," "activation," "composition," and "precedence," and the relation weight is dynamically assigned based on the criticality determined in the ablation verification step. The specific definitions of the four relation types "dependency," "activation," "composition," and "precedence" are as follows: 1. Dependency Relationship Definition: This refers to a strong logical binding between two test point nodes, where node A (the dependent party) is a necessary condition for node B (the dependent party) to be valid.

[0052] Specific explanation: If learners do not master the knowledge points corresponding to node A, they will not be able to understand or deduce the content of node B; the graph uses unidirectional directed edges to represent this relationship, with the edges pointing from basic concepts to derived concepts. For example, "the domain of a function" is a dependency of "finding the extrema of a function". If the domain is not clearly defined, the work of finding the extrema will not have a basis for implementation.

[0053] 2. Activation Relationship Definition: When a test point node is triggered, it can activate other related nodes to participate in problem-solving reasoning.

[0054] Detailed explanation: This relationship is used to dynamically call knowledge points during the problem-solving process: when the problem to be solved matches node A, the system can rely on the activation relationship to retrieve and call node B, providing matching problem-solving ideas and tips on common mistakes. In this solution's graph, this relationship is mostly used between nodes K4 and K3 / K5: K4 is responsible for identifying the problem structure and selecting suitable problem-solving methods. After selecting a method, it will call the specific calculations and reasoning tips corresponding to nodes K3 / K5 through the activation relationship.

[0055] 3. Combinatorial Relationship Definition: Multiple independent test point nodes are integrated based on specific logic and application scenarios to construct a new composite test point relationship.

[0056] Specific explanation: This relationship reflects the logic of horizontal comprehensive application of knowledge points; in the graph, the high-order K6 nodes bind multiple low-level nodes through combination relationships, integrating multiple basic test points to form complex test scenarios.

[0057] 4. Prerequisites Definition: Learning sequence constraints set according to the teaching syllabus and cognitive laws require that the learning schedule of node A be earlier than that of node B.

[0058] Specific explanation: Prerequisite relationships focus on teaching path planning and are fundamentally different from the hard logical constraints of dependency relationships: Dependency relationships are necessary conditions at the derivation level of knowledge points, while prerequisite relationships only restrict the order of instruction. Even if you skip A, you can theoretically master B, but it doesn't conform to the conventional teaching rhythm. For example, in the course arrangement, "basic operations of sets" is set as a prerequisite node for "function properties," allowing learners to become familiar with the set expression tools required for functions in advance.

[0059] Taking the knowledge graph fragment of "quadratic function extrema" as an example: node K4 "extrema structure trigger" points to node K5 "extrema solution reasoning chain" through the "activation" relation edge; node K2 "vertex formula" points to node K4 "extrema structure trigger" through the "dependency" relation edge; node K1 "quadratic function definition" points to node K2 "vertex formula" through the "precedence" relation edge.

[0060] Manual sampling inspection steps: Key nodes in the knowledge graph of test points are manually reviewed to confirm the accuracy of the node descriptions and the correctness of the K-type annotations. The review status is marked, and unqualified nodes are returned to the previous steps for correction.

[0061] (V) Anchor Point System Construction Sub-module The anchor point system construction submodule is responsible for constructing anchor points from teaching phrases and aligning the anchor points with Ki nodes, forming a three-layer mapping structure of "teaching language layer - independent anchor point layer - bottom Ki node layer".

[0062] Anchor point candidate generation steps: After the clustering and deduplication step, semantic embeddings are generated using BGE-M3 on all teaching phrases collected by the question parsing submodule. Semantically similar phrases are grouped into the same topic using a clustering algorithm. Each cluster corresponds to a teaching topic and is named, forming anchor point candidates. The anchor point identifier, anchor point name, and original phrase set are recorded. For example, multiple semantically similar teaching phrases such as "find the maximum and minimum values ​​of a quadratic function on a closed interval," "quadratic function maximum and minimum value problem," and "maximum and minimum values ​​of a quadratic function" are grouped into the same cluster and named the anchor point "quadratic function maximum and minimum value."

[0063] Anchor alignment steps: For each anchor candidate, firstly, the Top-K candidate Ki nodes are recalled by calculating the semantic embedding similarity using BGE-M3. Then, GLM-5.1 is called to accurately select the set of Ki nodes covered by the anchor from the candidate Ki nodes, forming a mapping from the anchor to the set of Ki nodes. The list of covered Ki node identifiers and the alignment confidence are recorded. For example, after the "quadratic function maximum / minimum value" anchor is aligned, it covers multiple Ki nodes such as K1 node "quadratic function concept", K2 node "vertex formula", K3 node "recipe operation", and K4 node "maximum / minimum value structure trigger".

[0064] Anchor points exist at an independent level in the knowledge graph of test points. They do not participate in the graph traversal of the question-generating engine, but only serve as the entry point for projecting user natural language input into the Ki node space. Anchor points and the set of Ki nodes they cover are written into the anchor point library, completing the construction of the three-layer mapping structure.

[0065] IV. Layered Question Engine Stage The tiered question generation engine module is based on five databases established by the knowledge system construction module. It receives the subject topics and target cognitive levels input by teachers, processes them in the following six steps, and outputs high-quality questions that meet the requirements of the target cognitive level.

[0066] Step 1, Anchor Point Analysis The system receives subject-specific text and target Bloom level input from teachers. It then performs exact name matching and fuzzy alias matching on the subject-specific text, retrieves matching anchors from the anchor library, and obtains the set of Ki nodes covered by the anchor. Subsequently, it filters the set of Ki nodes according to the target Bloom level and retains the Ki nodes that match the target level.

[0067] Taking this embodiment as an example: the teacher inputs the subject topic "quadratic function" and the target level "application layer". The system uses fuzzy matching to find the anchor point "quadratic function maximum and minimum value", obtains the covered Ki node set ["K1_quadratic function concept", "K2_vertex formula", "K3_completion operation"], filters by application layer and passes all of them, and proceeds to the second step.

[0068] The second step is knowledge subgraph retrieval. Using the filtered Ki nodes as seed nodes, a breadth-first search expansion is performed on the knowledge graph of the test points. The upper limit of the number of nodes in the knowledge subgraph is determined according to the target Bloom level. During expansion, only neighboring nodes that meet the constraints of the target level are included. The correspondence between each target level and the required K types is as follows: memory level corresponds to K1; understanding level corresponds to K1 and K2; application level corresponds to K1, K2, and K3; analysis level corresponds to K1, K2, K3, and K4; evaluation level corresponds to K1 to K5; and creation level corresponds to K1 to K6. The higher the target cognitive level, the more cross-cognitive level dependent nodes are included in the determined knowledge subgraph.

[0069] After expansion, the knowledge subgraph is verified to ensure it contains the necessary K types for the target level. If any are missing, the corresponding nodes are automatically added from the test point knowledge graph. Taking the application layer as the target: After BFS expansion, the knowledge subgraph contains 4 nodes and 3 edges: K1 node "Quadratic function concept", K2 node "Vertex formula", K3 node "Complete operation" and K4 node "Extremum structure trigger". Since the application layer only needs K1 to K3, node K4 is filtered out, resulting in a knowledge subgraph with 3 nodes and 2 edges.

[0070] The third step is to generate question prompts. The planning unit integrates the type, description, relationship information of each node in the knowledge subgraph, as well as the question skeleton, to generate structured question prompts. These prompts are grouped into K-type groups to organize the knowledge context, including a list of target knowledge points, multi-dimensional constraints, and quality requirements.

[0071] Step 4: Generating candidate questions The question generation unit calls the deepseek-v4-pro large language model to generate N candidate questions in parallel based on structured question prompts. Each candidate question contains the question text, reference answer and solution outline.

[0072] Step 5: Multi-dimensional quality assessment The evaluation unit calls GLM-5.1 to independently score each candidate topic from the following four dimensions, with each dimension having a maximum score of 10 points: Knowledge Point Coverage (S1) measures the completeness of the questions' coverage of the core Ki nodes of the target knowledge subgraph. The scoring criteria are as follows: 9 to 10 points indicate complete coverage of all target knowledge points with natural and in-depth connections between them; 7 to 8 points indicate coverage of major knowledge points (≥80%) with no omissions of core knowledge points; 5 to 6 points indicate coverage of some knowledge points (60% to 80%); 3 to 4 points indicate insufficient coverage (40% to 60%); and 0 to 2 points indicate severely insufficient coverage (less than 40%) or deviation from the target knowledge points.

[0073] Bloom Level Accuracy (S2) measures the degree of consistency between the cognitive level actually tested by the question and the target level. The scoring criteria are as follows: 9 to 10 points indicate complete alignment with the target level; 7 to 8 points indicate basic alignment with minor deviations; 5 to 6 points indicate significant level confusion; 3 to 4 points indicate deviation from the target level by one level; and 0 to 2 points indicate a significant deviation from the target level by more than two levels.

[0074] Clarity of Question (S3) measures whether the question is clearly and unambiguously worded. The scoring criteria are as follows: 9 to 10 points indicate that the wording is clear, accurate, logically sound, and unambiguous; 7 to 8 points indicate that it is basically clear, with minor inaccuracies that do not affect understanding; 5 to 6 points indicate that the wording is somewhat vague; 3 to 4 points indicate that there is significant ambiguity; and 0 to 2 points indicate that the question is incomprehensible.

[0075] Answer Uniqueness (S4) measures whether the answer to a question is unique and whether the solution process is rigorous. The scoring criteria are as follows: 9 to 10 points indicate that the answer is completely unique and the solution process is rigorous and without loopholes; 7 to 8 points indicate that the answer is basically unique; 5 to 6 points indicate that additional conditions are needed to completely determine the answer; 3 to 4 points indicate that there are multiple possible answers; 0 to 2 points indicate that the answer is not unique or has serious logical loopholes.

[0076] The overall score is calculated using the following formula: S = ω1×S1+ ω2×S2+ ω3×S3+ ω4×S4 Where S is the comprehensive evaluation score of the candidate question; S1 is the knowledge point coverage score; S2 is the Bloom level accuracy score; S3 is the question clarity score; S4 is the answer uniqueness score; ω1, ω2, ω3, and ω4 are the weight coefficients corresponding to the above four dimensions, with values ​​of 0.3, 0.3, 0.2, and 0.2 respectively, and ω1+ω2+ω3+ω4=1.

[0077] Questions with an overall score of 6.5 or higher and no single dimension score of 5.0 or higher are considered passing. The question with the highest overall score among the passing questions is selected as the best candidate question. Questions with an overall score of 8.5 or higher are marked as excellent, those between 7.0 and 8.5 are marked as passing, and those below 7.0 or below 5.0 in any dimension are marked as failing. The evaluation unit outputs scores for each dimension, comments for each dimension, an overall score, a general comment, suggestions for improvement, and a quality level.

[0078] Step 6: Reflect and Optimize If the overall score of the best candidate question is below 8.0, the correction unit extracts the dimension with the lowest score and its improvement suggestions from each dimension, calls GLM-5.1 to optimize the question specifically, generates an improved version, and re-executes step five for evaluation on the improved version. If the overall score of the improved version is higher than that of the original version, the improved version is adopted; otherwise, the original version is retained. The above process iterates for a maximum of two rounds, stopping when the score reaches 8.0 or the number of iterations is exhausted, and outputting the final question and a complete record of the optimization process. If the overall score of the best candidate question has reached or exceeded 8.0, the reflection and optimization step is skipped, and the final question is output directly.

[0079] Teacher feedback Teachers can provide written feedback on the generated problems. The system will submit the feedback along with the original problem to deepseek-v4-pro for targeted modifications and output a revised version, supporting closed-loop optimization through human-computer collaboration. For example, if a teacher suggests "the problem should be more difficult and require case-by-case discussion," the system will convert this feedback into a modification instruction, generating an improved problem that includes case-by-case discussion for both "vertex within the interval" and "vertex outside the interval."

[0080] Through the above embodiments, the present invention achieves the following technical effects: First, by defining the strict engineering boundaries of K4 (path routing) and K5 (execution trajectory), the problem of insufficient cognitive level granularity in traditional knowledge graphs is solved, enabling knowledge units to accurately correspond to different cognitive depths of Bloom classification, and avoiding the shift in question level caused by K4 / K5 confusion.

[0081] Second, by using the ablation verification mechanism, the judgment of the necessity of knowledge units is changed from relying on subjective annotation rules to objective judgment of problem-solving failures, which effectively filters out unnecessary annotations and improves the purity of nodes and the reliability of relation weights in the knowledge graph of test points.

[0082] Third, through the design of an independent anchor layer, a two-stage mapping from the teacher's natural teaching language to the underlying Ki node set is realized, enabling non-technical teachers to directly drive accurate knowledge subgraph retrieval with natural teaching language, which significantly reduces the threshold for using the system.

[0083] Fourth, through a four-dimensional quantitative scoring and multi-round correction and iteration mechanism, it ensures that the output questions meet the quality requirements in four dimensions: knowledge coverage, Bloom level accuracy, question clarity, and answer uniqueness. This avoids the one-sidedness caused by relying on a single indicator for evaluation and realizes the measurability and optimization of question quality.

[0084] Fifth, through a human-computer collaboration mechanism driven by teacher feedback, the teaching experience of professional teachers is organically combined with the generation capabilities of the large language model, supporting targeted adjustments to the output questions and enhancing the system's practicality and flexibility in real teaching scenarios.

[0085] To further illustrate the technical effectiveness of the proposed system and method, the following detailed explanation is provided in conjunction with specific experimental data and comparative tests.

[0086] 1. Experimental Design and Implementation Examples The experiment included two control groups: Example 1 (Full System): This example uses the Bloom's hierarchy of cognitive levels-based question generation method and system with controllable difficulty. Specifically, the question generation task uses the deepseek-v4-pro model (temperature value set to 0.7 to ensure question innovation), and the critique task uses the GLM-5.1 model (temperature value set to 0.1 to ensure rigorous logical review).

[0087] Comparative group (large model direct output llm_direct): Only the basic large language model is used to receive prompt words and directly output the question results. The basic large language model adopts the deepseek-v4-pro model (temperature value set to 0.7), without going through the reflection and independent evaluation module.

[0088] 2. Evaluation and Assessment System This experiment constructed an evaluation system that comprehensively and objectively quantifies the quality of the generated questions using the following five core indicators: Target Achievement Rate: A core metric for evaluating whether questions fully meet the initial design intent. It requires the system to simultaneously satisfy three dimensions—topic matching, question type matching, and Bloom level matching—provided the system generates questions successfully. The calculation formula is: Overall Target Achievement Rate = Successful Generation × (Topic Matching × Question Type Matching × Bloom Level Matching).

[0089] Solvability: This assesses whether the problem has structural flaws such as ambiguity or missing conditions. This application introduces a 5-round CoT (Co-Like) majority voting mechanism to automatically solve and verify the problem, ensuring its objective solvability.

[0090] Usability: From the perspective of teaching practice, comprehensively evaluate the readability of the questions, their applicability to teaching scenarios, and their learning support effect.

[0091] Batch diversity (MTLD and entropy): As an auxiliary monitoring metric, it is used to quantify the lexical richness (MTLD) and information entropy (Entropy) during batch generation, aiming to monitor whether the system has a tendency to use templates and lexical depletion problems.

[0092] 3. Experimental Results Table 1

[0093] Experimental results show that the system and method proposed in this invention outperform the basic model direct output scheme in several key indicators. Compared with the latter, the system exhibits higher stability in terms of overall pass rate (86.60%) and solvability (84.50%), indicating that the generated questions have significant advantages in logical consistency and task completion. This performance improvement may be related to the low-temperature criticism mechanism introduced in the system, which constrains and corrects the question structure and reasoning chain in the post-generation stage, thereby reducing the generation of inconsistencies or unsolvable problems. At the same time, the improvement in usability score (8.26) further illustrates that the generated results of this system have greater potential for direct application in teaching scenarios. In terms of batch diversity indicators (MTLD and entropy), the system shows a relatively reasonable convergence trend, indicating that the multi-model collaboration mechanism effectively limits unconstrained divergent generation while maintaining a certain degree of text diversity, achieving a more stable balance between innovation and standardization in question quality.

[0094] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for generating questions with controllable difficulty based on Bloom's hierarchy of cognitive levels, characterized in that, Includes the following steps: Define a classification system of six cognitive level knowledge units corresponding to the six cognitive levels of Bloom's Taxonomy of Cognition; Based on a large language model, the original questions and their reasoning trajectories are annotated in multiple rounds to automatically extract knowledge units at each cognitive level. The extracted knowledge units are then ablated and verified to see if removing each knowledge unit would cause the corresponding question to fail. Necessary knowledge units that cause the question to fail are retained while unnecessary annotations are filtered out. Semantic embedding vectors and hierarchical clustering algorithms are used to deduplicatize and merge the knowledge units. The deduplicated knowledge units are then used as nodes to construct a knowledge graph of test points with cognitive level annotations. Teaching phrases are automatically extracted from the question processing results. These phrases are then clustered using semantic clustering, with each cluster corresponding to a teaching theme and named to form anchor point candidates. The similarity between the anchor point candidates and knowledge unit nodes is calculated using semantic embedding similarity to obtain candidate nodes. Based on a large language model, the candidate nodes are refined to align the anchor point candidates with the knowledge unit node set. A three-layer mapping structure is established: a teaching language layer, an independent anchor point layer, and a bottom-level knowledge unit node layer. The independent anchor point layer serves as an intermediate mapping layer independent of the test point knowledge graph. The mapping from the teaching language layer to the independent anchor point layer is determined by semantically encoding the input subject theme and calculating its embedding similarity with each anchor point. The mapping from the independent anchor point layer to the bottom-level knowledge unit node layer is pre-calculated and stored during the anchor point alignment stage. The system receives a subject topic and a target Bloom's hierarchy of knowledge. Through a three-layer mapping structure, the subject topic is projected onto the corresponding set of knowledge unit nodes via an independent anchor layer. The set of knowledge unit nodes is then refined and filtered according to the target Bloom's hierarchy, retaining only those nodes that match the target hierarchy. Using these filtered knowledge unit nodes as seed nodes, a knowledge subgraph containing all dependent nodes is determined based on the target hierarchy. Structured question prompts are generated based on the knowledge subgraph, and a large language model is used to generate candidate questions. Finally, the candidate questions are comprehensively scored using preset multi-dimensional indicators. Through multiple rounds of iterative cycles of generation, evaluation, and correction, questions that meet the target cognitive level are output.

2. The method according to claim 1, characterized in that, The six cognitive levels of knowledge unit classification system include: K1 conceptual knowledge corresponding to the memory level, K2 formula or theorem knowledge corresponding to the understanding level, K3 operational knowledge corresponding to the application level, K4 structural triggering knowledge corresponding to the analysis level, K5 reasoning chain knowledge corresponding to the evaluation level, and K6 combination knowledge corresponding to the creation level; among them, K4 is path routing knowledge, representing structural triggering and path selection in problem-solving; K5 is execution trajectory knowledge, representing reasoning steps and execution processes in problem-solving.

3. The method according to claim 1, characterized in that, The multi-round hierarchical annotation includes five rounds of hierarchical annotation. The five rounds of hierarchical annotation are respectively aimed at different cognitive level attributes of knowledge units, and the same question and its reasoning trajectory are annotated in a progressive manner in rounds. The output results of each round serve as the reference input for the next round of annotation.

4. The method according to any one of claims 1 to 3, characterized in that, The ablation verification includes the following operations: for the set of knowledge units obtained through multiple rounds of hierarchical annotation, one knowledge unit is taken out one by one to form a verification set for the missing knowledge unit; the verification set is substituted into the problem-solving process of the corresponding question for verification; if the problem is solved successfully, the knowledge unit is determined to be an unnecessary annotation and is filtered out; if the problem is solved unsuccessfully, the knowledge unit is determined to be a necessary node and is retained in the test point knowledge graph.

5. The method according to any one of claims 1 to 3, characterized in that, The knowledge graph edges in the test point knowledge graph include four types: dependency, activation, combination, and prerequisite. Among them, dependency means that the dependent node A is a necessary condition for the existence of the dependent node B; activation means that when a node is triggered, it can activate other related nodes to participate in problem-solving reasoning; combination means that multiple independent nodes construct a new composite test point; prerequisite means that the learning sequence constraint is set according to the teaching syllabus and cognitive laws, requiring that the learning of node A be arranged earlier than that of node B.

6. The method according to any one of claims 1 to 3, characterized in that, The process of determining a knowledge subgraph containing all dependent nodes based on the target cognitive level includes the following: the higher the target cognitive level, the more cross-cognitive level dependent nodes are contained in the determined knowledge subgraph.

7. The method according to any one of claims 1 to 3, characterized in that, The multi-round generation-evaluation-correction iterative cycle includes: when the comprehensive score of the candidate topic is lower than the preset threshold, the scoring results and the analysis of deficiencies in each dimension are re-inputted into the large language model as reflection information to drive the large language model to make targeted corrections to the candidate topics; the corrected topics are scored again; and the process continues until the comprehensive score reaches the threshold requirement or the preset maximum number of iterations is reached.

8. The method according to any one of claims 1 to 3, characterized in that, After outputting questions that meet the target cognitive level, the process also includes: receiving textual feedback from teachers on the output questions; parsing the textual feedback into modification instructions for specific questions; inputting the modification instructions into a large language model to drive the large language model to make targeted modifications to the questions; and outputting the modified questions for teachers to further confirm.

9. A question-generating system with controllable difficulty based on Bloom's hierarchy of knowledge, characterized in that, include: The knowledge system construction module is used to define a classification system of six cognitive level knowledge units that correspond to the six cognitive levels of Bloom's Taxonomy of Cognition. Based on a large language model, the original questions and their reasoning trajectories are annotated in multiple rounds to automatically extract knowledge units at each cognitive level. The extracted knowledge units are ablation-verified to retain necessary knowledge units and filter out unnecessary annotations. The knowledge units are deduplicated and merged using semantic embedding vectors and hierarchical clustering algorithms. The deduplicated knowledge units are used as nodes to construct a knowledge graph of test points with cognitive level annotations. The three-layer mapping anchor system construction module is used to automatically extract teaching phrases from the question processing results. It clusters these teaching phrases using semantic clustering, with each cluster corresponding to a teaching theme and named accordingly, forming anchor candidate pairs. The similarity between the anchor candidate pairs and knowledge unit nodes is calculated using semantic embedding similarity to obtain candidate nodes. Based on a large language model, the candidate nodes are refined to align the anchor candidate pairs with the knowledge unit node set. A three-layer mapping structure is established: a teaching language layer, an independent anchor layer, and a bottom-level knowledge unit node layer. The independent anchor layer serves as an intermediate mapping layer independent of the test point knowledge graph. The mapping from the teaching language layer to the independent anchor layer is determined by semantically encoding the input subject theme and calculating its embedding similarity with each anchor point. The mapping from the independent anchor layer to the bottom-level knowledge unit node layer is pre-calculated and stored during the anchor alignment stage. The hierarchical question generation engine module receives subject topics and target Bloom's hierarchy of knowledge (HH) levels. Through a three-layer mapping structure, it projects the subject topics onto corresponding sets of knowledge unit nodes via independent anchor layers. Then, it filters and selects knowledge unit nodes that match the target HH level, retaining only those that do. Using these filtered knowledge unit nodes as seed nodes, it determines a knowledge subgraph containing all dependent nodes based on the target HH level. Based on this knowledge subgraph, it generates structured question prompts and calls a large language model to generate candidate questions. Finally, it performs a comprehensive scoring of the candidate questions using preset multi-dimensional indicators. Through multiple rounds of iterative cycles of generation, evaluation, and correction, questions that meet the target cognitive level are output.

Citation Information

Patent Citations

  • Intelligent question answering method and system for online education based on knowledge graph

    CN121835899A

  • Multi-dimensional examination question generation system based on knowledge graph and large model

    CN122287819A