Intelligent labeling method and system for test question knowledge system based on thinking tree enhancement

Abstract

The invention belongs to the technical field of education artificial intelligence and deep learning model optimization, and discloses a thinking tree enhancement-based test question knowledge system intelligent labeling method and system.The thinking tree enhancement-based test question knowledge system intelligent labeling method comprises the steps of collecting question data by introducing a thinking chain enhanced data generation mechanism, and generating a high-quality training sample; and a control mechanism is set, a question generation process is optimized, and the generated question is ensured to accord with teaching specifications in the aspects of knowledge point coverage, grade adaptability, difficulty matching and the like. Through a text encoder based on comparative learning optimization, the model can accurately carry out semantic alignment on questions and knowledge point labels, and label path information is fused through a double-coding mechanism, so that the hierarchical relationship between the labels and the capture capability of semantic dependence are improved. Finally, the constructed multi-label prediction model significantly improves the accuracy and generalization ability of knowledge point labeling, and can provide accurate support for personalized learning recommendation and teaching resource allocation.

Description

Technical Field

[0001] This invention belongs to the field of educational artificial intelligence and deep learning model optimization technology, and in particular relates to an intelligent annotation method and system for test knowledge system based on mind tree enhancement. Background Technology

[0002] As a fundamental module of intelligent education systems, knowledge point annotation technology faces the core challenge of accurately identifying the knowledge points corresponding to questions while maintaining structural consistency among tags in large-scale, multi-granularity question bank scenarios. While existing automatic annotation methods based on pre-trained language models have advantages in semantic understanding, these general-purpose models struggle to fully capture the deep semantic features of educational questions, which often contain implicit logic, complex reasoning, and knowledge transfer clues. This results in insufficient performance in fine-grained knowledge point differentiation and low-frequency tag recognition, thus affecting the overall annotation effect and generalization ability.

[0003] The long-tail distribution of knowledge tags makes models more susceptible to being dominated by high-frequency tags during training, while failing to adequately learn the boundaries of low-frequency tags, leading to significant biases and imbalances in prediction results. While some studies have attempted to alleviate this problem by generating extended samples using large language models, existing generation mechanisms lack effective control over teaching logic, cognitive load, and grade-level suitability. The generated content often fails to meet the actual needs of educational applications, frequently resulting in mismatches between the question and the knowledge points or insufficient pedagogical value.

[0004] Current data augmentation methods generally lack explicit alignment modeling between generated samples and target knowledge labels, leading to instability in the labeling of generated questions and diminishing the value of augmented samples in model training. Therefore, how to construct deep semantic parsing models for educational scenarios, improve the ability to model long-tail knowledge points, and introduce knowledge label consistency constraints in data augmentation are key issues that future knowledge point annotation technologies urgently need to address. Summary of the Invention

[0005] To address the problems existing in the prior art, this invention provides an intelligent annotation method and system for test knowledge systems based on mind tree enhancement.

[0006] This invention is implemented as follows: A method for intelligent annotation of test question knowledge systems based on mind tree enhancement includes:

[0007] Step 1: Construct a chain-thinking TOT-guided question generation system:

[0008] The design incorporates multi-layered Prompt templates based on pre-defined knowledge points, grade levels, question types, and difficulty levels to guide the large language model in generating test texts with standardized teaching requirements. By combining mechanisms such as knowledge point coverage assessment, question difficulty prediction, grade level suitability verification, diversity discrimination, and language logic review, a chain-like control process is constructed to ensure that the generated questions are structurally complete, content is standardized, and labels are accurately matched, thereby forming a high-quality training sample set.

[0009] Step 2: Expand and reconstruct the dataset for long-tail knowledge tags:

[0010] To address the issues of imbalanced label distribution and scarcity of low-frequency knowledge point samples in the original data, the TOT-based question generation system constructed in step one is used to enhance the training samples corresponding to low-frequency knowledge points as needed. Then, the diversity of the original dataset is increased through EDA enhancement strategies to further expand the samples of low-frequency labels. Finally, the original data, EDA-enhanced data, and TOT-generated samples are combined to construct a dataset with balanced label distribution and strong semantic consistency.

[0011] Step 3: Introduce a test item representation encoding model based on a contrastive learning mechanism:

[0012] A BGE-based encoder is used to semantically model the question text. Positive and negative question pairs are constructed by combining a contrastive learning strategy. By maximizing the vector proximity of questions with similar labels and minimizing the distance of questions with different labels, the discriminative ability of the embedding space is optimized. This enhances the model's ability to distinguish between questions with similar semantics but different labels, providing a stable representation basis for multi-label prediction.

[0013] Step 4: Construct a structure-aware multi-label knowledge point annotation model:

[0014] Based on the encoder representation, the semantic path and hierarchical structure between knowledge point tags are modeled. Combining test question information and tag path information, semantic alignment is achieved through dual-encoding semantic fusion. Finally, multiple knowledge point tags that conform to semantic association and structural hierarchy are output, realizing a multi-tag knowledge point annotation task with high accuracy and strong generalization ability.

[0015] Furthermore, step one includes:

[0016] Constructing the control input space for problem generation:

[0017] Let the total set of knowledge points be K = {k1, k2, ..., k}. L The set of grades is G = {g1, g2, ..., g}. J The question types are:

[0018] T = {t1, t2, ..., t} P The difficulty set is D = {d1, d2, ..., d}.Q Each problem generation task is defined by a control quaternion input:

[0019] x i =(K i ,g i ,t i ,d i )

[0020] in, g represents the set of knowledge points involved in the question. i ∈G represents the grade label, t i ∈T is the question type label, d i ∈D represents the difficulty level;

[0021] Based on the control input quadruple, the Prompt constructor is called:

[0022] P i =f prompt (K i ,g i ,t i ,d i )

[0023] This input space is used to construct structured natural language prompt templates and guide the large language model to initialize test texts that meet the conditions. The design of this input space ensures that the question generation process has knowledge point orientation, adaptability to teaching stages, question type constraints, and difficulty level classification.

[0024] Construct a tree-structured exploration mechanism driven by a closed loop of generation, evaluation, and feedback:

[0025] This invention proposes a tree-structured thinking problem exploration mechanism that integrates "generation-evaluation-feedback" closed-loop control to improve the controllability, stability, and goal alignment of educational large language models in teaching test question generation tasks. This mechanism regards the test question generation process as a step-by-step reasoning exploration process, constructs thinking branch paths in stages, and introduces automatic evaluation and strategic feedback at each step, thereby forming a structured multi-path generation representation.

[0026] Upon receiving the control input quadruple x i =(K i ,g i ,t i ,d i And construct the Prompt input P i The language model is invoked to generate initial candidate test questions. The process enters a hierarchical processing flow; this flow includes five processing levels, L0 to L4, which correspond to: knowledge point coverage check, difficulty level matching, grade suitability verification, expression diversity control, and language standardization and logical consistency review.

[0027] In any processing layer L j (j∈{0,1,2,3,4}), the system executes according to the following three-stage process:

[0028] Generation phase:

[0029] The system calls the language model based on the current question state. And a Prompt prompt to generate the next version of the test questions. Where r is the number of attempts at the current level;

[0030] Evaluation phase:

[0031] Call the evaluator module E j (·) For candidate questions Perform automatic evaluation across the specified dimensions and output an evaluation result vector:

[0032]

[0033] in Its dimensions and content vary depending on the processing level. For example, the knowledge point layer includes dimensional information such as "coverage", "error keywords", and "omission list".

[0034] Feedback and correction phase:

[0035] If the evaluation results do not meet the preset threshold (e.g., coverage is lower than θ), j If the feedback content is received, a targeted retry prompt will be automatically constructed.

[0036] This guides the language model to generate new exploration branches; each generation-evaluation-feedback constitutes a ToT branch path.

[0037] This process is executed iteratively within each level until the success condition is met or the maximum number of attempts R is reached. max All generation and correction attempts can be modeled as a branching tree within each layer:

[0038]

[0039] Each node records the result of one generation and evaluation; the final multi-level nested tree structure is defined as follows:

[0040]

[0041] Through this structure, each problem starts from the initial version. Starting from this point, through a multi-layered, progressive exploration of ToT (Total Time) and continuous refinement, a test version q that satisfies all constraints was finally obtained. final .

[0042] Furthermore, step two includes:

[0043] (2.1) Long-tail tag detection and low-frequency knowledge point recognition:

[0044] In the original dataset, each question corresponds to a set of knowledge point labels, denoted as label group C(q). This set may contain a single knowledge point or a combination of multiple knowledge points; let the total set of label groups be:

[0045] C = C1, C2, ..., C M

[0046] Each C m This represents a combination of labels; the original dataset can be formalized as:

[0047]

[0048] Where: q i Represents the test question text; C(q) i ) indicates the knowledge point tag group corresponding to the question; g i Indicates the grade level; t i Indicates the question type; d i Indicates the difficulty level;

[0049] Statistical analysis of each label group C m Number of samples in the original dataset:

[0050]

[0051] Set a long-tail threshold δ (e.g., if the number of samples is less than 15), and define the long-tail label group set as follows:

[0052]

[0053] Accurately identify label combinations with insufficient samples in the dataset and use them as the target set for subsequent data augmentation;

[0054] (2.2) Establish TOT-based directed sample generation rules and augment the data accordingly:

[0055] For each long-tail tag group It needs to be based on its existing quantity f(C) in the original dataset. m Using the threshold δ, calculate the number of samples that need to be added:

[0056] n gen (C m )=δ-f(C m )

[0057] To ensure that the generated samples conform to teaching principles, the allocation strategy of grade, difficulty, and question type is strictly controlled when using the ToT generation system:

[0058] The grade values ​​are taken from the existing grade set of this label group in the original dataset:

[0059] G m ={g i |C(q) i ) = C m}

[0060] During generation, samples are randomly drawn according to the original distribution to ensure that the grade distribution of the new samples is consistent with that of the original samples, thereby avoiding the offset of the dataset in the grade dimension;

[0061] To accurately reflect the actual distribution of educational question banks, the difficulty level is allocated according to a weighted ratio of "low:medium:high = 2:5:1".

[0062] p low :p mid :p high =2:5:1

[0063] During the generation process, the difficulty value d is randomly sampled in this proportion to maintain the balance and authenticity of the model training samples at different difficulties.

[0064] From the question type set T = {t1, t2, ..., t} P A question type is randomly selected evenly from the dataset to ensure that the generated samples cover a variety of question formats, thereby improving the diversity and generalization ability of the dataset.

[0065] Using the above control strategy, the ToT generation system in step one is invoked, with input parameters (C_m,g,t,d), to generate a set of augmented samples that meet the conditions:

[0066]

[0067] in To ultimately produce high-quality test questions through multiple rounds of evaluation and feedback optimization using a chain-thinking mechanism;

[0068] (2.3) Data augmentation based on EDA (Easy Data Augmentation) strategy:

[0069] In the generation of supplementary data for long-tail labels, in addition to using a TOT-based targeted sample generation system, the data is also augmented through an EDA strategy to increase the sample size of long-tail labels and enhance the diversity and generalization ability of the data.

[0070] Specific methods include synonym replacement, random insertion, random swapping, and random deletion. Synonym replacement replaces some words in a sentence with synonyms, thereby increasing vocabulary diversity and generating different sentence versions. Random insertion increases sentence complexity and diversity by randomly inserting synonyms or related words into the sentence. Random swapping changes the sentence structure while maintaining the original meaning by randomly swapping the positions of two words in the sentence. Random deletion removes certain words from the sentence with a certain probability, reducing redundant information and simulating noisy data. These methods enable each sentence in the original data to generate multiple enhanced versions. These versions not only cover the semantic information of the original data but also increase sentence diversity through various enhancement techniques, thus improving data quality.

[0071] In practice, two parameters, num_aug and alpha, were designed to influence the data augmentation effect. By setting num_aug, we control the number of augmented samples generated for each original sentence; for example, when num_aug = 1, each original sentence will generate one augmented sample. The alpha parameter determines the percentage of words modified in each sentence; we set alpha to 0.1, which means that approximately 10% of the words will be modified, thereby further improving the diversity and quality of the dataset. The final augmented dataset D is obtained. EDA ;

[0072] (2.4) Dataset reconstruction and consistency verification:

[0073] Augmentation of sample set D for generated long-tailed label groups gen Manual sampling is conducted, and the sampling process includes randomly selecting a portion of samples from the enhanced sample set. Reviewers then use the samples based on label group C. m The definition of knowledge points, for the extracted test questions The correspondence between the samples and the label groups is manually determined; samples that are determined to be non-compliant during the sampling process are removed from the augmented sample set, and the remaining samples constitute the sample set D that passed the sampling. gen,pass Compare this set with the original dataset D orig Merge to obtain a new dataset

[0074] D new =D orig ∪D gen,pass .

[0075] Furthermore, step three includes:

[0076] (3.1) Constructing contrastive learning sample pairs:

[0077] Specifically, for each sample question, its corresponding set of knowledge point tags is extracted, and the training set D is traversed. newCompare the label sets of other samples:

[0078] When the label sets of two questions are completely identical, they are determined to be a positive sample pair;

[0079] When the label sets of two questions have no overlap, they are determined to be a negative sample pair.

[0080] If the label sets partially overlap, they will not participate in the construction of sample pairs;

[0081] To ensure the balance of sample distribution and training efficiency, this invention selects a maximum of several positive and negative sample pairs for each query sample: the number of positive samples does not exceed 15, and the number of negative samples does not exceed 5. If a sample cannot form a positive sample pair, it is removed from the training data, resulting in the final positive and negative sample pair dataset D. 15-5 ;

[0082] (3.2) Hard Negative Sample Mining: In constructing the initial positive and negative sample pair dataset D 15-5 Based on this, the present invention further introduces a difficult negative sample mining strategy;

[0083] The specific process includes:

[0084] Embedded computation: All test item samples are vectorized and embedded using a pre-trained text representation model to form a unified vector retrieval space;

[0085] Similar sample retrieval: For each query question, retrieve question samples with inconsistent labels within a limited distance range, sort them from high to low vector similarity, and select a group of semantically similar label conflict samples;

[0086] Construction of difficult negative samples: A fixed number of high-similarity unlabeled samples are selected from the above screening results and added to the original training structure D as difficult negative samples. new-hard ;

[0087] (3.3) Training and optimization objectives of the contrastive learning model:

[0088] Based on the construction of a contrastive training sample set containing difficult negative samples, this invention further designs a contrastive learning training mechanism, which effectively improves the semantic representation ability of test questions through a target optimization strategy in the embedding space.

[0089] The specific training process is as follows:

[0090] Model structure initialization:

[0091] This invention employs a dual-tower text encoding structure based on the publicly released Chinese semantic vector model bge-large-zh-v1.5 to learn the embedding representation of test text; wherein the query end and the comparison sample end share encoder weights; each input test text is converted into a fixed-dimensional semantic embedding vector by the encoder;

[0092] Input sample composition:

[0093] The training inputs are organized in the form of triples, and the dataset comes from D. new-hard ,include:

[0094] Query test questions;

[0095] A positive sample item is one with the same label.

[0096] One or more hard negative sample items, i.e., items with no overlap in labels but similar semantics;

[0097] Each group of training samples constitutes a positive-negative comparison unit, used to compare the optimization of the objective function;

[0098] Training objective function:

[0099] This invention employs a contrast loss function based on temperature scaling, and its core objective is:

[0100] Maximize the similarity between the query embedding and the positive sample embedding;

[0101] Minimize the similarity between the query embedding and the negative sample embedding;

[0102] Furthermore, during training, all embedded vectors are normalized to maintain the dominance of directional information and improve the sensitivity of similarity discrimination.

[0103] By introducing a temperature coefficient to control the granularity of similarity distinction, the stability and generalization ability of the optimization are enhanced.

[0104] Training parameters and process control:

[0105] Model training employs a mini-batch iterative approach, with 10 training rounds and a low learning rate to accommodate fine-tuning scenarios, further ensuring model convergence stability. In each training round, the contrastive learning module continuously optimizes the structural boundaries of the embedding space, gradually improving the model's ability to model subtle semantic differences.

[0106] Task adaptation instructions:

[0107] To enhance the model's generalization ability in specific tasks, this invention introduces domain task prompts as additional instruction inputs during the training process. These prompts guide the model to focus on understanding and modeling "mathematical problems with similar query content," thereby achieving targeted alignment with educational scenarios.

[0108] Through the above training mechanism, the resulting model bge-large-zh-TOT-15-5 has a stronger semantic distribution expression ability in the text embedding space. In particular, it can effectively identify and distinguish mathematical test questions with similar expressions but different knowledge points, providing a more robust representation foundation for subsequent knowledge point prediction models.

[0109] Furthermore, step four includes:

[0110] (4.1) Model structure design:

[0111] This invention proposes a structure-aware multi-label text classification model. The overall architecture is based on a hierarchical perceptual network, optimized by a contrastive learning semantic encoder, forming a complete semantic structure hierarchical perceptual framework specifically designed for automatic multi-label annotation tasks from test questions to knowledge point labels. This architecture features the following modular structure:

[0112] Text encoding module (optimized core):

[0113] This module converts test question text into semantic vector representations. It is based on the Chinese semantic model bge-large-zh-TOT-15-5, optimized and trained using a contrastive learning method on the Chinese TOT augmented corpus. Structurally, this model employs a pre-trained Transformer framework, which, after optimization using the method of this invention, possesses stronger semantic discriminative capabilities and can effectively capture the semantic features of test questions. The model integrates a dedicated word segmenter and encoder. The input text is first encoded by the word segmenter, generating input tensors such as token sequences, attention masks, and positional information. These tensors are then input to the encoder module, which outputs a 1024-dimensional semantic embedding representation of the test questions, serving as input to the subsequent structural modeling module. The tag embedding layer...

[0114] Feature alignment and label copying module

[0115] This module converts the semantic vectors of test questions generated by the text encoding module into the label node input format for adaptive graph structure modeling, ensuring that the semantic features of the test questions can be aligned and semantically propagated one-to-one with multiple knowledge point labels in the label space; specifically, the test question vector output by the text encoding module is h. t ∈R d Where d = 1024 is the text embedding dimension; this module first copies the vector N times along the label dimension to construct the label node input matrix H. t∈R N×d , where N is the number of knowledge point tags; this copying operation ensures that the semantic information of the test questions can be evenly transmitted to each tag node, establishing a semantic alignment basis between the test questions and tags;

[0116] To further enhance the diversity and discriminative power of tag node representations, this module introduces two layers of linear transformations:

[0117] Feature projection layer: Performs a linear mapping on each copied label vector, projecting it from the original dimension d to the intermediate dimension d' required by the structure-aware module, thereby enhancing its expressive power in the label space;

[0118] Tag Dimension Adjustment Layer: The embedding of each tag node is reconstructed across dimensions to give it the tensor shape required to adapt to the structure propagation mechanism (as shown in the attention network), ensuring the coherence of information transmission in the structural path between tags;

[0119] The entire module is equipped with a Dropout layer (dropout rate adjustable, default is 0.1) to enhance the robustness and generalization ability of the training process; the output label alignment feature tensor will be used as input to the structural modeling module (such as HiTIN) to construct the label semantic graph, realize the interaction propagation between labels and semantic enhancement;

[0120] Structural Modeling Module

[0121] This module is used to model the structural information between tags, explicitly introducing hierarchical paths and semantic dependencies in the knowledge point tag system, thereby improving the model's ability to perceive and represent tag structures. During implementation, this module receives the tag semantic tensor H output by the feature alignment module. t ∈R B×N×d′ Where B is the batch size, N is the number of labels, and d′ is the aligned feature dimension; this tensor is input into the structural modeling network, where the following key steps are performed:

[0122] Label Structure Graph Construction: This module aims to construct a graph representation that explicitly expresses the structural relationships between labels, guiding the subsequent structure-aware propagation process. The system first parses externally defined teaching knowledge system files (such as syllabi or knowledge point tree structures) and automatically identifies the parent-child relationships between each knowledge point label. Each label is mapped to a node in the graph, and directed connections are established based on the hierarchical logic of "parent label pointing to child label," thus forming a topologically ordered label structure graph. During this process, the system simultaneously introduces a label transition probability file as a structural prior, where the transition probabilities between label pairs are annotated in dictionary form. However, to simplify the graph construction process and improve the model's adaptability to the structure, in the current implementation, only the connection relationship identifiers provided by this file are used, with all valid connections uniformly assigned a value of 1, constructing a binary adjacency matrix as the input structure for the graph neural network. This approach allows the model to automatically learn the propagation strength and semantic relationships between labels through the structure-aware mechanism during training, without relying on precise domain priors.

[0123] Structure-aware information propagation: To model the topological dependencies between labels, the system organizes the connections between labels into a multi-layered encoding tree structure, and implements multi-layer embedding propagation on the tree structure based on a graph neural network mechanism; each layer of propagation follows the following computational process:

[0124] Neighbor feature aggregation

[0125] For the node representation of the l-th layer Its upper-level representation is through the adjacency matrix A (l) Implemented by performing sparse multiplication with node features:

[0126]

[0127] in:

[0128] This represents the adjacency matrix from layer l to layer l+1 (automatically constructed from the coding tree);

[0129] MLP (l) It is a multilayer perceptron of the l-th layer;

[0130] σ(·) represents the activation function (ReLU);

[0131] N l This represents the number of nodes in the l-th layer;

[0132] The intermediate representation of the upper-level nodes after aggregation

[0133] Batch normalization and nonlinear activation

[0134] After aggregation, the outputs of each layer undergo Batch Normalization and non-linear activation to stabilize the training process.

[0135]

[0136] Hierarchical feature fusion and label pooling

[0137] The final output label embedding tensor Z∈R B×d′ It integrates the semantic information of the test questions and the structural information of the labels, which serve as the input for the subsequent classification module;

[0138]

[0139] Multi-label classification decision module.

[0140] Furthermore, the multi-label classification decision module:

[0141] This module is responsible for converting the structure-enhanced label embedding representation into the final multi-label prediction result; the input is the label-level logits tensor Z output by the model structure modeling module:

[0142] To achieve multi-label classification tasks, the system adopts a label-by-label independent activation mechanism, which converts the predicted score of each label dimension into a probabilistic form by performing a Sigmoid mapping.

[0143]

[0144] Subsequently, the system sets a fixed threshold τ∈(0,1) and performs a probability judgment on each label to determine whether to activate the label (default threshold τ=0.5):

[0145] if else 0

[0146] The final output is:

[0147] Binary label matrix This represents the set of predicted labels for each sample.

[0148] Another objective of this invention is to provide an intelligent annotation system for test question knowledge systems based on mind tree enhancement, comprising:

[0149] The data augmentation module receives test question text data and constructs high-quality training samples to support the training of the semantic coding model. This module first performs multiple rounds of semantic reconstruction on the original test question data based on the tree-like thinking chain data augmentation mechanism proposed in this invention, generating pseudo samples with clear logical structure and explicit thinking path, thereby enhancing the training semantic coverage of the original data. At the same time, it further augments the low-frequency label samples based on the EDA strategy.

[0150] The semantic contrast learning module, based on data augmentation, introduces Chinese semantic contrast learning methods to perform positive and negative contrast modeling on the generated samples, and trains a BGE Chinese semantic encoding model with domain adaptability. This model adopts a pre-trained Transformer architecture and can output a high-quality test question semantic embedding representation of dimension d, providing a robust semantic foundation for subsequent structural modeling.

[0151] The tag structure graph construction module is used to parse the parent-child relationship of tags in the teaching knowledge system file and construct a tag graph with a hierarchical topology. The system maps each knowledge point tag to a node in the graph, constructs a set of directed edges based on the parent-child path, and generates an adjacency matrix as the basic skeleton for graph structure propagation.

[0152] The label semantic alignment module is used to perform hierarchical propagation of semantic information in the label structure graph. This module is based on Tree Isomorphism Network (TIN), which aggregates label neighbor features layer by layer in a bottom-up manner, updates node representations through multilayer perceptron and normalization operations, and fuses hierarchical path semantics between labels to enhance the structural perception capability of label representation.

[0153] The multi-label classification prediction module is used to perform independent label prediction tasks on the structure-enhanced label representation. The system outputs the prediction probability of each label through the sigmoid activation function and sets a fixed threshold (such as 0.5) to determine whether the label is activated, thus forming a multi-label classification result. Finally, the system outputs the set of labels involved in each test sample, realizing the automatic multi-label annotation function.

[0154] Another object of the present invention is to provide a computer device, the computer device including a memory and a processor, the memory storing a computer program, the computer program being executed by the processor causing the processor to perform the steps of the intelligent annotation method for the test question knowledge system based on mind tree enhancement.

[0155] Another object of the present invention is to provide a computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to perform the steps of the intelligent annotation method for the test question knowledge system based on mind tree enhancement.

[0156] Another objective of this invention is to provide an information data processing terminal, which is used to implement the intelligent annotation system for the test knowledge system based on mind tree enhancement.

[0157] Based on the above technical solutions and the technical problems solved, the advantages and positive effects of the technical solution to be protected by this invention are as follows:

[0158] First, this invention proposes a multi-label automatic annotation method that integrates generative data augmentation and contrastive learning semantic optimization. This method expands semantic training data through a prompt generation mechanism and improves label discrimination ability by combining a semantic representation optimization model.

[0159] The method includes: First, a semantic data augmentation mechanism based on thought chain prompts: This invention introduces thought chain prompts as a data expansion method, using a language model to enhance the original test sample through a "step-by-step thinking" generation mode, generating new samples that are more natural, semantically complete, and fit the label system, thereby expanding the semantic coverage of the training data. This method can generate high-quality pseudo-test questions with educational context without manually constructing label rules, improving the training diversity of the model and its ability to express the label space; Second, data augmentation through EDA strategies to increase the sample size and data diversity of long-tail labels. Specific methods include synonym replacement, random insertion, random swapping, and random deletion. Synonym replacement increases vocabulary diversity by replacing some words; random insertion inserts synonyms or related words into sentences to increase complexity and diversity; random swapping swaps the positions of words in sentences while maintaining the original meaning; random deletion deletes words with a certain probability to reduce redundant information and simulate noisy data. These methods generate multiple enhanced versions, covering the original semantics and improving data quality. Third, an alternative to the contrastive learning-optimized Chinese semantic representation model: To improve the accuracy of question semantic expression and label alignment, this invention trains a BGE model (e.g., bge-large-zh-TOT-15-5) on the Chinese TOT-enhanced corpus. This model strengthens the semantic discrimination ability between sentence vectors through a contrastive learning mechanism, enabling the model to more clearly cluster the question semantics corresponding to similar labels, effectively improving the accuracy and robustness of multi-label matching.

[0160] Traditional multi-label text classification models suffer from significant performance bottlenecks when handling educational labeling tasks with clear hierarchical structures and long-tail label distributions. This is particularly true in Chinese test data, where the labeling system often exhibits a typical "head-tail" imbalance: head labels are easy to learn due to abundant samples, while long-tail labels suffer from scarce training samples, making it difficult for the model to learn their effective semantic features, resulting in extremely low accuracy in recognizing low-frequency labels. While existing methods introduce structured label graph modeling mechanisms to some extent compensate for the semantic dependencies between labels, their encoders still rely on the standard BERT model. They fail to provide customized optimizations for Chinese educational scenarios and lack specialized modeling capabilities for long-tail labels, leading to semantic ambiguity and unclear label targeting when processing inference-based or weakly suggestive questions.

[0161] To address the aforementioned issues, this invention introduces innovative optimization strategies in the two core stages of data generation and semantic encoding. First, in the data generation stage, an augmented data construction method based on a "tree-like thought chain" is proposed. This guided generation process allows the model to simulate human analogy and reasoning, injecting more semantic cues and label hints to construct more diverse and semantically relevant training questions. It is worth emphasizing that the generated data content itself does not explicitly contain thought chain steps; rather, the thought chain approach is introduced during the generation process, making the samples more semantically structured and logical, significantly improving the coverage and discriminative ability of long-tail labels. Simultaneously, EDA strategies are used to further augment low-frequency label samples. At the text representation level, this invention replaces the traditional BERT model with the bge-large-zh-TOT-15-5 semantic encoding model, finely tuned and optimized on Chinese education-related TOT data. While maintaining the model's lightweight nature, this significantly enhances its adaptability to question language styles and its ability to capture the semantic structure of labels. Through this dual optimization path of "generative enhancement + representation enhancement", this invention effectively alleviates the problem of low accuracy in long-tail label recognition in multi-label classification tasks, significantly improves the overall label prediction accuracy and the model's structural perception capability, and fully demonstrates its technological leadership and engineering adaptability in the field of educational knowledge annotation.

[0162] In this invention, the TOT data augmentation mechanism serves as a crucial supporting technology for improving data quality and the relevance of problem generation, permeating all stages of problem construction and evaluation. This mechanism not only generates more discriminative and diverse mathematical problems but also effectively alleviates the learning obstacles caused by the sparsity of training data under long-tailed labels. Its specific applications are reflected in the following aspects:

[0163] (1) Task construction mechanism based on expert examples: This invention introduces structured reference examples of real questions in the generation of prompts, and constructs a Prompt style template with teaching guidance significance, guiding the model to follow the question-setting path of human experts in the process of generating new questions. By simulating the "thinking chain" of human question-setting, a systematic transfer from question setting objectives, language style, structural organization to knowledge point combination methods is achieved, which effectively improves the rationality and consistency of the question construction process.

[0164] (2) Multi-round prompt generation mechanism integrating contextual feedback: This invention designs an automated assessment module targeting dimensions such as "knowledge point coverage," "difficulty adaptation," and "grade-level expression standardization," and dynamically generates corrective prompts based on the assessment results, constructing an iterative "thinking chain" adjustment path similar to human reasoning. After each generation failure, the system can simulate the feedback thinking of teaching experts, generating clear and structured secondary prompts based on issues such as uncovered knowledge points, difficulty deviations, or inappropriate expression, thereby completing targeted test question reconstruction and rewriting.

[0165] (3) Support for Data Augmentation of Long-Tail Knowledge Tags: This invention addresses the long-tail problem in tag distribution by proposing a targeted data augmentation mechanism for sparse knowledge tags. This mechanism analyzes the frequency distribution of knowledge tags in the training set to identify low-frequency long-tail tags. Combined with a thought-chain-style prompt generation strategy, it guides the model to actively generate more high-quality question samples around these long-tail tags, thereby improving the data completion and coverage capabilities for sparse knowledge points. Compared to traditional methods that passively rely on data distribution for generation, this mechanism significantly improves the training sufficiency and generation stability of the model on low-frequency knowledge points, enhancing the overall question generation system's adaptability and balance across the entire tag space.

[0166] The beneficial effects of this invention are as follows: Compared with traditional multi-label classification methods based on static corpora and general pre-trained models, this invention has higher accuracy and stability in terms of label semantic alignment and label discrimination precision; by introducing a question generation mechanism based on thought chain prompts, the quality and semantic coverage of the training corpus are significantly improved, enhancing the model's perception and prediction capabilities for low-frequency and deep labels; combined with a contrastive learning-optimized alternative to the Chinese semantic model, the representation power of the text encoder for question semantic features is effectively improved, enabling the model to exhibit stronger generalization ability and robustness under complex label systems; at the same time, the overall model exhibits faster training convergence speed and more stable performance in label prediction tasks, possesses good scalability, and is suitable for large-scale multi-label automatic annotation application scenarios.

[0167] This invention can be widely applied to personalized teaching recommendation platforms, intelligent question bank systems, multi-tag automatic annotation engines, and smart education content management platforms, providing teachers and teaching systems with accurate semantic analysis of test questions and automatic matching services of tag systems, significantly improving the organization efficiency and delivery effect of teaching resources.

[0168] This invention can be widely applied to personalized education platforms, adaptive learning systems, and intelligent teaching aids, providing teachers with accurate analysis of students' learning status, optimizing learning path design, and improving teaching effectiveness.

[0169] (i) Achieving high-quality label prediction based on structure awareness

[0170] By constructing an explicit label structure graph and introducing a graph neural network modeling mechanism, we can make full use of the hierarchical structure and semantic dependencies in the label system to achieve more accurate label propagation and filtering after the test text is encoded, thereby effectively improving the accuracy of label prediction, especially in label systems with a large number of labels, deep paths and semantic overlap.

[0171] (II) Constructing a test item-tag alignment mechanism under a unified semantic space

[0172] This invention integrates a Chinese semantic representation model trained by contrastive learning (such as bge-large-zh-TOT-15-5), which can effectively map the semantics of test questions to a vector space shared with the labels, achieving one-to-one semantic alignment between labels and test questions, enhancing the model's ability to distinguish and select labels, and showing good performance, especially in low-resource labels and weak supervision scenarios.

[0173] (III) Enhance the generalization ability of automatic multi-tag labeling

[0174] By combining the TOT (Total Time) prompt enhancement mechanism to generate high-quality pseudo-samples, the semantic boundaries and expression coverage of the training data are expanded, effectively solving problems such as insufficient original corpus samples and sparse labels. This significantly enhances the model's generalization ability to new questions and new label combinations, providing a guarantee for building a more robust multi-label system.

[0175] (iv) Promote the structural modeling practice of educational knowledge graphs and tagging systems

[0176] This invention uses the tag organization structure as the input skeleton of the graph network, which helps to promote the structured expression and utilization of educational tag ontology and knowledge point graph in practice, and provides methodological support for the future automatic construction and dynamic maintenance of educational tag system.

[0177] (V) Promote the intelligent development of education content management

[0178] The structure-aware automatic annotation system of this invention enables automatic semantic analysis and high-quality knowledge tag generation of large-scale test data, significantly reducing manual annotation costs, improving content aggregation efficiency, providing a technical foundation for intelligent teaching content distribution and recommendation, and accelerating the industrialization of educational resource management platforms. Attached Figure Description

[0179] Figure 1 This is a flowchart of the intelligent annotation method for test question knowledge system based on mind tree enhancement provided in an embodiment of the present invention;

[0180] Figure 2 This is an example diagram of the hierarchical distribution of knowledge point tags in the original question bank provided in this embodiment of the invention;

[0181] Figure 3 This is a statistical chart showing the distribution of tag combinations in the knowledge point tag set of the educational question bank provided in this embodiment of the invention;

[0182] Figure 4 This is a comparative statistical chart showing the number of tail tag sets after data augmentation provided in this embodiment of the invention;

[0183] Figure 5 This is a schematic diagram of the TOT question generation system provided in this embodiment of the invention;

[0184] Figure 6 This is a schematic diagram of the intelligent annotation method for test question knowledge system based on mind tree enhancement provided in this embodiment of the invention;

[0185] Figure 7 This is a structural diagram of the intelligent annotation system for test question knowledge system based on mind tree enhancement provided in this embodiment of the invention;

[0186] Figure 8 This is a comparison chart of experimental results provided in the embodiments of the present invention. Detailed Implementation

[0187] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0188] like Figure 1 As shown in the figure, the intelligent annotation method for test question knowledge system based on mind tree enhancement provided by this invention includes the following steps:

[0189] S101, Constructing a chain-thinking TOT-guided question generation system:

[0190] The design incorporates multi-layered Prompt templates based on pre-defined knowledge points, grade levels, question types, and difficulty levels to guide the large language model in generating test texts with standardized teaching requirements. By combining mechanisms such as knowledge point coverage assessment, question difficulty prediction, grade level suitability verification, diversity discrimination, and language logic review, a chain-like control process is constructed to ensure that the generated questions are structurally complete, content is standardized, and labels are accurately matched, thereby forming a high-quality training sample set.

[0191] S102, Expanding and reconstructing the dataset for long-tail knowledge labels:

[0192] To address the issues of imbalanced label distribution and scarcity of low-frequency knowledge point samples in the original data, the TOT-based question generation system constructed in step one is used to enhance the training samples corresponding to low-frequency knowledge points as needed. Then, the diversity of the original dataset is increased through EDA enhancement strategies to further expand the samples of low-frequency labels. Finally, the original data, EDA-enhanced data, and TOT-generated samples are combined to construct a dataset with balanced label distribution and strong semantic consistency.

[0193] S103, a test item representation coding model that introduces a contrastive learning mechanism:

[0194] A BGE-based encoder is used to semantically model the question text. Positive and negative question pairs are constructed by combining a contrastive learning strategy. By maximizing the vector proximity of questions with similar labels and minimizing the distance of questions with different labels, the discriminative ability of the embedding space is optimized. This enhances the model's ability to distinguish between questions with similar semantics but different labels, providing a stable representation basis for multi-label prediction.

[0195] S104, Constructing a structure-aware multi-label knowledge point annotation model:

[0196] Based on the encoder representation, the semantic path and hierarchical structure between knowledge point tags are modeled. Combining test question information and tag path information, semantic alignment is achieved through dual-encoding semantic fusion. Finally, multiple knowledge point tags that conform to semantic association and structural hierarchy are output, realizing a multi-tag knowledge point annotation task with high accuracy and strong generalization ability.

[0197] Some symbols appearing in the embodiments of this invention are shown in Table 1.

[0198] Table 1 Symbol Description

[0199]

[0200] Furthermore, S101 includes:

[0201] like Figure 2 Construct the control input space for question generation:

[0202] Let the total set of knowledge points be K = {k1, k2, ..., k}. L The set of grades is G = {g1, g2, ..., g}. J The question types are:

[0203] T = {t1, t2, ..., t} P The difficulty set is D = {d1, d2, ..., d}. Q Each problem generation task is defined by a control quaternion input:

[0204] x i =(K i ,g i ,t i ,d i )

[0205] in, g represents the set of knowledge points involved in the question. i ∈G represents the grade label, t i ∈T is the question type label, d i ∈D represents the difficulty level.

[0206] Based on the control input quadruple, the Prompt constructor is called:

[0207] P i =f prompt (K i ,g i ,t i ,d i )

[0208] This input space is used to construct structured natural language prompt templates, guiding the large language model to initialize test texts that meet the conditions. The design of this input space ensures that the question generation process is knowledge-point-oriented, adaptable to different teaching stages, constrained by question types, and graded in difficulty.

[0209] Construct a tree-structured exploration mechanism driven by a closed loop of generation, evaluation, and feedback:

[0210] This invention proposes a tree-like thinking chain-based question exploration mechanism integrating "generation-evaluation-feedback" closed-loop control to improve the controllability, stability, and goal alignment of educational large language models in teaching test question generation tasks. This mechanism views the question generation process as a step-by-step reasoning exploration process, constructing thought branches in stages and introducing automatic evaluation and strategic feedback at each step, thereby forming a structured multi-path generation representation.

[0211] Upon receiving the control input quadruple x i =(K i ,g i ,t i ,d i And construct the Prompt input P i The language model is invoked to generate initial candidate test questions. The process then proceeds to a hierarchical processing flow. This flow consists of five processing levels, L0 to L4, which correspond to: knowledge point coverage check, difficulty level matching, grade suitability verification, expression diversity control, and language standardization and logical consistency review.

[0212] In any processing layer L j (j∈{0,1,2,3,4}), the system executes according to the following three-stage process:

[0213] Generation phase:

[0214] The system calls the language model based on the current question state. And a Prompt prompt to generate the next version of the test questions. Where r is the number of attempts at the current level.

[0215] Evaluation phase:

[0216] Call the evaluator module E j (·) For candidate questions Perform automatic evaluation across the specified dimensions and output an evaluation result vector:

[0217]

[0218] in Its dimensions and content vary depending on the processing level. For example, the knowledge point layer includes dimensional information such as "coverage", "incorrect keywords", and "omission list".

[0219] Feedback and correction phase:

[0220] If the evaluation results do not meet the preset threshold (e.g., coverage is lower than θ), j Then, it will automatically respond based on the feedback content.

[0221] Table 2 Experimental parameters for step one

[0222] Construct a targeted retry prompt The language model is guided to generate new exploration branches. Each generation-evaluation-feedback process constitutes a ToT branch path.

[0223] This process is executed iteratively within each level until the success condition is met or the maximum number of attempts r is reached. max

[0224] All generation and correction attempts can be modeled as a branching tree within each layer:

[0225]

[0226] Each node records the result of one generation and evaluation. The final multi-level nested tree structure is defined as follows:

[0227]

[0228] Table 2 Experimental parameters for step one

[0229] Through this structure, each problem starts from the initial version. Starting from this point, through a multi-layered, progressive exploration of ToT (Total Time) and continuous refinement, a test version q that satisfies all constraints was finally obtained. final .

[0230] Furthermore, S102 includes:

[0231] like Figure 3 (2.1) Long-tail tag detection and low-frequency knowledge point recognition:

[0232] In the original dataset, each question corresponds to a set of knowledge point labels, denoted as label group C(q). This set may contain a single knowledge point or a combination of multiple knowledge points. Let the complete set of label groups be:

[0233] C = C1, C2, ..., C M

[0234] Each C m This represents a combination of labels. The original dataset can be formalized as:

[0235]

[0236] Where: q i Represents the test question text; C(q) i ) indicates the knowledge point tag group corresponding to the question; g i Indicates the grade level; t i Indicates the question type; d i Indicates the difficulty level.

[0237] Statistical analysis of each label group C m Number of samples in the original dataset:

[0238]

[0239] Set a long-tail threshold δ (e.g., if the number of samples is less than 15), and define the long-tail label group set as follows:

[0240]

[0241] Accurately identify label combinations with insufficient samples in the dataset and use them as the target set for subsequent data augmentation.

[0242] (2.2) Establish TOT-based directed sample generation rules and augment the data accordingly:

[0243] For each long-tail tag group It needs to be based on its existing quantity f(C) in the original dataset. m Using the threshold δ, calculate the number of samples that need to be added:

[0244] n gen (C m )=δ-f(C m )

[0245] To ensure that the generated samples conform to teaching principles, the allocation strategy of grade level, difficulty level, and question type is strictly controlled when using the TOT generation system:

[0246] The grade values ​​are taken from the existing grade set of this label group in the original dataset:

[0247] Gm ={g i |C(q) i ) = C m}

[0248] During generation, samples are randomly drawn according to the original distribution to ensure that the grade distribution of the new samples is consistent with that of the original samples, thereby avoiding the offset of the dataset in the grade dimension.

[0249] To accurately reflect the actual distribution of educational question banks, the difficulty level is allocated according to a weighted ratio of "low:medium:high = 2:5:1".

[0250] p low :p mid :p high =2:5:1

[0251] During the generation process, the difficulty value d is randomly sampled in this proportion to maintain the balance and authenticity of the model training samples at different difficulties.

[0252] From the question type set T = {t1, t2, ..., t} P A question type is randomly selected evenly from the dataset to ensure that the generated samples cover a variety of question formats, thereby improving the diversity and generalization ability of the dataset.

[0253] Using the above control strategy, the ToT question generation system from step one is invoked, with input parameters (C_m,g,t,d), to generate an enhanced sample set that meets the conditions:

[0254]

[0255] in This is to ultimately produce high-quality test questions through multiple rounds of evaluation and feedback optimization using a chain-thinking mechanism.

[0256] (2.3) Data augmentation based on EDA (Easy Data Augmentation) strategy:

[0257] In generating supplementary data for long-tail labels, in addition to using a TOT-based targeted sample generation system, the data is also augmented using an EDA (Easy Data Augmentation) strategy to increase the sample size of long-tail labels and enhance the diversity and generalization ability of the data.

[0258] Specific methods include synonym replacement, random insertion, random swapping, and random deletion. Synonym replacement replaces some words in a sentence with synonyms, increasing vocabulary diversity and generating different sentence versions. Random insertion increases sentence complexity and diversity by randomly inserting synonyms or related words. Random swapping alters the sentence structure while preserving the original meaning by randomly changing the positions of two words. Random deletion removes certain words from a sentence with a certain probability, reducing redundant information and simulating noisy data. These methods enable the generation of multiple enhanced versions for each sentence in the original data. These versions not only cover the semantic information of the original data but also increase sentence diversity through various enhancement techniques, thus improving data quality.

[0259] In practice, two parameters, `num_aug` and `alpha`, were designed to influence the data augmentation effect. By setting `num_aug`, we control the number of augmented samples generated for each original sentence. For example, when `num_aug = 1`, each original sentence will generate one augmented sample. The `alpha` parameter determines the percentage of words modified in each sentence. We set `alpha` to 0.1, meaning approximately 10% of the words will be modified, thereby further improving the diversity and quality of the dataset. The final augmented dataset D is obtained. EDA .

[0260] (2.4) Set reconstruction and consistency verification:

[0261] Augmentation of sample set D for generated long-tailed label groups cot D EDA Manual sampling is conducted, and the sampling process involves randomly selecting a portion of samples from the enhanced sample set. Reviewers then use the samples based on label group C. m The definition of knowledge points, for the extracted test questions The correspondence between the samples and the label groups is manually determined. Samples deemed non-compliant during the sampling process are removed from the augmented sample set, and the remaining samples constitute the sample set D that passed the sampling. gen,pass Compare this set with the original dataset D. orig Merge to obtain a new dataset

[0262] D new =D orig ∪D gen,pass

[0263] Furthermore, S103 includes:

[0264] like Figure 4 (3.1) Construct contrastive learning sample pairs:

[0265] Specifically, for each sample question, its corresponding set of knowledge point tags is extracted, and the training set D is traversed. new Compare the label sets of other samples:

[0266] When the label sets of two questions are completely identical, they are determined to be a positive sample pair;

[0267] When the label sets of two questions have no overlap, they are determined to be a negative sample pair.

[0268] If the label sets partially overlap, they will not participate in the construction of sample pairs.

[0269] To ensure the balance of sample distribution and training efficiency, this invention selects a maximum of several positive and negative sample pairs for each query sample: the number of positive samples does not exceed 15, and the number of negative samples does not exceed 5. If a sample cannot form a positive sample pair, it is removed from the training data, resulting in the final positive and negative sample pair dataset D. 15-5 .

[0270] (3.2) Hard Negative Sample Mining: In constructing the initial positive and negative sample pair dataset D 15-5 Based on this, the present invention further introduces a difficult negative sample mining strategy.

[0271] The specific process includes:

[0272] Embedded computation: All test item samples are vectorized and embedded using a pre-trained text representation model to form a unified vector retrieval space;

[0273] Similar sample retrieval: For each query question, retrieve question samples with inconsistent labels within a limited distance range, sort them from high to low vector similarity, and select a group of semantically similar label conflict samples;

[0274] Construction of difficult negative samples: A fixed number of high-similarity unlabeled samples are selected from the above screening results and added to the original training structure D as difficult negative samples. new-hard ;

[0275] (3.3) Training and optimization objectives of the contrastive learning model:

[0276] Based on the construction of a contrastive training sample set containing difficult negative samples, this invention further designs a contrastive learning training mechanism, which effectively improves the semantic representation ability of test questions through a target optimization strategy in the embedding space.

[0277] The specific training process is as follows:

[0278] Model structure initialization:

[0279] This invention employs a dual-tower text encoding structure based on the publicly released Chinese semantic vector model bge-large-zh-v1.5 to learn the embedding representation of test text. The query end and the comparison sample end share encoder weights. Each input test text is converted into a fixed-dimensional semantic embedding vector by the encoder.

[0280] Input sample composition:

[0281] The training inputs are organized in the form of triples, and the dataset comes from D. new-hard ,include:

[0282] Query test questions;

[0283] A positive sample item is one with the same label.

[0284] One or more hard negative sample items, i.e., items with no overlap in labels but similar semantics.

[0285] Each training sample constitutes a positive-negative comparison unit, used to compare the optimization of the objective function.

[0286]

[0287] Training objective function:

[0288] This invention employs a contrast loss function based on temperature scaling, and its core objective is:

[0289] Maximize the similarity between the query embedding and the positive sample embedding;

[0290] Minimize the similarity between the query embedding and the negative sample embedding;

[0291] Furthermore, during training, all embedded vectors are normalized to maintain the dominance of directional information and improve the sensitivity of similarity discrimination.

[0292] By introducing a temperature coefficient to control the granularity of similarity distinction, the stability and generalization ability of the optimization are enhanced.

[0293] Training parameters and process control:

[0294] Model training employs a mini-batch iterative approach, with 10 training rounds. The learning rate is adjusted to a low level to accommodate fine-tuning scenarios, further ensuring model convergence stability. In each training round, the contrastive learning module continuously optimizes the structural boundaries of the embedding space, gradually improving the model's ability to model subtle semantic differences.

[0295] Task adaptation instructions:

[0296] To enhance the model's generalization ability in specific tasks, this invention introduces domain task prompts as additional instruction inputs during the training process. These prompts guide the model to focus on understanding and modeling "mathematical problems with similar query content," thereby achieving targeted alignment with educational scenarios.

[0297] Through the above training mechanism, the resulting model bge-large-zh-TOT-15-5 has a stronger semantic distribution expression ability in the text embedding space. In particular, it can effectively identify and distinguish mathematical test questions with similar expressions but different knowledge points, providing a more robust representation foundation for subsequent knowledge point prediction models.

[0298] Table 3 Experimental parameters for step three

[0299]

[0300]

[0301] Figure 5 Schematic diagram of the TOT question generation system;

[0302] Furthermore, S104 includes:

[0303] (4.1) Model structure design:

[0304] This invention proposes a structure-aware multi-label text classification model. The overall architecture is based on a hierarchical perceptual network, optimized by a contrastive learning semantic encoder, forming a complete semantic structure-aware hierarchical framework specifically designed for automatic multi-label annotation tasks from test questions to knowledge point labels. This architecture features the following modular structure:

[0305] Text encoding module (optimized core):

[0306] This module converts test question text into semantic vector representations. It is implemented based on the Chinese semantic model bge-large-zh-TOT-15-5, optimized and trained using a contrastive learning method on the Chinese TOT augmented corpus. Structurally, this model employs a pre-trained Transformer framework, which, after optimization using the method of this invention, possesses stronger semantic discriminative capabilities and can effectively capture the semantic features of test questions. The model integrates a dedicated word segmenter and an encoder. The input text is first encoded by the word segmenter, generating input tensors such as token sequences, attention masks, and positional information. These tensors are then input to the encoder module, which outputs a 1024-dimensional semantic embedding representation of the test questions, serving as input to the subsequent structural modeling module. (Label embedding layer)

[0307] Feature alignment and label copying module

[0308] This module converts the semantic vectors of test questions generated by the text encoding module into the label node input format for adaptive graph structure modeling, ensuring that the semantic features of the test questions can be aligned and semantically propagated one-to-one with multiple knowledge point labels in the label space. Specifically, the test question vector output by the text encoding module is h. t ∈R d Where d = 1024 is the text embedding dimension. This module first copies the vector N times along the label dimension to construct the label node input matrix H. t ∈R N×d Where N is the number of knowledge point tags. This copying operation ensures that the semantic information of the test questions can be evenly distributed to each tag node, establishing a basis for semantic alignment between test questions and tags.

[0309] To further enhance the diversity and discriminative power of tag node representations, this module introduces two layers of linear transformations:

[0310] Feature projection layer: Performs a linear mapping on each copied label vector, projecting it from the original dimension d to the intermediate dimension d' required by the structure-aware module, thereby enhancing its expressive power in the label space;

[0311] Tag Dimension Adjustment Layer: The embedding of each tag node is reconstructed across dimensions to give it the tensor shape required to adapt to the structure propagation mechanism (as shown in the attention network), ensuring the coherence of information transmission in the structural path between tags.

[0312] The entire module is equipped with a Dropout layer (dropout rate adjustable, default is 0.1) to enhance the robustness and generalization ability of the training process. The output label alignment feature tensor will be used as input to the structural modeling module to construct the label semantic graph, realize the interaction propagation between labels, and semantic enhancement.

[0313] Structural Modeling Module

[0314] This module models the structural information between tags, explicitly introducing hierarchical paths and semantic dependencies in the knowledge point tag system to enhance the model's ability to perceive and represent tag structure. During implementation, this module receives the tag semantic tensor H output by the feature alignment module. t ∈R B×N×d′ Where B is the batch size, N is the number of labels, and d′ is the aligned feature dimension. This tensor is input into the structural modeling network, where the following key steps are performed:

[0315] Label Structure Graph Construction: This module aims to construct a graph representation that explicitly expresses the structural relationships between labels, guiding the subsequent structure-aware propagation process. The system first parses externally defined teaching knowledge system files (such as syllabi or knowledge point tree structures) and automatically identifies the parent-child relationships between each knowledge point label. Each label is mapped to a node in the graph, and directed connections are established according to the hierarchical logic of "parent label pointing to child label," thus forming a topologically ordered label structure graph. During this process, the system simultaneously introduces a label transition probability file as a structural prior, where the transition probabilities between label pairs are annotated in dictionary form. However, to simplify the graph construction process and improve the model's adaptability to the structure, in the current implementation, only the connection relationship identifiers provided by this file are used, with all valid connections uniformly assigned a value of 1, constructing a binary adjacency matrix as the input structure for the graph neural network. This approach allows the model to automatically learn the propagation strength and semantic relationships between labels through the structure-aware mechanism during training, without relying on precise domain priors.

[0316] Structure-aware information propagation: To model the topological dependencies between tags, the system organizes the connections between tags into a multi-layered encoding tree structure and implements multi-layer embedding propagation on the tree structure based on a graph neural network mechanism. Each layer of propagation follows the following computational process:

[0317] Neighbor feature aggregation

[0318] For the node representation of the l-th layer Its upper-level representation is through the adjacency matrix A (l) Implemented by performing sparse multiplication with node features:

[0319]

[0320] in:

[0321] This represents the adjacency matrix from layer l to layer l+1 (automatically constructed from the coding tree);

[0322] MLP (l) It is a multilayer perceptron of the l-th layer;

[0323] σ(·) represents the activation function (ReLU);

[0324] N l This represents the number of nodes in the l-th layer;

[0325] The intermediate representation of the upper-level nodes after aggregation

[0326] Batch normalization and nonlinear activation

[0327] After aggregation, the outputs of each layer undergo Batch Normalization and non-linear activation to stabilize the training process.

[0328]

[0329] Hierarchical feature fusion and label pooling

[0330] The final output label embedding tensor Z∈R B×d′ It integrates the semantic information of the test questions and the structural information of the labels, which serve as the input for the subsequent classification module.

[0331]

[0332] Multi-label classification decision module

[0333] This module is responsible for converting the structurally enhanced label embedding representation into the final multi-label prediction result. Input

[0334] The label-level logits tensor Z output by the model structure modeling module:

[0335] To achieve multi-label classification tasks, the system adopts a label-by-label independent activation mechanism, which converts the predicted score of each label dimension into a probabilistic form by performing a Sigmoid mapping.

[0336]

[0337] Subsequently, the system sets a fixed threshold τ∈(0,1) and performs a probability judgment on each label to determine whether to activate the label (default threshold τ=0.5):

[0338] if else 0

[0339] The final output is:

[0340] Binary label matrix This represents the set of predicted labels for each sample;

[0341] The experimental parameter settings for this embodiment of the invention are shown in Table 4.

[0342] Table 4 Experimental Parameter Settings

[0343]

[0344]

[0345] Figure 6 A schematic diagram of the intelligent annotation method for test question knowledge system based on mind tree enhancement;

[0346] like Figure 7 As shown in the embodiment of the present invention, the intelligent annotation system for test question knowledge system based on mind tree enhancement includes:

[0347] The data augmentation module receives test question text data and constructs high-quality training samples to support the training of the semantic coding model. This module first performs multiple rounds of semantic reconstruction on the original test question data based on the tree-like thinking chain data augmentation mechanism proposed in this invention, generating pseudo samples with clear logical structure and explicit thinking path, thereby enhancing the training semantic coverage of the original data. At the same time, it further augments the low-frequency label samples based on the EDA strategy.

[0348] The semantic contrast learning module, based on data augmentation, introduces Chinese semantic contrast learning methods to perform positive and negative contrast modeling on the generated samples, and trains a BGE Chinese semantic encoding model with domain adaptability. This model adopts a pre-trained Transformer architecture and can output a high-quality test question semantic embedding representation of dimension d, providing a robust semantic foundation for subsequent structural modeling.

[0349] The tag structure graph construction module is used to parse the parent-child relationship of tags in the teaching knowledge system file and construct a tag graph with a hierarchical topology. The system maps each knowledge point tag to a node in the graph, constructs a set of directed edges based on the parent-child path, and generates an adjacency matrix as the basic skeleton for graph structure propagation.

[0350] The label semantic alignment module is used to perform hierarchical propagation of semantic information in the label structure graph. This module is based on Tree Isomorphism Network (TIN), which aggregates label neighbor features layer by layer in a bottom-up manner, updates node representations through multilayer perceptron and normalization operations, and fuses hierarchical path semantics between labels to enhance the structural perception capability of label representation.

[0351] The multi-label classification prediction module is used to perform independent label prediction tasks on the structure-enhanced label representation. The system outputs the prediction probability of each label through the sigmoid activation function and sets a fixed threshold (such as 0.5) to determine whether the label is activated, thus forming a multi-label classification result. Finally, the system outputs the set of labels involved in each test sample, realizing the automatic multi-label annotation function.

[0352] The intelligent annotation method for test knowledge system based on mind tree enhancement provided in the application embodiment of the present invention is applied to a computer device, the computer device including a memory and a processor, the memory storing a computer program, and when the computer program is executed by the processor, the processor performs the steps of the learner cognitive tracking method for learning process evaluation.

[0353] The embodiments of the present invention have achieved some positive results during the research and development or use process, and have indeed great advantages compared with the prior art. The following content describes them in conjunction with the data, charts and other information of the experimental process.

[0354] To illustrate the positive effects of the embodiments of the present invention, the intelligent annotation method for test question knowledge system based on mind tree enhancement proposed in this invention is used to...

[0355] The educational question bank was enhanced. In addition, the experimental results of this method were compared with those of the traditional architecture. Micro-F1, MACRO-F1 and PRECISION were used as evaluation indicators. The comparison results are shown in Table 5.

[0356] Table 5 Experimental Results

[0357] method PRECISION Micro-F1 MACRO-F1 Traditional framework 87.44％ 86.09％ 69.66％ Semantic Structure Hierarchical Perception Framework 89.99％ 88.53％ 72.87％

[0358] Figure 8 This is a comparative diagram of experimental results provided by the embodiments of the present invention. The experimental results show that, compared with the traditional framework, the semantic structure hierarchy perception framework proposed in this invention improves the RECISION, Micro-F1, and MACRO-F1 scores by 2.55%, 2.44%, and 3.21%, respectively. This demonstrates that the present invention can better mine the hierarchical relationships between knowledge points in the educational question bank. By introducing a data generation mechanism enhanced by the thinking chain, it continuously optimizes and adjusts during the question generation process, thereby effectively improving the coverage and recognition ability of scarce knowledge points. Through a text encoder optimized based on contrastive learning, the model can more accurately match questions and knowledge point labels at the semantic level and effectively capture the hierarchical relationships and semantic dependencies between labels. Ultimately, it significantly improves the accuracy of annotation and the generalization ability of the model, exhibiting stronger adaptability and accuracy compared to traditional methods.

[0359] Experimental results demonstrate that the intelligent annotation method for test knowledge systems based on enhanced mind trees provided in this invention effectively improves the recognition ability of low-frequency knowledge points through a data generation mechanism enhanced by tree-like mind chains and a text encoder optimized based on contrastive learning. This method fully explores the hierarchical structure and semantic dependencies of each knowledge point in the educational question bank during question generation and label path modeling, and performs fine-grained embedding representation of knowledge points based on multi-dimensional information, thereby improving the semantic alignment between labels and questions. Simultaneously, by introducing a dual-encoding semantic fusion mechanism, the hierarchical semantic propagation ability between labels is enhanced, which helps to build a more accurate multi-label prediction model. Ultimately, the model significantly improves annotation accuracy and generalization ability to new label combinations, exhibiting stronger adaptability and accuracy compared to traditional methods, and can provide more efficient and accurate knowledge point annotation and personalized learning path planning for the education field.

[0360] Example 1

[0361] This embodiment provides an automatic annotation method for math test questions based on mind tree enhancement. The system first receives input conditions including the knowledge point "Functions and Derivatives," grade level 11, multiple-choice questions, and medium difficulty level. It then generates initial candidate questions by constructing a Prompt template. Subsequently, it enters a tree-like exploration structure, sequentially generating and evaluating multiple rounds of questions based on knowledge point coverage, difficulty matching, grade level suitability, expression diversity, and language logic consistency. Finally, it obtains multiple-choice questions that meet the requirements and automatically outputs the corresponding knowledge point tags.

[0362] This method introduces a closed-loop feedback mechanism during the question generation process. When the knowledge points covered by candidate questions are insufficient, the system automatically constructs targeted prompts for retrying, thereby ensuring an accurate correspondence between the generated questions and the labels. The final generated questions, after being verified by teachers, can be directly added to the training dataset, ensuring high data quality and controllability.

[0363] Example 2

[0364] In another implementation, the system performs data augmentation on long-tail knowledge points in the "probability and statistics" domain. Since the original dataset lacks sufficient questions on conditional probability, the system identifies that the label group belongs to the long-tail set, invokes a tree-structured data generation system, and automatically supplements the samples according to grade distribution and difficulty ratio, generating diverse questions ranging from easy to difficult.

[0365] The newly generated samples are enhanced by synonym replacement and random insertion, further increasing the diversity of sentence structures. After manual sampling and verification, the final enhanced dataset is fused with the original corpus, providing rich long-tail samples to support the training of multi-label classification models.

[0366] Example 3

[0367] In the semantic contrastive learning phase, questions related to "function monotonicity" in the original dataset are selected as query samples. The system automatically retrieves questions with identical labels to construct positive sample pairs and retrieves questions with completely different labels to construct negative sample pairs, forming positive and negative contrast units.

[0368] Furthermore, the system utilizes a semantic encoder to generate embedding vectors, and performs difficult negative sample filtering on questions with similar semantics but different labels, adding these negative samples to the training set. This method optimizes the boundary of the model's embedding space, enabling it to more accurately distinguish between questions with similar expressions but different knowledge points.

[0369] Example 4

[0370] In a practical deployment, a dual-tower encoder structure based on the BGE semantic model was used to embed the text of "analytic geometry" test questions. The encoder input was processed by word segmentation and vectorization, and the output was a 1024-dimensional semantic representation vector.

[0371] The system organizes input data in triplets, including query questions, positive samples, and difficult negative samples. It trains the semantic space using a comparative loss function, enabling the model to continuously improve its ability to discriminate semantic differences in mathematical questions during optimization. Testing shows that the model can accurately distinguish the knowledge point labels related to parabolas and ellipses.

[0372] Example 5

[0373] In this embodiment, a tag structure graph model is introduced during the knowledge point annotation process. The system first parses the national curriculum standard document, automatically identifies the hierarchical path of "numbers and algebra - equations and inequalities - quadratic equations in one variable", and constructs it into a directed graph structure, with parent-child relationships between nodes represented by an adjacency matrix.

[0374] During the label propagation process, the system uses graph neural networks to aggregate label features layer by layer, so that knowledge points such as the upper-level "quadratic equation" and the lower-level "discriminant of roots" have structural dependencies in the representation space, thereby enhancing the model's hierarchical perception capability.

[0375] Example 6

[0376] In the classification prediction stage, the system receives input text containing algebra questions, and outputs label-level logits after semantic encoding and structural modeling. The prediction result for each label dimension is mapped to a probability score by the Sigmoid function, and then a threshold is used to determine whether to activate the corresponding label.

[0377] For example, when given a problem about "solving the coordinates of the vertex of a quadratic function", the system will output a set of labels including "functions and derivatives" and "quadratic functions", represented in binary matrix form. The results have been manually checked and verified to be highly consistent with the actual labels.

[0378] Example 7

[0379] In practical teaching applications, the system receives test questions on "fraction addition and subtraction" from first-year junior high school students as input. The system first generates multiple candidate questions through a tree-structured generation mechanism, and then eliminates versions that omit key knowledge points during a coverage assessment, ultimately obtaining questions that meet the curriculum standards.

[0380] These questions are automatically annotated and stored in the question bank, which teachers can directly access, significantly reducing the time required for manual annotation and improving the efficiency of question bank construction.

[0381] Example 8

[0382] In cross-grade application scenarios, the system generates and labels questions on the same knowledge point, "similar triangles," for both junior high and senior high school levels. During the generation process, the grade-appropriateness verification module automatically identifies whether the terminology used in the question stem meets the requirements of the target grade level.

[0383] The results show that for questions generated at the high school level, the system can accurately distinguish between different knowledge point tags such as "similarity ratio" and "similarity conditions," achieving more refined annotation. This method ensures the teaching adaptability of test question corpora across different grade levels.

[0384] Example 9

[0385] During a single model training iteration, the system sets constraints on the number of positive samples and negative samples for comparative learning, limiting them to no more than 15 and 5 respectively, to ensure training efficiency and a balanced sample distribution. Questions whose label sets cannot form positive sample pairs are automatically removed to avoid training interference.

[0386] This sample construction mechanism enables the model to maintain efficient training under limited computing resources and demonstrates stronger semantic discrimination ability in subsequent tasks, especially in fine-grained knowledge point annotation of mathematical word problems.

[0387] Example 10

[0388] In the complete system implementation, the data augmentation module first generates augmented data using a tree-like thinking mechanism and EDA strategy; the semantic contrast learning module optimizes semantic embedding by mining difficult negative samples; the label structure construction module builds a topology graph based on the knowledge system; the semantic alignment module performs hierarchical propagation through a graph neural network; and the classification prediction module finally outputs multi-label results.

[0389] This system architecture can be deployed in real-world educational scenarios to automate the annotation of test questions and knowledge points. Large-scale experiments have verified that the system's annotation accuracy is significantly improved compared to traditional methods, and its performance on long-tail tags is also remarkably enhanced.

[0390] It should be noted that embodiments of the present invention can be implemented in hardware, software, or a combination of both. The hardware portion can be implemented using dedicated logic; the software portion can be stored in memory and executed by a suitable instruction execution system, such as a microprocessor or dedicated-design hardware. Those skilled in the art will understand that the above-described devices and methods can be implemented using computer-executable instructions and / or included in processor control code, for example, such code provided on a carrier medium such as a disk, CD, or DVD-ROM, a programmable memory such as read-only memory (firmware), or a data carrier such as an optical or electronic signal carrier. The devices and modules of the present invention can be implemented by hardware circuitry such as very large-scale integrated circuits or gate arrays, semiconductors such as logic chips, transistors, or programmable hardware devices such as field-programmable gate arrays, programmable logic devices, etc., or by software executed by various types of processors, or by a combination of the above-described hardware circuitry and software, such as firmware.

[0391] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications, equivalent substitutions, and improvements made by those skilled in the art within the scope of the technology disclosed in the present invention, and within the spirit and principles of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A method for intelligent annotation of test knowledge systems based on mind tree enhancement, characterized in that, Includes the following steps: A chain-thinking-guided question generation system is constructed, which forms an input control quadruple based on preset knowledge points, grade, question type and difficulty information, and generates candidate questions based on the Prompt template; In the generation process, a closed-loop control of generation, evaluation and feedback is introduced to form a tree-like exploration structure. Candidate test questions are checked and corrected in multiple rounds at five levels: knowledge point coverage, difficulty level, grade suitability, expression diversity and language logic, until the target test questions that meet the constraints are obtained. For low-frequency knowledge point samples in the original dataset, the test question generation system is used to supplement the samples in a targeted manner, and the dataset is expanded by combining data augmentation methods. A training model based on semantic encoding and contrastive learning is adopted to construct positive and negative sample pairs and mine difficult negative samples to optimize the semantic representation space of test questions; Based on semantic representation, and combined with the hierarchical path and semantic dependency of knowledge point tags, a structure-aware multi-label annotation model is constructed to achieve multi-label prediction of test knowledge points.

2. The method as described in claim 1, characterized in that, The control input quadruple includes a set of knowledge points, a grade label, a question type label, and a difficulty level. The set of knowledge points is selected from the complete set of knowledge points, the grade label is selected from the set of grades, the question type label is selected from the set of question types, and the difficulty level is selected from the set of difficulties.

3. The method as described in claim 1, characterized in that, The tree-like exploration structure executes a three-stage process of generation, evaluation, and feedback at each level. In the evaluation stage, a result vector is output based on the corresponding dimension. If the preset threshold is not met, the feedback stage constructs a new prompt based on the result vector to guide the language model to continue generating exploration branches.

4. The method as described in claim 1, characterized in that, For knowledge point labels with insufficient samples, an augmented sample is generated using a tree-structured thinking generation system based on grade distribution consistency and difficulty ratio strategies. This is combined with data augmentation methods based on synonym replacement, random insertion, random swapping, and random deletion to form an expanded dataset.

5. The method as described in claim 1, characterized in that, The contrastive learning training is carried out by constructing positive sample pairs and negative sample pairs, wherein the label sets of positive sample pairs are completely identical, the label sets of negative sample pairs are completely disjoint, and partially overlapping samples do not participate in the construction of sample pairs.

6. The method as described in claim 5, characterized in that, Further, we introduce difficult negative sample mining to screen test questions that are semantically similar but have inconsistent label sets, and add them to the training dataset as difficult negative samples to improve the model's ability to distinguish subtle semantic differences.

7. The method as described in claim 1, characterized in that, The semantic representation model adopts a dual-tower text encoding structure with shared weights, and performs vectorized embedding of test texts and sample pairs. It uses a temperature-scaling-based contrastive loss function to maximize positive sample similarity and minimize negative sample similarity.

8. The method as described in claim 1, characterized in that, The multi-label knowledge point annotation model includes a text encoding module, a label embedding module, a structure modeling module, and a multi-label classification decision module. The text encoding module converts the test text into semantic embeddings, the label embedding module copies and projects the test questions into the label nodes, the structure modeling module propagates the label structure information based on a graph neural network, and the multi-label classification decision module outputs the label prediction results.

9. The method as described in claim 8, characterized in that, The multi-label classification decision module performs Sigmoid mapping on the predicted score of each label dimension through a label-by-label independent activation mechanism, and converts the prediction results into a binary label matrix based on a fixed threshold.

10. A smart annotation system for test knowledge systems based on mind tree enhancement, implementing the method as described in any one of claims 1 to 9, characterized in that, include: The data augmentation module is used to expand test question data based on tree-like thinking chains and data augmentation strategies; The semantic contrastive learning module is used to optimize the semantic representation model by mining positive-negative sample pairs and hard negative samples; The tag structure diagram construction module is used to parse the teaching knowledge system and construct a tag hierarchy diagram. The label semantic alignment module is used to perform semantic propagation and feature fusion in the label structure graph; The multi-label classification prediction module is used to output multi-label classification results based on the semantically and structurally enhanced representation.

Images