Big language model teaching quality evaluation method based on Bloom target classification system

Abstract

The invention belongs to the technical field of natural language processing, and particularly relates to a Bloom target classification system-based large language model teaching quality evaluation method, which comprises the following steps of: constructing a structured cue word template according to a Bloom target classification system; driving the large language model to cooperatively generate a teaching quality evaluation data set according with a specified format and distribution requirements through the cue word template; a ChatGLM3 pre-training model is mainly composed of 28 layers of GLM modules based on a Transform network, a multi-layer perceptron of part of the GLM modules is replaced with a hybrid expert network, a trainable prefix code is added in front of each layer of the GLM module of the ChatGLM3 pre-training model, a Mama state space model is added behind a word embedding layer and in front of the GLM module, and a teaching quality assessment big language model is obtained. And teaching quality evaluation is carried out after training. According to the invention, the accuracy and pertinence of teaching quality evaluation can be improved.

Description

Technical Field

[0001] This invention belongs to the field of natural language processing technology, and constructs a teaching quality evaluation method based on Bloom's Taxonomy of Objectives. Background Technology

[0002] In the field of higher education, with the growth of student numbers, there is a certain contradiction between the limited number of teachers and the increasing demand for personalized teaching. Teachers find it difficult to fully and timely grasp the learning status and cognitive progress of each student, resulting in the inability to solve the problems encountered by students in the learning process in a timely manner. This not only affects students' knowledge construction and ability improvement, but also restricts the improvement of overall teaching quality to a certain extent.

[0003] In recent years, large language models, as an important technological breakthrough in the field of artificial intelligence, have achieved good results in many professional scenarios such as medical diagnosis and agricultural consultation, demonstrating their capabilities in natural language processing and problem diagnosis. Based on this technological trend, many universities have begun to try to introduce large language models into student learning management and teaching support platforms, using their efficient information retrieval and generation capabilities to provide students with instant Q&A and learning resource recommendations, thereby making up for the lack of semantic interaction support in traditional databases to a certain extent and promoting the improvement of education quality. However, existing large language models applied to education scenarios still have the following technical limitations: (1) The interaction mode is singular. At present, most large language models in the field of education adopt a passive response interaction mode, mainly in the form of question and answer, directly returning answers or explanations to user input. Although this "ask and answer" mechanism is direct, it weakens the willingness of students to actively explore and think deeply, which is not conducive to the cultivation of students' higher-order cognitive abilities. (2) Insufficient functional integration: Existing large language models still have significant limitations in educational applications. Most systems lack the ability to call and integrate external tools (such as computing platforms, databases, academic resource repositories, etc.) and do not support contextualized or continuous learning guidance. This limits their ability to cope with complex, dynamic, and interdisciplinary actual teaching needs. The model's response is usually limited to its internalized knowledge structure, making it difficult to achieve adaptive teaching guidance. (3) Lack of a scientific evaluation system: Existing technical solutions generally lack a multi-level evaluation mechanism for students' cognitive levels, making it difficult to accurately identify students' mastery at different cognitive levels. As a result, they cannot provide teaching support that matches students' actual cognitive development stages.

[0004] Therefore, it is necessary to further explore the cognitive assessment methods of the big language model in the theory of integrated education, promote its transformation from a single "answer provider" to a "capability enhancer", and establish a scientific cognitive ability assessment system to improve the accuracy and pertinence of teaching quality assessment through more reasonable instructional design and more effective interaction strategies. Summary of the Invention

[0005] The purpose of this invention is to provide a teaching quality assessment method based on Bloom's classification system using a large language model, addressing the problems of insufficient automation and lack of multi-level cognitive ability assessment mechanisms in existing technologies. This invention assesses teaching quality based on a large language model, considering both the assessment questions and student responses. Specifically, this invention first drives the large language model to generate a student answer assessment dataset that conforms to the specified cognitive level format and question type requirements through structured question prompt templates. Second, it employs parameter fine-tuning techniques, using strategies such as separate training, instruction fine-tuning, and mixed prompt settings to further improve the balance and overall performance of the large language model across different cognitive level assessment tasks. Simultaneously, it replaces the Multilayer Perceptron (MLP) of the GLM module in layers 20-22 of the ChatGLM3 pre-trained model with a Hybrid Expert Network (MoE), and adds a Mamba state-space model after the word embedding layer and before the GLM module in the ChatGLM3 pre-trained model to reduce computational resource consumption, maintain model performance, effectively prevent overfitting, and improve the model's generalization ability.

[0006] This invention is achieved through the following technical solution: A method for evaluating the teaching quality of a large language model based on Bloom's Taxonomy of Objectives, comprising the following steps: Step 1: Construct a structured prompt word template based on Bloom's Taxonomy; Step 2: Drive the large language model through prompt word templates to collaboratively generate a teaching quality assessment dataset that meets the specified format and distribution requirements. The teaching quality assessment dataset contains questions that examine different Bloom objectives and corresponding, diverse student answer samples. Step 3: Based on the ChatGLM3 pre-trained model, optimize its architecture to obtain a large language model for teaching quality assessment, including: Add a trainable prefix code before each GLM module in the ChatGLM3 pre-trained model; Add the Mamba state space model after the word embedding layer and before the GLM module; Replace the multilayer perceptron (MLP) of some layers of the GLM module in the ChatGLM3 pre-trained model with a hybrid expert network (MoE). Step 4: Use the teaching quality assessment dataset to train the large language model for teaching quality assessment, and obtain the final large language model for teaching quality assessment, which can then be used for teaching quality assessment.

[0007] Further preferably, the processing procedure for the Mamba state-space model includes: The word embedding vector is mapped to the target dimension through linear projection to obtain the projected input vector; the projected input vector is divided into two branches: the main branch and the gated branch. In the main branch, the projected input vector is processed by an activation function to obtain the main branch feature vector; In the gated branch, the projected input vector is processed sequentially through a convolutional layer, a selective state-space model, and an activation function to obtain the gated branch feature vector; The feature vectors of the main branch and the gated branch are fused to obtain the fused feature vector. The fused feature vector is subjected to linear projection and dimensionality compression, combined with positional encoding and prefix encoding, and then output to the subsequent GLM module.

[0008] Further preferred, in the gated branch: Local contextual features of the projected input vector are extracted through convolutional layers; The local context features are input into four different linear transformation layers to simultaneously generate the four dynamic parameters required for the selective state-space model, including: dynamic state transition matrix, dynamic input matrix, dynamic output matrix, and dynamic direct path vector. The selective state-space model utilizes four dynamic parameters, local context features, and the hidden state from the previous time step to perform state updates and output computation, yielding the final output of the selective state-space model. The final output of the selective state-space model is then processed... The activation function is used to process the gated branch feature vectors.

[0009] Further preferably, the state update process of the selective state-space model is as follows: The state of the previous time step is decayed based on the dynamic state transition matrix of the current time step to obtain historical retained information. The current input features, after gated nonlinear transformation, are weighted according to the dynamic input matrix at the current time step to obtain new information injection; The historical information is added to the new information to obtain the state at the current time step.

[0010] Further preferably, the processing procedure of the hybrid expert network includes: The input features are fed into the feedforward network to generate the gating weights for each expert network; The top k expert networks are selected based on the gating weights, and the input features are assigned to the selected expert networks according to the weights. Each expert network processes the input features independently, and the output of the expert network is obtained. The outputs of each expert network are weighted and summed according to the gating weights to obtain the final output of the hybrid expert network.

[0011] Further preferably, the process of generating the gate weights by the feedforward network includes: Introduce a noise term when calculating the gating weights; A Top-K sparse gating mechanism is adopted, which only activates the expert network with the top k gating weights.

[0012] Further preferred, the training employs P-Tuning v2 fine-tuning technology to freeze the weights of all parts except the Mamba state-space model and the hybrid expert network, and inserts trainable prefix codes before each layer of the GLM module. This, combined with separate training, instruction fine-tuning, and hybrid prompting setting strategies, improves the balance and overall performance of the large language model in different cognitive level assessment tasks for teaching quality evaluation.

[0013] Further optimization is made by constructing the prompt word template based on the six cognitive levels of Bloom's Taxonomy of Objectives, which include memory, understanding, application, analysis, evaluation, and creation.

[0014] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor implements the teaching quality assessment method when executing the program.

[0015] The present invention also provides a computer-readable storage medium having a computer program stored thereon, characterized in that the program, when executed by a processor, implements the teaching quality assessment method described above.

[0016] The present invention has the following advantages: 1. Comprehensive dataset coverage and complete assessment dimensions. The generated teaching quality assessment dataset covers student answer samples of different question types, different Bloom's cognitive objectives, and different quality levels. Each data point includes "score reasons" and "improvement suggestions," which can simulate the teacher's assessment thinking process. It is suitable for training automated scoring models and research on teaching feedback generation, and can effectively alleviate the problem of difficulty in comprehensively assessing student learning quality due to teachers' limited energy.

[0017] 2. High efficiency and low resource consumption in long text processing. By integrating the Mamba state-space model into the ChatGLM3 pre-trained model, inference efficiency is optimized, reducing computational resource consumption while maintaining model performance. This further improves the model's performance in long text processing tasks, while preserving its capabilities in general dialogue tasks and enhancing its ability to handle long contexts.

[0018] 3. Large model capacity and high computational efficiency. By replacing the Multilayer Perceptron (MLP) in the GLM module of layers 20-22 in the ChatGLM3 pre-trained model with Hybrid Expert Network (MoE) layers, the model can dynamically allocate computational resources according to the input content, significantly improving the model capacity and expressive power without increasing the computational load. The multi-expert network mechanism enables fine-grained knowledge partitioning and storage, resulting in faster convergence compared to denser models with equivalent computational load; simultaneously, the larger parameter capacity enhances the model's knowledge storage and expressive capabilities.

[0019] 4. High fine-tuning efficiency and strong cognitive guidance capabilities. By employing a parameter-efficient fine-tuning method, freezing the weights of the ChatGLM3 pre-trained model, and inserting trainable prefix codes before each GLM module, combined with strategies such as separate training, instruction fine-tuning, and mixed prompt settings, the model's balance and overall performance across different cognitive levels of assessment tasks are further improved. While retaining the original functions of the ChatGLM3 pre-trained model, it enables the assessment of students' learning status at different learning levels, overcoming the limitation of traditional dialogue models that directly provide answers and cannot effectively guide students to engage in deep thinking, thus contributing to the cultivation of students' higher-order cognitive abilities. Attached Figure Description

[0020] Figure 1 This is a flowchart of the teaching quality assessment method for a large language model based on Bloom's Taxonomy of Objectives, as presented in this invention. Figure 2 This is the architecture diagram of the large language model for teaching quality assessment of the present invention. Detailed Implementation

[0021] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments.

[0022] Reference Figure 1 A teaching quality assessment method based on Bloom's Taxonomy of Objectives using a large language model, comprising the following steps: Step 1: Construct a structured prompt word template based on Bloom's Taxonomy; Step 2: Drive the large language model through prompt word templates to collaboratively generate a teaching quality assessment dataset that meets the specified format and distribution requirements. The teaching quality assessment dataset contains questions that examine different Bloom objectives and corresponding, diverse student answer samples. Step 3: Based on the ChatGLM3 pre-trained model, optimize its architecture to obtain a large language model for teaching quality assessment, including: (1) Add trainable prefix codes before each GLM module in the ChatGLM3 pre-trained model to achieve efficient parameter fine-tuning; (2) Add the Mamba state space model after the word embedding layer and before the GLM module to optimize inference efficiency, reduce the amount of computation and memory consumption in the long text processing process, and expand the supported context length. (3) Replace the multilayer perceptron (MLP) in the GLM module of layers 20-22 in the ChatGLM3 pre-trained model with a hybrid expert network (MoE) to improve the model capacity without significantly increasing the actual computational cost; through the gating network, the dynamic distribution of input and the competition mechanism between expert network modules are realized, which enhances the sparsity and controllability of the model, effectively prevents overfitting, and improves the generalization ability. Step 4: Use the teaching quality assessment dataset to train the large language model for teaching quality assessment, and obtain the final large language model for teaching quality assessment, which can then be used for teaching quality assessment.

[0023] In this embodiment, based on the six cognitive levels of Bloom's Taxonomy (memory, comprehension, application, analysis, evaluation, and creation) and actual assessment needs, a structured prompt word template is constructed, clearly specifying the data fields, question types, question frequencies, and scoring criteria for each prompt word template.

[0024] To alleviate teachers' teaching burden, structured prompt word templates are input into different large language models. These prompt word templates drive the large language models to collaboratively generate a teaching quality assessment dataset that conforms to specified format and distribution requirements. The teaching quality assessment dataset contains questions that examine different Bloom's cognitive objectives and corresponding, diverse student answer samples. Subsequently, the teaching quality assessment dataset is divided into training, validation, and test sets for training the teaching quality assessment large language model.

[0025] Figure 2 The present invention demonstrates the architecture of a large language model for teaching quality assessment, which mainly consists of a trainable cue layer, a newly added Mamba state space model, and a ChatGLM3 pre-trained model that replaces the hybrid expert network (MoE).

[0026] Because the computational complexity of the self-attention layer in the ChatGLM3 pre-trained model is O(n^2). 2 ), The sequence length is so long that as the sequence length increases, the computational cost and memory consumption of the model increase quadratically. This severely limits the model's ability to handle long contexts. Therefore, to improve the linear computational complexity advantage of the original model in handling long sequences and to maintain or even improve the model's performance while significantly reducing computational and memory overhead in long sequence scenarios, this invention introduces the Mamba State Space Model. Its linear time complexity means that the computational cost increases linearly with the sequence length. Therefore, it can handle extremely long sequences very efficiently, which is a significant advantage over the traditional Transformer. Thus, this invention's large language model for teaching quality assessment adds a Mamba State Space Model layer before the input to the GLM module, based on the original ChatGLM3 pre-trained model. This allows for efficient encoding of long sequences using the Mamba State Space Model, capturing long-distance dependencies, and outputting a refined sequence representation before inputting it into the subsequent GLM module. This enables the subsequent self-attention layer to... Working on more compact and richer representations can significantly reduce the overhead spent on self-attention. The self-attention layer in the ChatGLM3 pre-trained model was not replaced with the Mamba state space model because adding it is a more conservative and robust strategy than replacing it. Although module replacement has great potential, the risks are also relatively higher. If the Mamba state space model does not perform as expected on certain tasks, the retained self-attention layer can still serve as a strong guarantee, ensuring that the lower limit of the model's performance is not too bad. For those with limited resources, this can achieve the effect of reducing computational resource consumption while maintaining or even improving model performance. By combining the advantages of the two paradigms, a 1+1>2 effect can be achieved.

[0027] First, the input is a mixture of course materials, questions, and answers. Data cleaning is performed first, and then the resulting vector is input into the word embedding layer. After exiting the word embedding layer, the word embedding vector undergoes a linear projection to map its original dimension to the target dimension of the Mamba state space model. Then, the input with the magnified dimension is divided into two branches: one is the main data channel, and the other is a gated branch, used to determine the final output. This process can be represented by formula (1): (1); in, The word embedding vector represents the feature vector obtained by splicing and fusing course materials, questions, and student answers. The input projection weight matrix represents the teaching quality assessment task; Indicates input bias; This represents the input vector of the projected Mamba state-space model.

[0028] Then, in the gated branch, The convolutional layers extract local contextual features, and through the local connectivity and weight sharing characteristics of the convolutional layers, smooth and context-rich continuous segments are transformed, providing a more convenient input for subsequent selective state-space models. This complements the long sequence modeling capabilities of selective state-space models, enhancing the efficiency and representational ability of large language models for teaching quality assessment on sequence data. The process through the convolutional layer can be represented by formula (2): (when (2) if If so, then the value of this item is 0 (meaning that future time steps are not visited). in, Indicates the kernel size, controlling the range of local dependencies. The larger the value, the more past time steps the current output can see; Indicates the current time step The local context features output by the convolutional layer, after being expanded in dimension by an expansion factor, are simultaneously fed into the state update gate and the output gate. This represents the convolution kernel weights; each row represents a convolution kernel with one channel and a length of [length value missing]. Its function is to learn how to extract the local temporal features of the channel. The overall effect of this process is to provide local information of the most recent time steps to help the gated branch decide to update the state and output, thereby ensuring that the model will not use the information of future words in future tasks. i is the index within the convolution window, used to traverse and accumulate the input in the past period that the current time step t depends on.

[0029] Next, we arrive at the core innovation of the Mamba state-space model, which distinguishes it from the traditional state-space model SSM. The key parameters of the state-space model dynamically change with the input, unlike the traditional SSM which uses fixed parameters. This invention dynamically generates the four core dynamic parameters required for the selective state-space model through a linear projection mechanism, including the dynamic state transition matrix, dynamic input matrix, dynamic output matrix, and dynamic direct path vector. This process can be expressed by formula (3): (3); The four parameters on the left come from the core matrices of the traditional state-space model SSM, but in this model, a dynamically changing output is implemented. This represents the dynamic state transition matrix, also called the decay gate, which controls the state at the previous time step. To affect the state of the current time step Similar to memory decay rate, through An activation function is used to ensure its positive orientation, highlighting the current new information. The decay of the historical state used in subsequent calculations determines how much historical information is retained. This represents the dynamic input matrix, also called the input gate, which controls the current input features. How to inject weights into the new state, similar to the current information, and reduce the interference of other information on the state, determines what proportion of the current candidate states will be included in the new state. The dynamic output matrix, also known as the candidate state gate, controls the state at the current time step. How historical information is extracted into the output is similar to retrieving information from memory, and it is the core source of information for the new state; Represents the dynamic direct path vector, also called the output gate, which determines how much state information is ultimately output to the next layer; , , and These are the linear transformation weight matrices for the attenuation gate, input gate, candidate state gate, and output gate, used to... The dimension is mapped to the dimension of the gate vector, and the local temporal features captured by the convolution are learned to be transformed into decision signals for the gate to influence the output of each gate. , , and These are the bias terms of the decay gate, input gate, candidate state gate, and output gate, respectively, used to fine-tune the linear transformation results. The entire process is to provide a decision basis for subsequent selective scanning. By working together with different gates, selective forgetting of historical states, selective integration of current information, and selective output of states are achieved, enabling the model to accurately and efficiently process long sequence tasks.

[0030] Then, the local context features extracted by convolution are projected into a high-dimensional space. Selective switching is used to remember key feature information and forget irrelevant noise, and finally, features with long-term temporal dependence are extracted. This process can be represented by formula (4): , , (4); in, It is a dynamic state transition matrix generated from formula (3), which determines the speed of information updates at the current moment. This represents the system evolution matrix, responsible for capturing long-term memories. This represents the discretized state transition matrix, indicating how much historical information has been retained to the present. The discretized input matrix represents how much new information is retained in the state at the current time step. This represents the hidden state at the current time step, containing not only information about the current time step t, but also compressed historical information from previous times. Representing the hidden state at the previous time step, the selective state-space model uses... Selective forgetting of old memories, through This allows for the absorption of new information, ultimately achieving long-sequence modeling and ensuring training stability. This represents the final output of the selective state-space model.

[0031] Then comes the final stage of the gated branch, where the final output of the selective state-space model is further introduced into nonlinearity through the SiLu activation function. This process can be expressed by formula (5): (5); in, and This represents the two linear transformation weight matrices of the gating. Able to Mapping to a "candidate feature" space preserves the potentially useful information of the input; Can Mapped to the "gating signal" space, it is equivalent to a switch used to filter information; , These are two bias terms of the linear transformation, used to fine-tune the distribution of candidate features and gated signals, thereby avoiding the transformation being restricted to the space passing through the origin and increasing flexibility. express The activation function restricts the output range to between 0 and 1; its function is to... Convert to a gating mask, where 1 indicates that the feature is preserved and 0 indicates that the feature is suppressed; This represents the feature vector of the gated branch, which retains only the local features that are more important to the current task.

[0032] In the main branch, the projected input vector is passed through The activation function is used to obtain the main branch feature vector; the main branch feature vector is then fused with the gated branch feature vector, a process that can be represented by formula (6): (6); in, express Passing through the main branch road The main branch feature vector obtained after activation function processing, This represents the fused feature vector output by the Mamba state-space model, which integrates features from historical memory and the current input, and is also the basis for subsequent dimensionality compression; the fusion of these two features ensures the output... It includes both long-term dependencies and local features.

[0033] Finally, the fused feature vector output by the Mamba state-space model is linearly projected and its dimensions compressed to standardize the output and adapt it to downstream tasks. This process can be represented by formula (7): (7); in, This represents the feature vector after linear projection and dimensional compression. , This represents two parallel linear transformation matrices that fuse the feature vectors output by the Mamba state-space model. Mapping to two different subspaces and then multiplying them element-wise enhances the nonlinear expressive power. It is the weight of the global gating demultiplexer. These are the learnable bias vectors for the globally gated branch; all of these are learnable parameters. This is the final projection matrix, its function is to map the compressed feature dimensions to the output dimensions, adjusting the compressed feature distribution to make the output more suitable for the subsequent GLM module. More on that later. The vector is concatenated with the positional encoding and prefix encoding to obtain a new vector. This new vector is then input into the normalization layer of the GLM module for distribution normalization, enabling the subsequent self-attention layer to capture the correlation between features more efficiently. After normalization, it enters the self-attention layer to model the global and local correlations of features at different positions in the sequence, complementing the Mamba state space model and enhancing the expressive power of features.

[0034] Another optimization of this invention is to replace the multilayer perceptron (MLP) in layers 20-22 of the 28-layer GLM module in the ChatGLM3 pre-trained model with a hybrid expert network (MoE). Compared with replacing it in later or earlier layers, this position has better stability during training. It is a flexible replacement method that does not significantly alter the original ChatGLM3 pre-trained model. Traditional ChatGLM3 pre-trained models use only MLP modules. However, an MLP is a dense network, activating and utilizing all parameters of the entire MLP for each input token. This results in a massive computational burden, impacting model efficiency. The core idea of ​​Hybrid Expert Networks (MoE) is to decompose a large dense network (MLP) into multiple smaller, functionally distinct expert networks. A gating network or router is introduced, allowing the gating network to determine which most relevant expert networks to process for each input token, while other expert networks remain dormant. This replacement achieves a significant increase in model parameters and knowledge capacity without increasing or even reducing computational cost, thus optimizing model performance. Furthermore, this invention employs Hybrid Expert Networks (MoE), which aim to activate only a selected subset of expert networks during each forward propagation. Sparsity is achieved by weighting the outputs of the expert networks, rather than aggregating all expert network outputs together. When data passes through the Hybrid Expert Network (MoE), it first undergoes preprocessing to ensure that the input shape and data type are compatible with the gating and expert networks, enabling the MoE to support different input formats. Then, the routing module calculates the gating score and score probability for each input feature, and dynamically selects the appropriate gating method. The process involves assigning fusion weights to several expert networks, with more important expert networks receiving higher weights. This process can be represented by formulas (8), (9), and (10): (8); (9); Condition: If belong The first k elements, (10); in, This represents a sparse gated function. Indicates the first Gating weights of an expert network, This represents the input data. The parameters representing the gating network, This indicates selecting the top k elements with the largest gating weights, and then... Normalize the weights into a probability distribution Representing noise terms and increasing randomness, this formula implements a sparse activation mechanism, selecting only the most important terms. A network of experts is used to process the current input; formula (9) is... The condition is defined such that when the condition satisfies formula (10), the original gating weights are retained. When the condition is met, it is set to -∞, because In the activation function, -∞ is replaced with 0 to achieve "hard" sparsity. This defines the conditions under which expert networks are activated; only those with the highest gating weights are activated. The expert network will be preserved, generally by activating only... To improve computational efficiency, one expert network, rather than all expert networks, can be used. In addition, increasing the number of expert networks can improve model scalability without significantly increasing the computational load. Different expert networks can be used to process data of different patterns, and noise can be added to avoid overloading some expert networks.

[0035] Then we move on to the expert network processing, which includes parallel computing and sparse activation. The core mechanism is that each expert network only processes the gated inputs, not all inputs, which can significantly reduce the amount of computation. At the same time, the parallel processing also means that each expert network runs independently and can be computed in parallel on the hardware. This process can be expressed by formula (11): (11); in, Indicates the first The network of experts has an internal three-layer structure of dimensionality increase, activation, and dimensionality reduction. Represents the input features of the expert network; Indicates the first The weights of the expert networks during the merging process are derived from the gating network. probability; This represents the final output vector after the hybrid expert network fusion, with dimensions equal to or greater than the input. Consistency facilitates subsequent residual connections. In summary, this module implements the logic of parallel expert network division of labor, dynamic selection, and weighted fusion, which can further improve the model's ability to represent complex features.

[0036] In step four, most weights are frozen using P-Tuning v2 fine-tuning technology, followed by separate training, instruction fine-tuning, and hybrid cueing strategies. P-Tuning v2 is a parameter-efficient fine-tuning method. Its core idea is to freeze all parameters of the pre-trained model, aiming to insert some additional, trainable prefix codes into each layer of the Transformer model. By training these few parameters, the model is guided to adapt to downstream tasks, making its performance approach that of full fine-tuning. Because all parameters are frozen, catastrophic forgetting during model training can be avoided, preserving the good performance of the original model. However, because the teaching quality assessment large language model of this invention adds a Mamba state space model and replaces the MLP module with a hybrid expert network (MoE), these two modules will participate in training, and their parameters are not frozen, allowing the training of these two modules to be completed smoothly.

[0037] To further verify the performance of the large language model for teaching quality assessment constructed in this invention, a comparative experiment was designed. The large model constructed in this invention was compared with three other large language models on five automatic evaluation metrics for evaluating text. The models compared included Jamba1.5, a Transformer-based model incorporating a hybrid expert network (MoE) and a Mamba state-space model, somewhat similar in architecture to the large language model for teaching quality assessment in this embodiment. ChatGLM3-6b is the ChatGLM3 pre-trained model used in this embodiment, a model entirely composed of Transformers. The last comparative model was Llama3:8B, also a standard Transformer pre-trained model, whose parameter count is very close to that of the large language model for teaching quality assessment in this embodiment, and therefore it was also used for the comparative experiment. EduAssessLM is the large language model for teaching quality assessment obtained in this embodiment. Format accuracy is used to measure whether the generated text conforms to the preset format specifications. The primary focus is on the correctness of the generated results in terms of surface-level form, such as structure, separators, and field arrangement. This is a fundamental requirement for structured output tasks (such as information extraction, code generation, and table filling), as the evaluation results of structured output are needed. Therefore, this metric is indispensable. Bloom's score accuracy is typically used to evaluate the cognitive accuracy of the generated content, i.e., whether it meets the requirements of a certain level in Bloom's taxonomy (such as "memory," "understanding," "application," etc.). This metric focuses on whether the generated text correctly answers questions or completes tasks at the expected cognitive level. A high score accuracy is also a measure of model performance, and the correctness of scores plays a crucial role in subsequent tasks. Content correctness accuracy evaluates the correctness of the generated text in terms of factual accuracy, logical coherence, or task execution. It focuses not only on surface form but also on the accuracy of the content itself, and is one of the core metrics for measuring generation quality. Field completeness measures whether the generated text covers all necessary fields or information points. It is commonly used for tasks such as filling out forms, generating reports, and completing templates, reflecting the completeness and information coverage of the generated content. Suggestion completeness reflects whether the teaching suggestions generated by the model meet the required length and format. The higher the above indicators, the better the performance. The data of this invention has been bolded for better comparison.

[0038] Table 1. Comparison of Evaluation Results of Large Models

[0039] As can be seen from the data in Table 1, the large model constructed by this invention has relatively stable effects in both the background and the foreground. In the comparison of the above five indicators, each indicator shows outstanding performance compared to other large models.

[0040] Another embodiment of the present invention provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor implements the teaching quality assessment method when executing the program.

[0041] Another embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, characterized in that the program, when executed by a processor, implements the teaching quality assessment method described above.

[0042] The above-described invention merely illustrates implementation methods of the present invention and should not be construed as limiting the scope of the invention patent, nor as imposing any form of limitation on the structure of the embodiments of the present invention. It should be noted that those skilled in the art can make various changes and improvements without departing from the concept of the embodiments of the present invention, and these all fall within the protection scope of the embodiments of the present invention.

Claims

1. A large language model teaching quality evaluation method based on Bloom's target classification system, characterized by, The method comprises the following steps: Step 1: constructing a structured prompt template according to the Bloom Taxonomy classification system; Step 2: generating a teaching quality evaluation dataset in a specified format and distribution requirement through the prompt template driving a large language model, wherein the teaching quality evaluation dataset comprises questions investigating different Bloom Taxonomy and corresponding student answer samples with diversity; Step 3: based on a ChatGLM3 pre-training model, optimizing the architecture to obtain a teaching quality evaluation large language model, comprising: adding a trainable prefix encoding before each GLM module of the ChatGLM3 pre-training model; adding a Mamba state space model after the word embedding layer and before the GLM module; replacing the multi-layer perceptron of the GLM module of part of the layers in the ChatGLM3 pre-training model with a hybrid expert network; Step 4: training the teaching quality evaluation large language model using the teaching quality evaluation dataset to obtain a final teaching quality evaluation large language model for teaching quality evaluation.

2. The teaching quality evaluation method according to claim 1, wherein the processing process of the Mamba state space model comprises: mapping the word embedding vector to the target dimension through linear projection to obtain a projected input vector; the projected input vector is divided into two branches: a main branch and a gating branch; in the main branch, the projected input vector is processed through an activation function to obtain a main branch feature vector; in the gating branch, the projected input vector is sequentially processed through a convolution layer, a selective state space model and an activation function to obtain a gating branch feature vector; the main branch feature vector and the gating branch feature vector are fused to obtain a fused feature vector; the fused feature vector is output to the subsequent GLM module after linear projection and dimension compression combined with position encoding and prefix encoding. In the gating branch:

3. The teaching quality evaluation method according to claim 2, wherein the local context features of the projected input vector are extracted through the convolution layer; the local context features are input into four independent linear transformation layers to synchronously generate four dynamic parameters required by the selective state space model, including: a dynamic state transition matrix, a dynamic input matrix, a dynamic output matrix and a dynamic direct access vector; the selective state space model performs selective state update using the dynamic state transition matrix and the dynamic input matrix, combined with the hidden state of the previous time step and the local context features of the current time step; and performs calculation based on the updated hidden state of the current time step using the dynamic output matrix and the dynamic direct access vector to obtain the final output of the selective state space model. The selective state space model utilizes four dynamic parameters and local context features and a hidden state of a previous time step to perform state updating and output calculation to obtain a final output of the selective state space model, and the final output of the selective state space model is processed by an activation function to obtain a gating branch feature vector. The gating branch feature vector is obtained by processing of an activation function.

4. The teaching quality evaluation method according to claim 3, wherein The processing process of the hybrid expert network comprises:

5. The method of claim 1, wherein, inputting the input features into a feedforward network to generate gating weights of each expert network; selecting the top k expert networks according to the gating weights and distributing the input features to the selected expert networks according to the weights; each expert network independently processes the input features to obtain an expert network output; the expert network outputs are weighted and summed according to the gating weights to obtain the final output of the hybrid expert network. The process of generating gating weights by the feedforward network comprises:

6. The method of assessing the quality of teaching according to claim 5, wherein, ​ Introducing a noise term when calculating the gating weight; Adopting Top-K sparse gating mechanism, only activating the top-k expert networks of the gating weight.

7. The teaching quality evaluation method according to claim 1, wherein The training adopts P-Tuning v2 fine-tuning technology to freeze the weights of other parts except the Mamba state space model and the mixed expert network, and inserts trainable prefix coding in front of each layer GLM module, combines separate training, instruction fine-tuning and mixed prompt setting strategy, to improve the balance and overall performance of the large language model in different cognitive level evaluation tasks.

8. The teaching quality evaluation method according to claim 1, wherein The prompt word template is constructed based on the six cognitive levels of Bloom's Taxonomy, including memory, understanding, application, analysis, evaluation and creation.

9. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor implements the teaching quality evaluation method of any one of claims 1-7 when executing the program.

10. A computer-readable storage medium having stored thereon a computer program, characterized in that, The program is executed by the processor to implement the teaching quality evaluation method of any one of claims 1-7.

Images