A cognitive diagnosis method for long-period evaluation

Abstract

The present application belongs to the field of educational data mining, and provides a cognitive diagnosis method for long-period evaluation, comprising: (1) constructing a cognitive diagnosis framework for long-period evaluation; (2) fusing the extracted student features, test features, interaction features and time sequence features to obtain a final input representation vector; (3) using a neural network structure to model a diagnosis algorithm, taking the final input representation vector obtained in step (2) as the input of the network structure, and outputting the student answer result; the diagnosis algorithm is composed of a neural network structure and a loss function; (4) collecting a data set, training the network structure, and predicting the student answer response; (5) according to the specific application scene, designing a cognitive diagnosis system to obtain the student's diagnosis report. The present application method respectively meets the single education measurement demand without long-period evaluation data accumulation and the education diagnosis demand with long-period evaluation data accumulation, and better solves the new problems brought by the change of education data form.

Description

Technical Field

[0001] This invention belongs to the field of educational data mining, specifically relating to a cognitive diagnostic method for long-term assessment, used to intelligently diagnose learners' mastery of knowledge and skills. Background Technology

[0002] Cognitive diagnostic theory, as a new generation of educational measurement theory, models learners' cognitive processing and uncovers their potential abilities and skill levels, enabling timely feedback on learners' weak knowledge and skills. Accurate skill diagnostic results are also applied in many educational contexts, including assisting teachers in providing differentiated instruction and promoting personalized learning and resource recommendations.

[0003] Cognitive diagnostic models are an important means of achieving cognitive diagnosis, and an increasing number of researchers are dedicated to their development. Traditional cognitive diagnostic models use different learning hypotheses to probabilistically model students' answering processes, thereby diagnosing learners' skill mastery. On one hand, based on learners' skill status, it can generally be divided into two categories: latent characteristic abilities and specific knowledge skills. Cognitive diagnostic models based on learners' latent characteristic abilities are represented by item response theory, which assumes that learners' answers are influenced by their latent abilities and the difficulty of the test questions, and models students' latent cognitive abilities as continuous parameters. On the other hand, cognitive diagnostic models based on specific knowledge skill status are represented by the connected deterministic input noise AND gate model. This model models students' cognitive status as a binary discrete vector, where each dimension of the vector represents the student's mastery of a specific knowledge skill. It assumes that learners can only answer questions correctly if they master all the knowledge points tested in the test. Traditional cognitive diagnosis assesses students' skills and knowledge mastery based on their answer records, comprehensively analyzing students' learning abilities and cognitive levels. While the emergence of cognitive diagnostic theory is sufficient to meet the needs of educational diagnosis of students under ideal conditions, educational measurement models still have considerable room for improvement. For example, CDT, which is mainly combined with probability theory, suffers from low robustness to real-world applications, limited applicability (restricted to a single static exam), difficulty in handling educational data with missing values, and low overall utilization of educational data—problems that are difficult to solve solely within the scope of probability theory. Benefiting from the rapid development of next-generation information technology, the challenges of probability-based cognitive diagnosis have been readily resolved by embracing next-generation information technologies represented by deep learning.

[0004] To utilize the textual information of test questions during cognitive diagnosis, researchers have implemented an IRT model based on neural networks. To make the quantification of fuzzy abilities related to subjective questions possible, researchers have used the concept of fuzzy sets to achieve cognitive diagnosis. In the process of combining cognitive diagnosis with neural networks, much research has focused on improving students' ability to predict correct answers, without deeply exploring the intermediate products of cognitive diagnosis (i.e., students' skill mastery status). Although Neural CDM has achieved some success in diagnosing students' skill mastery, these results are somewhat vague, with low differentiation across all students' skill mastery statuses, which does not accurately reflect reality. In conclusion, while deep learning technology shows unique advantages in cognitive diagnosis, how to overcome the "black box" nature of deep learning and enhance the interpretability of the diagnostic process remains a pressing issue.

[0005] Deep learning technology has shown unique advantages in cognitive diagnosis. However, current CDA (Consciousness-Based Assessment) focuses more on students' static cognitive status, i.e., using lateral cognitive diagnosis to diagnose students based on a single test. This results in educational measurement being limited to the current moment's answer data, neglecting the process data in long-term assessments. As the proportion of online education platform data in total educational data increases year by year, the problem of historical answer records being ignored by deep learning-based CDA is becoming increasingly serious and urgently needs to be addressed. Summary of the Invention

[0006] To address the aforementioned deficiencies or improvement needs of existing technologies, this invention provides a cognitive diagnostic method for long-term assessments. It satisfies both the needs of single-test education measurement where there is no long-term assessment data accumulation and the needs of educational diagnosis where there is long-term assessment data accumulation. Furthermore, it covers diagnostic scenarios for both single and multiple tests and is applicable to both traditional and online educational datasets, effectively solving new problems arising from changes in the form of educational data.

[0007] The objective of this invention is achieved through the following technical measures.

[0008] A cognitive diagnostic method for long-term assessment includes the following steps:

[0009] (1) Construct a cognitive diagnostic framework for long-term assessment; including feature extraction, feature fusion, two-level enhanced cognitive diagnostic modeling for single assessment and cognitive diagnostic modeling for multiple assessments by fusing temporal features;

[0010] (2) The extracted student features, test question features, interaction features and time series features are integrated to obtain the final input representation vector;

[0011] (3) Use the neural network structure to model the diagnostic algorithm, take the final input representation vector obtained in step (2) as the input of the network structure, and output the student's answer; the diagnostic algorithm consists of a neural network structure and a loss function.

[0012] (4) Collect datasets, train network structures, and predict student responses;

[0013] (5) Design a cognitive diagnostic system to obtain students’ diagnostic reports based on specific application scenarios.

[0014] In the above technical solution, step (1) of “constructing a cognitive diagnostic framework for long-term assessment” specifically includes:

[0015] (1-1) Feature extraction, including extracting student features, test question features, interaction features and time series features. Student features include the level of test question mastery. Test question features include question difficulty, discrimination, and Q matrix (Q matrix is ​​used to represent the knowledge points tested by the test questions. The columns are knowledge points, the rows are test questions, and the elements are only 0 or 1. For example, if the first question tests knowledge point 1, then the first row and first column are labeled 1, and the other columns in the first row are labeled 0). Interaction features include guessing factor and error factor. Time series features include timestamp, that is, the time of answering the test questions.

[0016] (1-2) Two-level enhanced cognitive diagnostic modeling for single assessment: Starting from the students' familiarity with and difficulty of the test questions, calculate the students' mastery of the knowledge points required for the test, and then filter through error gate and guess gate to correct the students' mastery of the test questions' skills and predict the students' final score on the test.

[0017] (1-3) Based on the two-level enhanced cognitive diagnostic model for single assessment, the extracted temporal features are integrated to establish a cognitive diagnostic model for multiple assessments, which is used to label the weight of assessments at different time points on the final diagnostic result.

[0018] In the above technical solution, the specific methods for fusing features in step (2) include:

[0019] (2-1) Integrate student characteristics such as test item mastery and test item characteristics such as question difficulty, discrimination, and Q matrix using the traditional IRT model;

[0020] (2-2) After integrating the features in step (2-1), the guess parameters and error parameters are fused together to obtain features for a single evaluation;

[0021] (2-3) The features for single-test obtained in step (2-2) are fused with temporal features to obtain the final input representation vector.

[0022] In the above technical solution, the neural network structure modeling and diagnostic algorithm in step (3) specifically includes:

[0023] (3-1) Select a suitable network structure, fit the students and test questions based on the strong fitting ability of the neural network, and then construct the network structure by combining the parameter estimation method of manual modeling.

[0024] (3-2) Randomly initialize parameters, including initializing students' level of mastery of the test questions, initializing the difficulty of the test questions, and initializing guessing and error parameters;

[0025] (3-3) Apply deep residual networks and introduce residual blocks during the construction of neural networks so that the model can strengthen the input;

[0026] (3-4) The error gradient under each weight is calculated in real time, and gradient descent is used to reduce the loss function in order to optimize the parameters.

[0027] In the above technical solution, the specific method for training the network structure in step (4) includes:

[0028] (4-1) Collect three real-world datasets: PISA2015, Math, and Assist;

[0029] (4-2) In the feedback neural network structure, the cross-entropy loss function is chosen as the loss function to measure the loss between the predicted value and the true value;

[0030] (4-3) Perform backpropagation and select a method to calculate the error gradient under each weight in real time to update the parameters;

[0031] (4-4) Select the optimization algorithm optimizer.step() and the backpropagation algorithm backward() to minimize the loss function.

[0032] In the above technical solution, the cognitive diagnostic system designed in step (5) includes:

[0033] The user management module is used to enable users to upload single files, batch uploads, and query upload records.

[0034] The response data preprocessing module performs data cleaning and time-series weight labeling services on the incoming raw information;

[0035] The cognitive diagnosis presentation module uses a two-level enhanced cognitive diagnosis model for single-test assessment and a cognitive diagnosis model for multiple-test assessment that incorporates temporal features to learn from the user's input answers and corresponding annotations. It outputs a simulation matrix of the user's test mastery level and the final result of the predicted answers.

[0036] Compared with existing technologies, the cognitive diagnostic method of this invention, which is geared towards long-term assessment, has the following advantages:

[0037] 1. To address the problem that traditional cognitive diagnostic models are insufficient in modeling students' learning process data, this invention proposes a cognitive diagnostic framework for long-term assessment. Based on cognitive diagnostic theory, it represents the multidimensional features of students and test questions, as well as interaction and temporal features. Furthermore, it uses deep learning methods to model and train the multidimensional features, diagnose students' knowledge mastery status, and predict students' future performance.

[0038] 2. Cognitive diagnostic methods for long-term assessments include those for single-assessment and those for multiple-assessment. The single-assessment approach proposes two levels of enhancement: enhancing students' mastery of test items and incorporating guessing and error gates. The multiple-assessment approach integrates temporal features into the single-assessment approach. This method learns from past educational data on the same skill matrix and treats the time dimension itself as information that can influence cognitive diagnostic results, using it to label the weight of assessments at different time points on the final diagnostic outcome.

[0039] 3. A cognitive diagnostic system for long-term assessments was designed and implemented. By embedding the implemented cognitive diagnostic methods for single and multiple assessments into the system's backend service, the system is equipped with the ability to perform cognitive diagnosis and prediction on multiple time segments of answer records. Attached Figure Description

[0040] Figure 1 This is a scenario diagram for cognitive diagnosis.

[0041] Figure 2 This is a schematic diagram of the cognitive diagnostic framework for long-term assessment in this invention.

[0042] Figure 3 This is the diagnostic report page. Detailed Implementation

[0043] This invention discloses a cognitive diagnostic framework for long-term assessment. It primarily represents learners and assessment items using different methods. By fusing temporal features (time weights), it combines long-term assessment data with item and student characteristics, assigning different time weights to the student's assessment responses at time t. This combined data is then fed into a neural network to obtain the learner's current item mastery level. Specifically, it first proposes a two-level augmented diagnostic method for single assessments, utilizing fused student, item, and interaction features to learn the learner's mastery level. Next, to fully utilize educational data and integrate historical information, it proposes a cognitive diagnostic method for multiple assessments, providing interpretability to the diagnostic results of timestamped datasets. Finally, a cognitive diagnostic system for long-term assessments is designed and implemented.

[0044] To make the objectives, technical solutions, and advantages of the present invention clearer, the embodiments of the present invention will be described in further detail below with reference to the accompanying drawings.

[0045] (1) Construct a cognitive diagnostic framework for long-term assessment

[0046] A scenario diagram for cognitive diagnosis is shown below. Figure 1 As shown, student responses are used to learn the difficulty and discrimination of test questions, students' mastery of the questions, and temporal characteristics. These features, along with expert-annotated Q-matrix models, are then used to learn students' level of question mastery and their current learning state.

[0047] like Figure 2 As shown, the cognitive diagnostic framework for long-term assessment includes feature extraction, feature fusion, and modeling diagnostic algorithms. Features include student features (skill mastery level), test item features (Q-matrix, difficulty, discrimination), interaction features, and temporal features; however, temporal features are not considered in a single cognitive diagnosis. Feature fusion aggregates the extracted features to obtain the final input representation vector, which is then fed into a deep neural network for training.

[0048] (1-1) Feature Extraction

[0049] Based on their different sources, the extracted features can be categorized into: student features, test item features, interaction features, and time-series features.

[0050] (1-1-1) Student characteristics: During the assessment of student responses, each assessment result is based on the student's current level of mastery of the test questions, which in this model is the student's level of mastery of the test questions, S. i =[α1,α2,...,α k ].

[0051] (1-1-2) Test Item Characteristics: Assessment items possess diverse potential characteristics. The same student may exhibit different responses to different questions testing the same skill. The Q-matrix is ​​insufficient to directly describe test item characteristics; therefore, the Q-matrix and difficulty matrix were selected as test item characteristics. For representing the skill difficulty matrix, neural network fitting was used to extract test item difficulty features. With the powerful fitting ability of neural networks, the difficulty level can be fitted using student response data, and this difficulty level is incorporated into the backward feedback iteration. This improves the indicators while making the fitted test difficulty approach the actual test difficulty.

[0052] (1-1-3) Interaction Features: After obtaining the features of the students and the test questions themselves, it is necessary to further consider the relationship between the two. The interaction itself also contains extractable features; the guessing coefficient and the error coefficient are considered interaction features.

[0053] (1-1-4) Temporal characteristics: Time weights obviously also need to be considered. A corresponding time weight is generated only when a student answers a test item, used to indicate the weight of the impact of the current answer on the potential cognitive state at time t. By introducing time weights, previous test data can be used in diagnosis as an effective supplement to the test data at time t.

[0054] (1-2) Two-level enhanced cognitive diagnostic modeling for single assessment

[0055] To achieve accurate diagnosis, the two-level enhanced cognitive diagnosis for a single assessment consists of three parts: initialization parameters, secondary enhanced input, and deep diagnosis.

[0056] (1-2-1) Initialize parameters as tensor parameters. Initialize each student's continuous mastery of each concept. This initialization pattern provides a wider correction space for reverse feedback iteration, making students' mastery of each knowledge point closer to the true value.

[0057] (1-2-2) In the second-level augmented input, the parameters of cognitive psychology (difficulty, discrimination, Q matrix, mastery level) are fully utilized to increase and optimize the dimension of the input information (interaction features), which not only provides interpretability and theoretical support for the neural network training parameters, but also improves the training effect of the neural network.

[0058] The main function of the two-level augmentation is to combine various randomly generated parameters using the HODINA formula, and then provide the result as input to the next step. A two-layer model is used for input synthesis.

[0059] The first layer passes through α iThe Q matrix is ​​used to obtain the students' ideal mastery of the test questions. The knowledge points mastered by the i-th student are calculated in relation to the difficulty of the test questions and the relevant knowledge points for the j-th question, yielding η. ij =(α i -k j )*q j , where q j The expert annotation matrix uses values ​​of 0-1 to mark the corresponding knowledge points for the problem verification.

[0060] In the second layer, based on the influencing factors of cognitive diagnostic theory, an error gate (s) is set. j ) and guessing gate (g j The information is filtered to optimize students' understanding of the test questions, resulting in the input matrix E.

[0061]

[0062] Parameter η ij ∈[0,1] represents the degree of mastery that student I has over item J. j It's an error gate; it filters out situations where a student answers incorrectly even when they have mastered all the necessary knowledge points for the question. j It's a guessing gate that filters out situations where students answer correctly even if they haven't fully grasped all the necessary knowledge points for the question. This two-gate mechanism enhances the fit between the guessing gate and the student's actual knowledge level.

[0063] (1-2-3) Deep diagnostics, building upon the first two methods, utilizes the good fit of neural networks to predict the probability of students' correct answers. In this process, backfeedback can yield rich intermediate results, such as the student's mastery of a specific knowledge point or the difficulty of the knowledge points in the exam questions. Such intermediate conclusions are a valuable supplement to cognitive diagnostics.

[0064] (1-3) Cognitive diagnostic modeling for multiple assessments

[0065] Cognitive diagnosis for multiple assessments outputs final diagnostic predictions based on long-term diagnostic data of test takers (such as knowledge mastery status and test taker scores on predicted responses to test items). This multiple-assessment-oriented cognitive diagnosis consists of extracting temporal features, mixed input, and deep diagnosis. First, the timeline information of the dataset is processed; second, the inherent relationships between different dimensions of information in the mixed input interaction are explored; and finally, deep diagnosis is used to model the student's cognitive state. This fusion approach provides an optimized information space for the final diagnosis, filling in skill mastery gaps not covered by tests in the current time period.

[0066] Extracting temporal features: The historical information temporal weights used were obtained through linear function normalization, and the earliest answer record timestamp was used as T.min The current diagnosis time point timestamp is used as T. max For a response record to be processed with timestamp T, the time weight of that record on the current diagnostic node can be obtained through linear function normalization:

[0067]

[0068] Hybrid input: Hybrid input incorporates information from the subject, test items, and exchange relationships, and then combines this information with time-weighted information to form a unified input for deep diagnostics.

[0069] To better utilize discrete online education data, a temporal dimension of external information from student responses is introduced on top of the two-level enhanced cognitive diagnostic approach for single-assessment testing. For multiple-assessment cognitive diagnostics, three matrices are concatenated from the mixed input: a question matrix E provided by the two-level enhanced cognitive diagnostic approach for single-assessment testing, containing outputs such as the Q matrix, S / G (error / guess) matrix, and the test difficulty matrix K; a question mastery matrix S representing the student's internal factors; and a T matrix provided by the time-weighted module.

[0070] Deep diagnostics: Through appropriate network structures and feedforward and backward feedback, it adjusts response predictions to the optimal state. Deep diagnostics predicts students' performance on other assessments based on their responses to assessment items.

[0071] Deep diagnostics, comprised of a neural network structure and a loss function, accepts a final input of mixed inputs and returns the student's responses. The main task of deep diagnostics is to fit a student's knowledge and skills mastery matrix and predict the student's responses on a given assessment, assuming knowledge of the student, the assessment items, and their interaction. These two processes occur simultaneously; the neural network-based cognitive diagnostic model gradually improves the accuracy of its predictions of student responses while simultaneously driving the fitted knowledge and skills mastery matrix closer to the true mastery matrix.

[0072] (2) Different representations of fusion features

[0073] After obtaining the different representations of students, assessment items, and interaction features, it is necessary to aggregate the extracted features into a holistic feature representation to obtain the final input representation vector x. ij .

[0074] By referring to traditional cognitive diagnostic theories and combining feature representations from different dimensions, we can obtain the interpretability of neural network parameters and achieve the goal of constructing parameters based on theory.

[0075] x ij =F[S iE ij ,T ij ]

[0076] F represents a function that aggregates multidimensional features, S i E ij ,T ij These represent the student's overall mastery of knowledge and skills, the student's predicted responses on assessment items, and the student's response time on assessment items, respectively. When long-term assessment data is unavailable in the dataset, a cognitive diagnostic method oriented towards a single assessment is used, and T is ignored. ij .

[0077] (2-1) Integration of student characteristics, test item mastery level, and test item characteristics, including difficulty, discrimination, and Q-matrix features.

[0078] Student-assessment interaction representation at the input end x ij In the middle, x ij It will also contain multiple types of hidden information depending on the different inputs, x ij =F[S i E ij ,T ij ], S i =[α1,α2,...,α k ] represents the level of knowledge and skills possessed by the i-th student at time t.

[0079] Through α i The Q matrix is ​​used to obtain the students' ideal mastery of the test questions. The knowledge points mastered by the i-th student are calculated along with the relevant knowledge points tested in question j, yielding η. ij =(α i -k j )*q j , representing the response of the i-th student to the j-th knowledge skill, taking into account the student's mastery level, the difficulty of the test questions, and the Q matrix.

[0080] (2-2) Fusion of guessed parameters and error parameters

[0081] Based on the influencing factors of cognitive diagnostic theory, an error gate (s) is set up. j ) and guessing gate (g j The information is filtered to optimize students' understanding of the test questions, resulting in the input matrix E.

[0082]

[0083] Parameter η ij ∈[0,1] represents the degree of mastery that student I has over project J.

[0084] (2-3) Fusion of temporal features

[0085] Based on the timestamp of each response, the system performs linear function normalization. After converting the timestamp into time weights, it projects the weights to the appropriate dimensional space through a fully connected layer for use with mixed inputs.

[0086] Cognitive diagnosis has two tasks: (1) predicting the student's mastery of knowledge and skills at the current time t, S = [α1, α2, ..., α...]. k (2) Predict students' responses to unknown assessments belonging to the same Q matrix. Therefore, the objective function of cognitive diagnosis can be expressed as:

[0087]

[0088] Where x ij It includes information on three aspects: students, assessment items, and the interaction between the two. The student dimension has α. k The assessment items have corresponding Q values. j k represents the difficulty of the test questions. j The test questions and the sample students together constitute s j g j F represents the cognitive diagnostic function for long-term assessment, α k This represents the student's mastery of knowledge point k, and represents their mastery of knowledge and skills at time t. This represents the student's score at time t on item j that tested skill k.

[0089] (3) Construct a cognitive diagnostic network for long-term needs

[0090] (3-1) Selecting a suitable network structure

[0091] The network leverages the strong fitting capabilities of neural networks to fit both students and test questions. Methods include parameter estimation that combines fitted parameters with manual modeling, as well as end-to-end models that directly fit the entire process. The fitted parameters are diverse, ranging from traditional cognitive diagnostic parameters such as test difficulty, guessing error coefficient, and discrimination index, to emerging parameters specific to deep learning, such as test text representation and knowledge point relationship graph representation. The specific formula for the network structure is as follows:

[0092] f1=φ(w1×x T +b1)

[0093] f2 = φ(w2 × f1 + b2)

[0094] f3 = [x T [f2]

[0095] y = φ(W³ × f³ + b³)

[0096] Here, f1 and f2 are the outputs of the first and second fully connected layers, respectively, and f3 is the output of the residual network, which is also a concatenation of f2 and x. i b represents the weight parameters for each fully connected layer. i Let y be its bias parameter, and y be the final output prediction result.

[0097] (3-2) Randomly initialize parameters

[0098] The parameters are taken from the cognitive diagnostic model HODINA and represented in the neural network in an appropriate data format. Assume a skills test with J questions, testing K skills, and answered by I students.

[0099] Matrix Q = {q jk} J×K It is a correlation matrix between test questions and skills, q jk =1 indicates that question j tests skills k and q. jk =0 indicates that question j does not test skill k. Student answer matrix Y i ={y ij} I×j y ij =1 indicates that student I answered question j correctly; otherwise, y = 1. ij =0. To build the model, initialize the following parameters:

[0100] Problem initialization: Initialize the question difficulty matrix K = {k jk} j×R k jk ∈[0,1] represents the difficulty coefficient of skill K in applying problem J. Two parameter vectors S and G are randomly initialized, representing the subject's error coefficient and guessing coefficient, respectively. S=[s1,s2,...,s j ],G=[g1,g2,…,g j ] are the error coefficient and guess coefficient of test j, respectively.

[0101] Student initialization: Initialize student i's skill mastery mode α i ={α ik}, α ik ∈[0,1] represents the mastery status of student i with skill k.

[0102] (3-3) Application of deep residual networks

[0103] Deep residual networks introduce residual blocks during the construction of neural networks to enhance the input. The residual model takes X as input, passes through multiple hidden layers to obtain a mapping X2, and then concatenates X and X2 directly as a whole into the output layer.

[0104] (3-4) The error gradient under each weight is calculated in real time.

[0105] It is a classic method for training neural networks that combines optimization methods (such as gradient descent) and consists of two parts: activation propagation and weight update.

[0106] During the incentive propagation phase, each iteration consists of two steps:

[0107] 1) Input the training results into the network to obtain stimulus responses;

[0108] 2) Difference the excitation response with the corresponding output target to obtain the response error of the output layer and the hidden layer.

[0109] During the weight update phase, two steps are performed for each weight:

[0110] 1) Multiply the input excitation and response errors to obtain the weight gradient;

[0111] 2) Multiply this gradient by the learning rate, then take its inverse and add it to the weights.

[0112] (4) Collect the dataset and train the network structure.

[0113] (4-1) Collect three real-world datasets: PISA2015, Math, and Assist.

[0114] The Math dataset consists of objective and subjective questions, containing 15 objective questions and 5 subjective questions. PISA 2015 consists of 17 objective questions. Assist consists of 123 objective questions. Each dataset is represented by an educational expert using a rating matrix and a given Q matrix for the assessed skills. Both objective and subjective questions were predicted in a predictive model.

[0115] Math: Math is a dataset from a high school final math exam. It consists of answers to objective and subjective questions and is one of the traditional static single-test datasets.

[0116] PISA 2015: PISA is a globally recognized authoritative online assessment with high-quality tests. This method selects 17 binary classification items that have been scored by computer for analysis.

[0117] Assist: Assist is an open dataset that only provides student response logs and knowledge concepts corresponding to test questions.

[0118] (4-2) Choose the cross-entropy loss function as the loss function

[0119] In the structure of a feedback neural network, the cross-entropy loss function is chosen as the loss function to measure the loss between the predicted and true values, and the effectiveness of the model is demonstrated by pursuing a lower loss value. The formula for the cross-entropy loss function can be described as follows:

[0120]

[0121]

[0122]

[0123] (4-3) Perform backpropagation and select a method to calculate the error gradient under each weight in real time to update the parameters.

[0124] The preceding analysis, which integrates the difficulty and discrimination characteristics of test questions, the students' level of mastery of the test questions, and the guessing and error parameters, yields X, expressed by the following formula:

[0125]

[0126] Upon receiving the mixed input X, X is transferred to the first fully connected layer (Linear layer). X undergoes a linear mapping in the first fully connected layer to obtain z1, which is then processed by the sigmoid activation function to obtain X1. X1 is then passed to the second fully connected layer, and the above steps are repeated. After repeating the linear-sigmoid process twice, the mapped product X2 is obtained. The formula is described as follows:

[0127]

[0128] X i+1 =sigmoid(z i )

[0129] In this embodiment, backpropagation serves to update parameters for fitting, ΔW ij The formula for updating the parameters is described below:

[0130]

[0131]

[0132] variable W ij Let ΔW represent the neuron weights between i and j. ij For weight updates, η is the learning rate. X represents the partial derivative of the squared error function. i δ is the output of the current neuron. jThe error generated by neuron j in the current layer (i.e., the error between the actual value and the predicted value). The input part X leading to neuron j. i It is the output X of the upper neuron I i The weighted sum is obtained.

[0133] (4-4) Select the optimization algorithm optimizer.step() and the backpropagation algorithm backward() to minimize the loss function.

[0134] W ij =W ij +ΔW ij Therefore, W ij =W ij —ηX i δ j

[0135] (5) Design a cognitive diagnostic system to obtain students' diagnostic reports.

[0136] The system is built on a web platform using a B / S framework and consists of three modules: user management, response data preprocessing, and cognitive diagnosis. User management includes two functions: visualization of diagnostic results and user upload services (such as uploading and querying upload records). The response data preprocessing module includes two types of functions: response data cleaning and automatic time-series annotation. The cognitive diagnosis module primarily provides two functions: single-case cognitive diagnosis and global cognitive diagnosis.

[0137] The system's main functional modules include: a user management module, a response data preprocessing module, and a cognitive diagnosis module. The user management module primarily provides single upload, batch upload, and upload record query functions. Diagnostic reports processed by the cognitive diagnosis module are also displayed on the user's page. The response data preprocessing module performs data cleaning and time-series weighting services on the incoming raw information.

[0138] (5-1) User Management Module

[0139] The user management module provides users with upload-related functions and user diagnostic reports. Single upload and multiple upload functions correspond to different user identities; the user could be an individual student or an individual teacher. For the former, a single upload is usually sufficient, while for the latter, batch operations are more convenient. Similarly, considering the different user identities, the visualization of user diagnostic reports is also optimized.

[0140] (5-2) Answer Data Preprocessing Module

[0141] The answer data preprocessing module primarily provides users with data cleaning and time weight labeling services. Data cleaning mainly includes detecting consistency in answer information (e.g., negative scores or scores outside the target range), handling invalid answer information (e.g., an empty user_id field in the answer information), and handling missing values ​​(e.g., an empty score field in the answer information). Time weight labeling, based on the current system time, is calculated using a linear function normalization method and stored in the time_weight field of the corresponding user's answer information as part of the training information fusion.

[0142] (5-3) Cognitive Diagnosis Presentation Module

[0143] The cognitive diagnosis presentation module primarily utilizes a two-level enhanced cognitive diagnosis method for single-assessment testing to provide services for single-assessment cognitive diagnosis, and a cognitive diagnosis method that integrates historical data to support cognitive diagnosis for long-term assessments. The backend methods are divided into two parts: automatic training and diagnostic prediction. Automatic training refers to the continuous optimization of the existing model and optimal weight parameter files deployed on the online server based on newly uploaded response information, aiming to achieve better prediction results while ensuring that the diagnostic results are closer to the latest response results.

[0144] Diagnostic prediction refers to the process where the backend model invokes different methods to process data based on user selections. During cognitive diagnosis, the system's instantaneous state is used to summarize the simulated weights of the user information requiring diagnosis and to visualize the prediction results of the pre-selected questions. In this process, the model only performs forward propagation and parameter output; because there is no backpropagation, the current parameter matrix is ​​not updated.

[0145] like Figure 3 As shown, the user diagnostic report page summarizes information from both the test questions and the test takers. The test questions are uploaded by users, while the test taker information is generated by the cognitive diagnostic module. Regarding the test questions, in addition to basic visualization of the assessment points, the system also provides statistical information, using red, orange, and blue colors to indicate the overall performance of each knowledge point. The test takers' performance is displayed in a radar chart format, showing the individual or class's performance across various knowledge point dimensions.

[0146] Users can select different diagnostic modes by switching page tabs. A single diagnostic will output results based on the user's entire uploaded record library and matrix parameter model, while a process-based diagnostic will regenerate time weights based on a specified diagnostic date for forward feedback. Both will ultimately be displayed as radar charts and table summaries. This page provides a summary of all user responses, the score rate for each knowledge point practice, and a knowledge tree-like chart. This page allows users to view their recent knowledge point practice performance and focus on improving their skills in areas with lower average scores. The knowledge point radar chart helps users adjust their training direction and intensity based on their weaknesses.

[0147] The contents not described in detail in this specification are existing technologies known to those skilled in the art.

[0148] It should be noted that the above description of the examples of the present invention is for the purpose of explaining the technical features of the present invention in detail. Various improvements and modifications made without departing from the present invention are also protected by the present invention; therefore, the scope of protection of the present invention should be determined by the content defined in the claims of this application.

Claims

1. A cognitive diagnostic method for long-term assessment, characterized in that... The method includes the following steps: (1) Construct a cognitive diagnostic framework for long-term assessments; including feature extraction, feature fusion, two-level enhanced cognitive diagnostic modeling for single assessments, and cognitive diagnostic modeling for multiple assessments by fusing temporal features; specifically including the following steps: (1-1) Feature extraction, including extracting student features, test question features, interaction features and time series features. Student features include the level of mastery of the test questions. Test question features include question difficulty, discrimination and Q matrix. Interaction features include guessing factor and error factor. Time series features include timestamp, i.e. the time of answering the test questions. (1-2) Two-level enhanced cognitive diagnostic modeling for single assessment: Starting from the students’ familiarity with and difficulty of the test questions, calculate the students’ mastery of the knowledge points required for the test, and then filter through error gate and guess gate to correct the students’ mastery of the test questions’ skills and predict the students’ final score on the test. (1-3) Based on the two-level enhanced cognitive diagnostic model for single assessment, the extracted temporal features are integrated to establish a cognitive diagnostic model for multiple assessments, which is used to label the weight of assessments at different time points on the final diagnostic result. (2) Integrate the extracted student features, test question features, interaction features, and temporal features to obtain the final input representation vector; specifically, this includes the following steps: (2-1) Integrate student characteristics such as test item mastery and test item characteristics such as question difficulty, discrimination, and Q matrix through the traditional IRT model; (2-2) After integrating the features in step (2-1), the guess parameters and error parameters are fused together to obtain features for a single evaluation; (2-3) The features obtained in step (2-2) for single-test evaluation are fused with temporal features to obtain the final input representation vector; (3) Using a neural network structure to model a diagnostic algorithm, the final input representation vector obtained in step (2) is used as the input of the network structure to output the student's answer; the diagnostic algorithm consists of a neural network structure and a loss function; specifically, it includes the following steps: (3-1) Select a suitable network structure, fit the students and test questions based on the strong fitting ability of the neural network, and then construct the network structure by combining the parameter estimation method of manual modeling. (3-2) Randomly initialize parameters, including initializing students' level of mastery of the test questions, initializing the difficulty of the test questions, and initializing guessing and error parameters; (3-3) Apply deep residual networks and introduce residual blocks during the construction of neural networks to enhance the model's input; (3-4) The error gradient under each weight is calculated in real time, and gradient descent is used to reduce the loss function in order to optimize the parameters; (4) Collect datasets, train network structures, and predict student responses; (5) Design a cognitive diagnostic system to obtain students’ diagnostic reports based on specific application scenarios.

2. The cognitive diagnostic method for long-term assessment according to claim 1, characterized in that... The specific methods for training the network structure in step (4) include: (4-1) Collect three real-world datasets: PISA2015, Math, and Assist; (4-2) In the feedback neural network structure, the cross-entropy loss function is chosen as the loss function to measure the loss between the predicted value and the true value; (4-3) Perform backpropagation and select a method to calculate the error gradient under each weight in real time to update the parameters; (4-4) Select the optimization algorithm optimizer.step() and the backpropagation algorithm backward() to minimize the loss function.

3. The cognitive diagnostic method for long-term assessment according to claim 1, characterized in that... The cognitive diagnostic system designed in step (5) includes: The user management module is used to enable users to upload single files, batch uploads, and query upload records. The response data preprocessing module performs data cleaning and time-series weight labeling services on the incoming raw information; The cognitive diagnosis presentation module uses a two-level enhanced cognitive diagnosis model for single-test assessment and a cognitive diagnosis model for multiple-test assessment that integrates temporal features to learn from the user's input answer information and corresponding annotations. It outputs a simulation matrix of the user's test mastery level and the final result of the predicted answers.

Images