An education cognitive diagnosis method based on adversarial bayesian personalized ranking

Abstract

An education cognitive diagnosis method based on adversarial Bayesian personalized ranking belongs to the field of education data mining and artificial intelligence, and first acquires historical answering data of students and a Q matrix; a shared embedding space is constructed based on the Q matrix, each student is mapped into a student embedding vector, and each question and its correct answer and wrong answer are respectively mapped into a question-answer pair embedding vector; a compatibility score is calculated by using a compatibility measurement relationship, a pairwise ranking loss function is constructed based on a Bayesian personalized ranking framework; an adversarial perturbation is applied on the embedding vector and an adversarial loss function is constructed; two kinds of loss functions are fused to obtain a total loss function, the embedding vector is iteratively updated by minimizing the total loss function, and a cognitive diagnosis result is obtained. By introducing an adversarial training mechanism, the robustness and prediction accuracy of the cognitive diagnosis model in a sparse and noisy data environment are improved, and the explainability of the diagnosis result is ensured by the Q matrix.

Description

Technical Field

[0001] This invention belongs to the fields of artificial intelligence, educational data mining and personalized learning technology, and specifically relates to an educational cognitive diagnosis method based on adversarial Bayesian personalized ranking. Background Technology

[0002] With the rapid development of internet technology and artificial intelligence, online education platforms have been widely used globally. These platforms have accumulated massive amounts of student interaction data, including students' answer records, learning time, and click behavior. How to fully utilize this data to accurately diagnose students' knowledge mastery status—that is, cognitive diagnosis—is one of the key technologies for achieving personalized teaching, adaptive learning, and intelligent recommendations.

[0003] The core objective of cognitive diagnosis is to infer students' underlying knowledge structure and ability level from their responses, thereby providing each student with targeted learning suggestions and resource recommendations. An ideal cognitive diagnosis model should simultaneously possess three characteristics: high predictive accuracy, strong robustness, and good interpretability. Existing cognitive diagnosis models are mainly classified into the following categories: (I) Traditional cognitive diagnostic model Examples of models based on deterministic input noisy AND gate (DINA) and multidimensional item response theory (MIRT) include these two approaches. These models have a solid foundation in psychological theory and offer strong interpretability through explicit Q-matrices (item-knowledge point correlation matrices), allowing teachers and students to intuitively understand students' mastery of various knowledge points. However, traditional models typically assume that student responses follow a specific probability distribution. When faced with large-scale, highly sparse real-world educational data, their parameter estimation becomes unstable, leading to a significant drop in predictive performance. Furthermore, these models struggle to effectively handle noisy data in student responses (such as random guesses and errors), exhibiting poor robustness.

[0004] (II) Model based on matrix decomposition For example, the matrix factorization (MCD) model for cognitive diagnosis can alleviate the data sparsity problem to some extent by decomposing the student-question interaction matrix into low-dimensional latent factors. However, the matrix factorization model lacks explicit modeling of the knowledge concept structure, and its latent factors are often difficult to interpret and cannot be directly mapped to specific knowledge points, thus limiting its application value in educational scenarios.

[0005] (III) Cognitive Diagnostic Model Based on Deep Learning In recent years, deep learning methods, represented by Neural Cognitive Diagnosis Models (NCDM), have been introduced into the field of cognitive diagnosis. These models leverage the powerful fitting capabilities of neural networks to capture the complex nonlinear interactions between students and questions, achieving significant improvements in prediction accuracy. However, deep learning models are often considered "black boxes," as their internal decision-making processes are difficult to interpret, leading to low trust from teachers and students in their diagnostic results. Furthermore, deep models have high requirements for data quality and quantity; in scenarios where educational data is generally sparse and noisy, they are prone to overfitting, limiting their generalization ability.

[0006] (iv) Cognitive Diagnostic Model Based on Bayesian Personalized Ranking Bayesian Personalized Ranking (BPR) is an optimization framework based on pairwise ranking, originally applied in recommender systems. In recent years, researchers have introduced BPR into cognitive diagnostics, proposing models such as CD-BPR. Unlike traditional methods that directly predict answer probabilities, the BPR framework learns by comparing students' relative preferences for correct and incorrect answers, making it naturally suitable for handling the sparsity and orderliness of educational data. However, existing BPR-based cognitive diagnostic models remain vulnerable to noisy data: when students engage in random answering, fatigue-based answering, or error-based operations, the model tends to learn these abnormal patterns as normal patterns, leading to diagnostic results that deviate from the student's true ability level.

[0007] In summary, existing cognitive diagnostic methods struggle to balance prediction accuracy, robustness, and interpretability, especially when dealing with sparse and noisy educational data, where model stability and generalization ability urgently need improvement. Therefore, there is a pressing need for a cognitive diagnostic method that can significantly enhance robustness while maintaining interpretability to address the complex data environments in real-world educational scenarios. Summary of the Invention

[0008] To overcome the problems of insufficient robustness, poor interpretability, and low prediction accuracy under sparse data in existing cognitive diagnostic technologies, this invention proposes an educational cognitive diagnostic method based on adversarial Bayesian personalized ranking, which achieves high-precision, robust, and interpretable personalized cognitive diagnosis.

[0009] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is as follows: An educational cognitive diagnostic method based on adversarial Bayesian personalized ranking includes the following steps: Step 1: Obtain students' historical answer data and the Q-matrix between questions and knowledge concepts; Step 2: Construct a shared embedding space based on the Q matrix, map each student to a student embedding vector, and map each question and its corresponding correct and incorrect answers to question-answer pair embedding vectors respectively; Step 3: Calculate the compatibility score using the compatibility metric relationship between the student embedding vector and the question-answer pair embedding vector. Construct a pairwise ranking loss function based on the Bayesian personalized ranking framework, ensuring that the compatibility score of the correct answer is higher than that of the incorrect answer. Step 4: Apply adversarial perturbations to the student embedding vector and the question-answer pair embedding vector, and construct an adversarial loss function based on the perturbated embedding vector; Step 5: Fuse the pairwise ranking loss function with the adversarial loss function to obtain the total loss function, and iteratively update the embedding vector by minimizing the total loss function to obtain a robust cognitive diagnosis result.

[0010] Furthermore, in step 1, the historical answer data includes multiple student answer records, each answer record being represented as a triple. ,in, Indicates student identification. Indicates the title identifier. Students For the question The answer results; the Q matrix is ​​the correlation matrix between questions and knowledge concepts, where the rows of the matrix correspond to questions, the columns correspond to knowledge concepts, and the matrix elements are used to indicate whether the corresponding questions test the corresponding knowledge concepts.

[0011] Furthermore, in step 2, the dimension of the shared embedding space is consistent with the total number of knowledge concepts in the Q matrix; the student embedding vector is represented as... , among which, the Dimensional value represents students For the The degree of mastery of each knowledge concept; the question-answer pair embedding vector includes the correct answer embedding vector. and incorrect answer embedding vector , respectively representing the title The characteristic requirements for each knowledge concept when the answer is correct and incorrect, among which... This represents the total number of knowledge concepts.

[0012] Furthermore, in step 3, the compatibility metric is a negative squared Euclidean distance, and the compatibility score is calculated using the following formula: (1); in, Students For the question The answer result is The compatibility score at that time Embed vectors for students, For question-answer pair embedding vectors, It represents the square of the second norm of a vector.

[0013] Preferably, in step 3, the pairwise sorting loss function is expressed as: (2); in, For the training sample set, and Students For the question The correct answer and wrong answers Compatibility score, It is the sigmoid activation function. The regularization coefficient is . For the set containing all student embedding vectors and question-answer pair embedding vectors, This represents the square of the second norm.

[0014] In step 4, the adversarial perturbation is generated using the Fast Gradient Sign Method (FGSM), and is represented as follows: (3); in, , This is the upper limit of the disturbance amplitude. For counter-loss, For the perturbation applied to the embedding vector, The norm of a vector.

[0015] In step 4, the adversarial loss function is expressed as: (4); in, For students, For the title, The correct answer is... This is an incorrect answer. This is the set of basic parameters for the model. For the aforementioned adversarial perturbation, The regularization coefficient is . and Students For the question The correct answer and wrong answers Predicted score given embedding parameters.

[0016] In step 5, the fusion is a weighted summation, and the total loss function is expressed as: (5); in, Let be the pairwise sorting loss function. Let be the adversarial loss function. This is the balance coefficient.

[0017] In step 5, the embedding vector is iteratively updated using the stochastic gradient descent method, as follows: (6); in, This is the set of basic parameters for the model. For learning rate, Let be the total loss function.

[0018] In step 5, a multi-response learning strategy is adopted, in which multiple student-question interaction samples are sampled simultaneously in each iteration to form a training batch, and the total loss function is jointly optimized based on the training batch to enhance the learning stability under sparse data.

[0019] In step 5, the iterative update employs a minimax optimization paradigm for adversarial training. This process involves continuously generating adversarial perturbations against the current model parameters within a preset upper limit for the perturbation amplitude, in order to maximize the pairwise ranking loss function. Simultaneously, the basic model parameters and adversarial perturbations are jointly optimized to minimize the total loss function. The process is repeated iteratively until convergence.

[0020] The beneficial effects of this invention are: 1. High Robustness: Addressing the diverse noise sources in educational data (such as random guessing, fatigued responses, and operational errors), this invention introduces an adversarial training mechanism for the first time within the BPR pairwise ranking framework. By applying FGSM adversarial perturbations to student embedding vectors and question-answer pair embedding vectors, and jointly optimizing the ranking loss and adversarial loss, the model maintains stable ranking discrimination ability under various noise interferences, solving the problem of insufficient robustness of traditional BPR models in educational noise environments.

[0021] 2. Strong interpretability: This invention uses Q-matrix to explicitly model the relationship between questions and knowledge concepts, and aligns the dimensions of the shared embedding space with the total number of knowledge concepts, so that each dimension of the student's embedding vector directly corresponds to the degree of mastery of a specific knowledge concept. This design differs from the latent factor model commonly used in recommendation systems, giving the diagnostic results natural semantic interpretability, making it easier for teachers and students to intuitively understand the mastery of each knowledge point.

[0022] 3. High Predictive Performance: This invention employs a Bayesian personalized ranking framework, learning by comparing the relative compatibility scores of correct and incorrect answers, thus avoiding the inherent modeling difficulties of directly predicting absolute probabilities. Furthermore, this invention uses adversarial training as a regularization constraint within the BPR framework; the two work synergistically to significantly improve the model's predictive stability and diagnostic accuracy in highly sparse educational data scenarios.

[0023] 4. Strong applicability: The method framework of this invention fully considers the characteristics of educational data such as sparsity, noise diversity and hierarchical knowledge structure. It is applicable to subject scenarios with clear knowledge structure and hierarchical dependencies, such as mathematics, physics and programming. It can also be extended to other application tasks that need to infer user status from sparse interactive data, and has good versatility and transferability. Attached Figure Description

[0024] Figure 1 This is an overall flowchart of the present invention; Figure 2 A comparison of the accuracy convergence curves of different models on the ASSIST09 dataset; Detailed Implementation

[0025] The preferred embodiments of the present invention will now be described in detail with reference to the accompanying drawings, so that the advantages and features of the present invention can be more easily understood by those skilled in the art, thereby providing a clearer and more explicit definition of the scope of protection of the present invention.

[0026] Reference Figure 1 and Figure 2 An educational cognitive diagnostic method based on adversarial Bayesian personalized ranking includes the following steps: Step 1: Obtain students' historical answer data and the Q-matrix between questions and knowledge concepts; The historical response data includes multiple student response records, each represented as a triplet. ,in, Indicates student identification. Indicates the title identifier. Students For the question The answer results (1 indicates correct, 0 indicates incorrect); the Q matrix is ​​the correlation matrix between the questions and the knowledge concepts, represented as... ,in The total number of questions. Total number of knowledge concepts; matrix elements Indicates the topic The test examined knowledge concepts , This indicates that it was not examined.

[0027] In this embodiment, the publicly available educational data mining dataset ASSISTments2009 (hereinafter referred to as ASSIST09) is used for validation. This dataset contains student identifiers, question identifiers, answer results, and Q-matrices between questions and knowledge concepts. The raw data is cleaned to remove missing values ​​and duplicate records.

[0028] Step 2: Construct a shared embedding space based on the Q matrix, map each student to a student embedding vector, and map each question and its corresponding correct and incorrect answers to question-answer pair embedding vectors respectively; The dimension of the shared embedding space is consistent with the total number of knowledge concepts in the Q matrix, i.e., the dimension. The student embedding vector is represented as: , among which, the Dimensional value represents students For the first The degree of mastery of each knowledge concept; the question-answer pair embedding vector includes the correct answer embedding vector. and incorrect answer embedding vector , respectively representing the title The characteristic requirements of each knowledge concept when the answer is correct and incorrect.

[0029] In this embodiment, the embedding dimension of the ASSIST09 dataset... All embedding vectors are randomly initialized using a Gaussian distribution with a mean of 0 and a standard deviation of 0.01.

[0030] Step 3: Calculate the compatibility score using the compatibility metric relationship between the student embedding vector and the question-answer pair embedding vector. Construct a pairwise ranking loss function based on the Bayesian personalized ranking framework so that the compatibility score of the correct answer is higher than that of the incorrect answer. The compatibility metric is the negative squared Euclidean distance, and the compatibility score is calculated using the following formula: (1); in, Students For the question The answer result is The compatibility score at that time Embed vectors for students, For question-answer pair embedding vectors, The square of the second norm of a vector; For each training sample Sample an incorrect answer As negative samples, construct the pairwise ranking loss function: (2); in, For the training sample set, It is the sigmoid activation function. The regularization coefficient is . This is a set containing all student embedding vectors and question-answer pair embedding vectors. The value of needs to take into account both the sparsity and noise level of the educational data. When the data sparsity is high, the model is prone to overfitting to a small number of observation samples; in this case, the value should be appropriately increased. To enhance the regularization effect and prevent the embedded vector from overfitting noise, the regularization value can be appropriately reduced when the data density is high. To retain more of the model's expressive power. In this embodiment, considering the extreme sparsity of the ASSIST09 dataset, the following settings are adopted: =1 to achieve the best balance between model fitting ability and generalization ability.

[0031] Step 4: Apply adversarial perturbations to the student embedding vector and the question-answer pair embedding vector, and construct an adversarial loss function based on the perturbated embedding vector; The adversarial perturbation is generated using the Fast Gradient Sign Method (FGSM), and is represented as follows: (3); in, , This is the upper limit of the disturbance amplitude. For counter-loss, For the perturbation applied to the embedding vector, The norm of a vector. The settings need to consider the diversity and intensity of noise in educational data. Noise in educational data typically manifests as random fluctuations in student responses, with relatively small amplitude but high frequency. If... Setting the value too high can cause adversarial perturbations to mask the true response patterns, making it difficult for the model to learn effective diagnostic features; if... If the setting is too small, it will not be able to form effective adversarial training for the model, and the improvement in robustness will be limited. In this embodiment, after grid search optimization on the validation set, the setting is... =0.5 to achieve the best balance between robustness and diagnostic accuracy.

[0032] Based on the perturbed embedding vector, construct an adversarial loss function: (4); in, For students, For the title, The correct answer is... This is an incorrect answer. This is the set of basic parameters for the model. For the aforementioned adversarial perturbation, The regularization coefficient is . and Students For the question The correct answer and wrong answers Predicted score given embedding parameters.

[0033] Step 5: Fuse the pairwise ranking loss function with the adversarial loss function to obtain the total loss function, and iteratively update the embedding vector by minimizing the total loss function to obtain a robust cognitive diagnosis result; The fusion is a weighted summation, and the total loss function is expressed as: (5); in, Let be the pairwise sorting loss function. Let be the adversarial loss function. This is the balance coefficient. In this embodiment, the balance coefficient... .

[0034] The embedding vector is iteratively updated using stochastic gradient descent, as follows: (6); in, This is the set of basic parameters for the model. For learning rate, Let be the total loss function. The settings need to consider the sparsity of educational data and the dimensionality of the embedding space. The high sparsity of educational data means that only a small number of samples participate in gradient calculation in each iteration. If the learning rate is too large, the gradient update direction will be unstable, making it difficult for the model to converge; if the learning rate is too small, the training speed will be too slow, and it will easily get stuck in local optima. In this embodiment, the embedding dimension... =64, which is a medium-sized embedding space. Set the learning rate... A value of 0.05 achieves a good balance between convergence speed and training stability.

[0035] A multi-response learning strategy is adopted, in which 256 student-question interaction samples are sampled simultaneously in each iteration to form a training batch. The total loss function is jointly optimized based on the training batch to enhance the learning stability under sparse data.

[0036] Furthermore, this embodiment employs a multi-response learning strategy, simultaneously sampling multiple student-question interaction samples to form a training batch during each iteration, and jointly optimizing the total loss function based on the training batches to enhance learning stability under sparse data.

[0037] The iterative update employs a minimax optimization paradigm for adversarial training, specifically: within a preset upper limit for perturbation amplitude, adversarial perturbations targeting the current model parameters are continuously generated to maximize the pairwise ranking loss function. Simultaneously, the basic model parameters and adversarial perturbations are jointly optimized to minimize the total loss function. The training is iterated alternately until convergence. In this embodiment, the number of training rounds is set to 100, an early stopping mechanism is adopted (training stops if the performance on the validation set does not improve for 5 consecutive rounds), and the batch size is set to 512.

[0038] To verify the effectiveness of the technical solution of this invention, this embodiment employs 5-fold cross-validation, dividing the data into training, validation, and test sets at a ratio of 60%, 20%, and 20%, respectively. Several mainstream models are selected as baselines for comparison, including traditional cognitive diagnostic models DINA and MIRT, matrix factorization model MCD, neural network model NCDM, and BPR-based variant models CD-BPR, FairABPR, and VarBPR. Evaluation metrics cover predictive performance metrics (Accuracy, Precision, Recall, F1, ROC-AUC, RMSE) and cognitive diagnostic quality metrics (DOA, PC-ER).

[0039] like Figure 2 As shown, the accuracy of the method of this invention (WEBKID) on the ASSIST09 dataset significantly outperforms all the comparison models in terms of the trend of change in accuracy over training epochs. This invention converges rapidly in the early stages of training and reaches a stable state after approximately 15 epochs, with the final convergence accuracy significantly higher than traditional matrix factorization models (such as MCD and NCDM) and existing BPR-based variants (such as CD-BPR and FairABPR). This fully demonstrates that the invention has excellent convergence performance and training stability.

[0040] Table 1 shows the performance comparison of different algorithms on the ASSIST09 dataset; As shown in Table 1, this invention achieved superior performance across all evaluation metrics on the ASSIST09 dataset. The accuracy of this invention reached 0.746, and the ROC-AUC reached 0.795, both significantly outperforming the second-best baseline model. Particularly noteworthy is its Degree of Agreement (DOA), a core metric measuring consistency in cognitive diagnostic ranking, where it achieved 0.811, significantly outperforming the comparison models. In contrast, traditional Deterministic Noise Input (DINA) and Multi-User Perceptron (MIRT) models performed poorly on this dataset, while existing BPR-based variants (such as FairABPR and VarBPR) lagged behind this invention in all metrics due to a lack of robustness constraints on noisy data.

[0041] The experimental data above fully verify that the joint adversarial training framework proposed in this invention can effectively improve the prediction accuracy, ranking consistency and overall generalization ability of the cognitive diagnostic model in sparse interactive data scenarios, and has important application value.

[0042] Algorithms, calculation processes, and measurement devices not described in detail in this article are all general techniques in this field.

[0043] The above description is merely an embodiment of the present invention and does not limit the patent scope of the present invention. Any equivalent structural or procedural transformations made based on the content of the present invention's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of the present invention.

Claims

1. A diagnostic method for educational cognition based on adversarial Bayesian personalized ranking, characterized in that, The cognitive diagnostic method includes the following steps: Step 1: Obtain students' historical answer data and the Q-matrix between questions and knowledge concepts; Step 2: Construct a shared embedding space based on the Q matrix, map each student to a student embedding vector, and map each question and its corresponding correct and incorrect answers to question-answer pair embedding vectors respectively; Step 3: Calculate the compatibility score using the compatibility metric relationship between the student embedding vector and the question-answer pair embedding vector. Construct a pairwise ranking loss function based on the Bayesian personalized ranking framework so that the compatibility score of the correct answer is higher than that of the incorrect answer. Step 4: Apply adversarial perturbations to the student embedding vector and the question-answer pair embedding vector, and construct an adversarial loss function based on the perturbated embedding vector; Step 5: Fuse the pairwise ranking loss function with the adversarial loss function to obtain the total loss function. Iteratively update the embedding vector by minimizing the total loss function to obtain a robust cognitive diagnosis result.

2. The educational cognitive diagnosis method based on adversarial Bayesian personalized ranking as described in claim 1, characterized in that, In step 1, the historical answer data includes multiple student answer records, each of which is represented as a triple. ,in, Indicates student identification. Indicates the title identifier. Students For the question The answer results; the Q matrix is ​​the correlation matrix between questions and knowledge concepts, where the rows of the matrix correspond to questions, the columns correspond to knowledge concepts, and the matrix elements are used to indicate whether the corresponding questions test the corresponding knowledge concepts.

3. The educational cognitive diagnosis method based on adversarial Bayesian personalized ranking as described in claim 2, characterized in that, In step 2, the dimension of the shared embedding space is consistent with the total number of knowledge concepts in the Q matrix; the student embedding vector is represented as... , among which, the Dimensional value represents students For the first The degree of mastery of each knowledge concept; the question-answer pair embedding vector includes the correct answer embedding vector. and incorrect answer embedding vector , respectively representing the title The characteristic requirements for each knowledge concept when the answer is correct and incorrect, among which... This represents the total number of knowledge concepts.

4. The educational cognitive diagnosis method based on adversarial Bayesian personalized ranking as described in claim 3, characterized in that, In step 3, the compatibility metric is the negative squared Euclidean distance, and the compatibility score is calculated using the following formula: （1）； in, Students For the question The answer result is The compatibility score at that time Embed vectors for students, For question-answer pair embedding vectors, It represents the square of the second norm of a vector.

5. The educational cognitive diagnosis method based on adversarial Bayesian personalized ranking as described in claim 4, characterized in that, In step 3, the pairwise sorting loss function is expressed as: （2）; in, For the training sample set, and Students For the question The correct answer and wrong answers Compatibility score, It is the sigmoid activation function. The regularization coefficient is . For the set containing all student embedding vectors and question-answer pair embedding vectors, This represents the square of the second norm.

6. The educational cognitive diagnosis method based on adversarial Bayesian personalized ranking as described in claim 5, characterized in that, In step 4, the adversarial perturbation is generated using the fast gradient sign method, and is represented as follows: （3）； in, , This is the upper limit of the disturbance amplitude. For counter-loss, For the perturbation applied to the embedding vector, The norm of a vector.

7. The educational cognitive diagnosis method based on adversarial Bayesian personalized ranking as described in claim 6, characterized in that, In step 4, the adversarial loss function is expressed as: （4）； in, For students, For the title, The correct answer is... This is an incorrect answer. This is the set of basic parameters for the model. For the aforementioned adversarial perturbation, The regularization coefficient is . and Students For the question The correct answer and wrong answers Predicted score given embedding parameters.

8. The educational cognitive diagnosis method based on adversarial Bayesian personalized ranking as described in claim 7, characterized in that, In step 5, the fusion is a weighted summation, and the total loss function is expressed as: （5）; in, Let be the pairwise sorting loss function. Let be the adversarial loss function. This is the balance coefficient.

9. The educational cognitive diagnosis method based on adversarial Bayesian personalized ranking as described in claim 8, characterized in that, In step 5, the embedding vector is iteratively updated using stochastic gradient descent, as follows: （6）; in, This is the set of basic parameters for the model. For learning rate, Let be the total loss function.

10. The educational cognitive diagnosis method based on adversarial Bayesian personalized ranking as described in claim 9, characterized in that, In step 5, a multi-response learning strategy is adopted, in which multiple student-question interaction samples are sampled simultaneously in each iteration to form a training batch, and the total loss function is jointly optimized based on the training batch; The iterative update employs a minimax optimization paradigm for adversarial training. The process involves continuously generating adversarial perturbations against the current model parameters within a preset upper limit for perturbation amplitude, in order to maximize the pairwise ranking loss function. Simultaneously, the basic model parameters and adversarial perturbations are jointly optimized to minimize the total loss function. The process is repeated iteratively until convergence.

Images