Test question parameter consistency detection method and system for group knowledge diagnosis

Abstract

The invention provides a test question parameter consistency detection method and system for group knowledge diagnosis. The method comprises the steps that group, student, test question and answer result data are collected and preprocessed to obtain an interaction matrix; individual knowledge distribution is constructed, group knowledge generality characteristics are obtained, and group knowledge generality and dependency are captured based on an attention mechanism; then test question parameters are coded, and the answer correct probability is calculated based on a cognitive diagnosis model; by taking variation lower bound (ELBO) maximization as a target and knowledge distribution KL divergence minimization as a constraint, iteratively updating parameters of an encoder and the like; and finally, outputting a knowledge diagnosis result and completing DIF detection through layered likelihood ratio test. According to the group knowledge diagnosis and test question function difference (DIF) detection method based on the hierarchical variation model, the problem that knowledge dependence of students in a group is ignored in traditional detection is solved, the knowledge point prediction precision of group knowledge diagnosis is improved, objectivity and precision of test question DIF detection are achieved, knowledge state distribution KL divergence is reduced, and inference efficiency is improved.

Description

Technical Field

[0001] This invention relates to the fields of machine learning and educational data mining, and in particular to a method and system for detecting the consistency of test parameters in group knowledge diagnosis. Background Technology

[0002] Knowledge diagnosis and mining, as a core technology at the intersection of intelligent educational data mining (EDM) and artificial intelligence, achieves quantitative modeling of learners' knowledge mastery status (such as binary classification "mastered / not mastered" or multi-order probabilistic assessment) and the correlation matrix between test questions and knowledge points through in-depth analysis of learners' answer sequence data and interaction behavior data. In recent years, group-level knowledge diagnosis (covering group granularity such as class and grade) has become a research hotspot. The mainstream technical paths include: jointly modeling the interaction relationship between "student-test question" and "group-test question" through the multi-task learning (MTL) framework, and improving the generalization performance of the model in cross-group scenarios by utilizing the implicit correlation between tasks; some solutions introduce graph neural networks (GNN) to construct a knowledge association graph of learners within the group (with learners as nodes and knowledge point mastery similarity or common answer records as edges) to achieve explicit modeling of group knowledge dependencies, or use data denoising algorithms (such as outlier filtering based on statistical thresholds and reconstruction of normal answer patterns driven by autoencoders) to optimize the quality of the original data.

[0003] However, existing technological systems still have key bottlenecks, failing to achieve objective and automated detection of Differential Item Functioning (DIF). In the field of educational data mining, DIF is defined as "a statistically significant difference in the performance of learners with the same level of knowledge on the same question across different groups." As a core indicator for analyzing question differences and ensuring the effectiveness of knowledge diagnosis results, its absence directly leads to diagnostic bias.

[0004] Existing DIF detection methods have the following limitations: From the perspective of model assumptions, traditional DIF detection methods are based on the assumption that learners' knowledge states are independently and identically distributed within a group. This assumption fundamentally conflicts with real-world educational scenarios, where learners' knowledge mastery exhibits a significant intra-group knowledge dependency structure (such as the synergy and correlation of knowledge point mastery). This assumption causes the model to fail to capture the complex distribution characteristics of real data, leading to model fitting bias and reducing the reliability of DIF detection results.

[0005] From the perspective of detection logic, existing methods rely on a single statistic for DIF determination, lacking stratified validation and multi-dimensional cross-validation logic. This deficiency easily leads to a high false positive rate during the detection process (such as misjudging "differences in knowledge point associations" as "question bias"), failing to effectively distinguish between "differences in group knowledge structure" and "question problems," and ultimately failing to meet the core requirement of standardized and refined analysis of question differences in knowledge diagnosis scenarios.

[0006] Traditional DIF detection methods assume that "students' knowledge states are independent within a group," ignoring the knowledge dependency structure within the group (such as the correlation of students' knowledge mastery in the same teaching environment). This leads to insufficient fitting of the model to the distribution of real answer data, limiting the accuracy of DIF detection. Traditional methods (such as Logistic Regression and MH methods) lack the logic of "hierarchical parameter verification," judging DIF only through a single model, and cannot distinguish between "item parameter bias" and "differences in group knowledge distribution," resulting in insufficient DIF detection power. Traditional variational inference (such as the VIBO framework) assumes that "students' knowledge states are independent," resulting in a high KL divergence (>0.8) between the model's posterior distribution and the real knowledge state's prior, which cannot accurately characterize knowledge dependencies within the group, affecting the objectivity of DIF detection. Summary of the Invention

[0007] To address the problems in the prior art, this application proposes a method and system for detecting the consistency of test parameters in group knowledge diagnosis. This invention designs a hierarchical variational model that captures knowledge dependencies within the group by introducing auxiliary latent variables. This invention uses an autoregressive attention layer to achieve variational approximation of the group's hierarchical structure, enhancing the flexibility of the posterior distribution. This invention constructs an end-to-end framework compatible with existing knowledge diagnosis models, supporting objective detection of DIF using hierarchical likelihood ratio tests.

[0008] A method for testing the consistency of test parameters in a group knowledge diagnostic approach, in one embodiment, includes:

[0009] Step S1: Obtain L disjoint groups, students, test questions, knowledge points, and interaction data; construct the Q-matrix of practice-to-concept relationships; label anchor questions and DIF questions; remove outlier data; divide the dataset into training set, validation set, and test set;

[0010] Step S2: Construct a student encoder consisting of a 3-layer fully connected neural network to map student answer records into variational parameters and generate an initial knowledge state distribution. ;

[0011] Step S3: Based on the second-layer feature values ​​of all students in the student encoder, generate common features of group knowledge through linear transformation and sigmoid activation, which serve as the initial input for autoregressive attention; Autoregressive attention optimization: introduce a linear attention mechanism to model knowledge dependencies among students, calculate attention weights, adjust the distribution of student knowledge states, and obtain the final knowledge distribution that integrates group associations.

[0012] Step S4: Construct a question encoder that includes question parameters, including but not limited to difficulty and discrimination.

[0013] Step S5: Construct an interactive encoder and calculate the probability that student i answers question j correctly based on the knowledge diagnostic model;

[0014] Step S6: With the goal of maximizing the variational lower bound and the constraint of minimizing the KL divergence of the knowledge distribution, iteratively update the model parameters and output the diagnostic results based on the optimized knowledge state.

[0015] Step S7: Perform DIF detection and construct an invariant model M1 where all question parameters are consistent across the group, a difficulty-variable model M2 where only the difficulty of DIF questions varies with the group, and a fully variable model M3 where both the difficulty and discrimination of DIF questions vary with the group. Based on the test set data, calculate the log-likelihood values ​​of M1, M2, and M3 respectively, and calculate the LR. u With LR n The statistical measure, p-value, is used to determine the type and significance of the DIF statement in the test item. Among these, LR... u To test whether there are group differences in the difficulty of DIF questions, the log-likelihood ratio statistic was used. n To further test whether there are additional group differences in the discrimination of DIF items, the log-likelihood ratio statistic is used, where p is a measure of LR. u / LR n The probability index of the extreme degree of the statistic represents the probability of observing the current or more extreme LRu / LRn value without the presence of DIF.

[0016] In one implementation, in step S2, the student encoder records each student's answers using a 3-layer neural network. Mapping to variational parameters Generate the initial knowledge state distribution Where N is a normal distribution, Let i represent the knowledge state vector of student i.

[0017] ;

[0018] ;

[0019] ;

[0020] W1-W3 are the network weight matrices, b1-b3 are the bias terms, sigmoid is the activation function to ensure the output is in the interval [0,1], and f1 is the output feature of the first layer of the student encoder neural network, which records the student's answers. After being input into the first fully connected neural network, the intermediate feature vector is processed by the sigmoid activation function. f2 is the output feature of the second neural network of the student encoder, which is the depth intermediate feature vector after the first layer output f1 is input into the second fully connected neural network and processed by the sigmoid activation function. Let be the mean parameter of the distribution of students' initial knowledge state. The standard deviation parameter of the distribution of students' initial knowledge state. and It is the output of the third layer of the student encoder's neural network.

[0021] In one implementation, in step S3, the overall knowledge representation h of the group is calculated as the initial input for autoregressive attention.

[0022] ;

[0023] in, Output the second layer of the network for the i-th student;

[0024] Modeling the dependencies of students' knowledge using a linear attention mechanism, the adjusted knowledge distribution is output. ;

[0025] ;

[0026] Where Qi is the knowledge query vector of student i, i is a student in the group, j is the other students in the group besides the current student i, that is, other students who provide knowledge association references for student i, and Kj is the knowledge key vector of student j.

[0027] In one implementation, in step S4, anchor questions use a shared embedding layer We to ensure that question parameters are consistent across populations; DIF questions: an additional population-specific embedding layer We′ is introduced to allow question parameters to vary with the population;

[0028] In step S5, the formula for calculating the probability of answering correctly is: , Let j be the set of knowledge points associated with question j. To assess student i's level of understanding of knowledge point c, r ij This is the core variable describing the interaction between student i and question j, and it takes the value 0 or 1. Let be the parameter set vector that describes question j, generated by the question encoder.

[0029] In one implementation, step S6, variational lower bound ELBO,

[0030] ;

[0031] in, These are auxiliary latent variables in the hierarchical variational model. Let be the set of student i's answers to all questions, consisting of the answers to individual questions. ,Right now m is the total number of test questions. It is the group affiliation label of student i. This is a global set of question parameters generated by the question encoder, integrating the core characteristic parameters of all questions. This represents the set of parameters for the variational distribution within the variational inference framework.

[0032] KL divergence of knowledge distribution

[0033] ;

[0034] in, The knowledge points that student i needs to master Let i be the knowledge state vector. Total number of dimensions The prior distribution variance set for student i's knowledge state. The standard deviation vector of the initial knowledge state distribution of student i, output by the student encoder.

[0035] A test parameter consistency detection system for group knowledge diagnosis, in one embodiment, includes,

[0036] The data acquisition module acquires L disjoint groups, students, test questions, knowledge points, and interaction data; constructs a Q-matrix representing the relationship between practice and concepts; labels anchor questions and DIF questions; removes outlier data; and divides the dataset into training, validation, and test sets.

[0037] The student encoder module constructs a student encoder consisting of a 3-layer fully connected neural network, which maps student answer records to variational parameters and generates an initial knowledge state distribution.

[0038] The group encoder module generates common features of group knowledge based on the second-layer feature values ​​of all students in the student encoder through linear transformation and sigmoid activation, which serve as the initial input for autoregressive attention. The autoregressive attention optimization introduces a linear attention mechanism to model the knowledge dependencies between students, calculates attention weights, adjusts the distribution of student knowledge states, and obtains the final knowledge distribution that integrates group associations.

[0039] The question encoder module constructs a question encoder that includes question parameters, including but not limited to difficulty and discrimination.

[0040] The interactive encoder module constructs an interactive encoder and calculates the probability that student i answers question j correctly based on the knowledge diagnosis model.

[0041] The optimization and diagnosis module aims to maximize the variational lower bound and minimize the KL divergence of the knowledge distribution. It iteratively updates the model parameters and outputs the diagnosis results based on the optimized knowledge state.

[0042] The DIF detection module performs DIF detection and constructs an invariant model M1 where all question parameters are consistent across the population, a difficulty-variable model M2 where only the difficulty of DIF questions varies with the population, and a fully variable model M3 where both the difficulty and discrimination of DIF questions vary with the population. Based on the test set data, the log-likelihood values ​​of M1, M2, and M3 are calculated respectively, and the LR is calculated. u With LR n The statistical measure, p-value, is used to determine the type and significance of the DIF statement in the test item. Among these, LR... u To test whether there are group differences in the difficulty of DIF questions, the log-likelihood ratio statistic was used. n To further test whether there are additional group differences in the discrimination of DIF items, the log-likelihood ratio statistic is used, where p is a measure of LR. u / LR n The probability index of the extreme degree of the statistic represents the probability of observing the current or more extreme LRu / LRn value without the presence of DIF.

[0043] In one implementation, the student encoder module records each student's answers using a three-layer neural network. Mapping to variational parameters Generate the initial knowledge state distribution Where N is a normal distribution, Let i represent the knowledge state vector of student i.

[0044] ;

[0045] ;

[0046] ;

[0047] W1-W3 are the network weight matrices, b1-b3 are the bias terms, sigmoid is the activation function to ensure the output is in the interval [0,1], and f1 is the output feature of the first layer of the student encoder neural network, which records the student's answers. After being input into the first fully connected neural network, the intermediate feature vector is processed by the sigmoid activation function. f2 is the output feature of the second neural network of the student encoder, which is the depth intermediate feature vector after the first layer output f1 is input into the second fully connected neural network and processed by the sigmoid activation function. Let be the mean parameter of the distribution of students' initial knowledge state. The standard deviation parameter of the distribution of students' initial knowledge state. and It is the output of the third layer of the student encoder's neural network.

[0048] In one implementation, the swarm encoder module computes a swarm-wide knowledge representation h as the initial input to the autoregressive attention.

[0049] ;

[0050] in, Output the second layer of the network for the i-th student;

[0051] Modeling the dependencies of students' knowledge using a linear attention mechanism, the adjusted knowledge distribution is output. ;

[0052] ;

[0053] Where Qi is the knowledge query vector of student i, i is a student in the group, j is the other students in the group besides the current student i, that is, other students who provide knowledge association references for student i, and Kj is the knowledge key vector of student j.

[0054] In one implementation, the question encoder module uses a shared embedding layer We to ensure consistency of question parameters across groups for anchor questions; for DIF questions, an additional group-specific embedding layer We′ is introduced to allow question parameters to vary with the group; in the interactive encoder module, the formula for calculating the probability of answering correctly is... , Let j be the set of knowledge points associated with question j. To assess student i's level of understanding of knowledge point c, r ij This is the core variable describing the interaction between student i and question j, and it takes the value 0 or 1. Let be the parameter set vector that describes question j, generated by the question encoder.

[0055] In one implementation, the optimization and diagnostic module uses the variational lower bound ELBO.

[0056] ;

[0057] in, These are auxiliary latent variables in the hierarchical variational model. Let be the set of student i's answers to all questions, consisting of the answers to individual questions. ,Right now m is the total number of test questions. It is the group affiliation label of student i. This is a global set of question parameters generated by the question encoder, integrating the core characteristic parameters of all questions. This represents the set of parameters for the variational distribution within the variational inference framework.

[0058] KL divergence of knowledge distribution

[0059] ;

[0060] in, The knowledge points that student i needs to master Let i be the knowledge state vector. Total number of dimensions The prior distribution variance set for student i's knowledge state. The standard deviation vector of the initial knowledge state distribution of student i, output by the student encoder.

[0061] The above-mentioned technical features can be combined in various suitable ways or replaced by equivalent technical features, as long as the purpose of the present invention can be achieved.

[0062] The present invention provides a method and system for detecting the consistency of test parameters in group knowledge diagnosis, which, compared with the prior art, has at least the following advantages:

[0063] 1. Improve the accuracy of knowledge point prediction in group knowledge diagnosis. This invention proposes a hierarchical variational model that explicitly models the covariance structure of students' knowledge states within a group through an autoregressive attention network, capturing shared knowledge patterns within the group. This improves the prediction of students' knowledge point mastery status and solves the fitting error problem caused by the "independence assumption" in traditional methods.

[0064] 2. Improve the objectivity and accuracy of DIF detection. This invention introduces a stratified variational likelihood ratio test, which quantifies the DIF probability by comparing the likelihood values ​​of three models of test parameters, thereby reducing the false positive rate of DIF detection, improving sensitivity, and providing a standardized detection tool for analyzing group test item differences in knowledge diagnosis.

[0065] 3. Reduce KL divergence in knowledge state distribution modeling and improve inference efficiency. This invention improves the autoregressive attention encoder by fitting the "group knowledge state full covariance matrix" through Cholesky decomposition, generating a more flexible posterior distribution and providing a more accurate parameter estimation basis for DIF detection. Attached Figure Description

[0066] The invention will now be described in more detail with reference to embodiments and the accompanying drawings.

[0067] Figure 1 A schematic diagram of the group knowledge diagnosis and DIF detection process of the present invention is shown;

[0068] Figure 2 The diagram shows the architecture of the group knowledge diagnosis and DIF detection system of the present invention;

[0069] Figure 3 The graph shows a comparison of the statistical distributions of uniform and non-uniform difference terms in the DIF detection of this invention. Detailed Implementation

[0070] The invention will now be further described with reference to the accompanying drawings.

[0071] This invention proposes a group-level DIF detection method based on Evidence Lower Bound for Knowledge Diagnosis (GIBO) to address the problem of traditional DIF detection neglecting the knowledge dependencies among students within a group. As shown in Figure 1, considering two disjoint student groups (e.g., different classes), the method captures the covariance of students' knowledge states within the groups (assuming non-independence) and determines whether a DIF exists based on the knowledge diagnosis results.

[0072] Given L disjoint groups , No. Group of students M exercises and K knowledge concepts The interaction between students and exercises is represented by a quadruple set. Here, r is the variable indicating whether the binary exercise is correct or incorrect. r=1 indicates a correct answer, and vice versa. The relationship from exercise to concept is given by the Q matrix, where... and =0 indicates practice Assess students' understanding of knowledge concepts The level of proficiency.

[0073] Definition 1 (Student Performance Prediction). Student performance prediction aims to predict student performance. Can you answer the exercise correctly in group g? That is, training a classifier to predict students proficiency and practice The parameters are determined. Model optimization is guided by the Q-matrix and the monotonicity assumption, and then the students' proficiency and practice characteristics are quantified based on the estimated parameters.

[0074] Definition 2 (Different Item Functions). Item Function Difference (DIF) represents the difference in performance of the same exercise among students of the same proficiency in different groups. In other words, DIF is a statistical attribute indicating the expected response. ,in This is called the reference group. This is called the subject group. DIF analysis aims to identify the degree to which test items are biased towards different groups with similar proficiency levels, i.e., practice attributes. Differences exist among different groups.

[0075] In one embodiment, the present invention proposes a group hierarchical knowledge diagnosis DIF detection framework based on variational inference, comprising:

[0076] Hierarchical variational model: introducing auxiliary latent variables Modeling the knowledge dependency structure of students within a group avoids the limitations of the traditional method's assumption of "independent knowledge state".

[0077] Autoregressive attention mechanism: It uses a linear attention layer to optimize variational approximation so that the posterior distribution can fit the complex knowledge structure of the group (such as the cross-dependency of multiple knowledge points).

[0078] End-to-end compatibility: The framework is compatible with existing knowledge diagnostic models (such as IRT, DINA, NeuralCD) and supports DIF analysis based on the Log-Likelihood Ratio Test (LRT).

[0079] In one embodiment, the system architecture and implementation process are as follows: Figure 2As shown. The GroupEncoder takes as input: the answer records of all students within the group g. .

[0080] Steps: Student encoder: Records each student's answers using a 3-layer neural network. Mapping to variational parameters Generate the initial knowledge state distribution .

[0081] Where N is a normal distribution. Represents the knowledge state vector of student i

[0082] ;

[0083] ;

[0084] ;

[0085] (W1-W3 is the network weight matrix, b1-b3 is the bias term, and sigmoid is the activation function, ensuring that the output is in the interval [0,1], matching the quantization range of the knowledge state "mastered / not mastered");

[0086] f1 is the output feature of the first layer of the student encoder neural network, which is the record of the student's answers. (e.g., the vector representing student i's answers to all questions, with elements 0 / 1 representing correct / incorrect answers respectively) is the intermediate feature vector processed by the sigmoid activation function after being input into the first fully connected neural network. f2 is the output feature of the second neural network of the student encoder, which is the deep intermediate feature vector processed by the sigmoid activation function after inputting the first layer output f1 into the second fully connected neural network. The mean parameter of the student's initial knowledge state distribution is one of the outputs of the third layer of the student encoder's neural network, and is related to... Together they constitute the variational parameters. The standard deviation parameter of the student's initial knowledge state distribution is another output of the third layer of the student encoder neural network, and is related to... Together they constitute the variational parameters, which are the standard deviation vectors of student i's "initial knowledge state distribution".

[0087] Group aggregation: Calculate the overall knowledge representation h of the group as the initial input for autoregressive attention.

[0088] ;

[0089] ( The output of the second layer network for the i-th student reflects the intermediate characteristics of the student's knowledge status; h represents the common characteristics of the group's knowledge, such as "the average mastery rate of knowledge points in the class").

[0090] Autoregressive attention layer: Models the dependencies of students' knowledge through a linear attention mechanism, and outputs an adjusted knowledge distribution. .

[0091] ;

[0092] Qi is the knowledge query vector for student i, which is based on the deep intermediate features of student i (the output of the second layer of the student encoder). The generated query vector is used to "describe the knowledge needs and characteristics of student i". i is a unique index of a student within the group, corresponding to the "target student whose knowledge state distribution needs adjustment", such as "the i-th student" (denoted as student i). j is the index of the reference student within the group, which is the index of other students in the group besides the current student i (the value ranges from 1 to i, representing "reference to students analyzed in the previous stage of the group in sequence"), corresponding to "other students who provide knowledge association references for student i" (denoted as student j). Kj is the knowledge key vector of student j, based on the deep intermediate features of the reference student j (…). The generated key vector is used to describe the knowledge reserves and characteristics of student j.

[0093] In one embodiment, there is an exercise encoder.

[0094] Anchor questions (standard questions with verified no DIF): We use a shared embedding layer to ensure that question parameters are consistent across groups.

[0095] Suspected DIF question: An additional population-specific embedding layer We′ is introduced, which allows question parameters (difficulty, discrimination) to vary with the population.

[0096] In one embodiment, an interaction decoder.

[0097] Calculate the probability of answering correctly using existing knowledge diagnostic models (such as IRT and DINA). That is, student i (knowledge state) In question j (parameter) The probability of "mastering related knowledge points and answering correctly" on the exam, the core formula is:

[0098] ;

[0099] Let j be the set of knowledge points associated with question j. To assess student i's level of understanding of knowledge point c, r ij As the core variable describing the interaction result between student i and test question j, it is a binary discrete variable that takes only 0 or 1 values. This is a parameter set vector generated by the "question encoder" that describes the "core characteristics of question j". The question encoder includes at least key parameters such as "difficulty" and "discrimination". That is, a vector that integrates these parameters.

[0100] In one embodiment, DIF detection (LRT test) is used.

[0101] step:

[0102] 1) Train three models:

[0103] M1 (Invariant Model): All problem parameters are consistent across populations.

[0104] M2 (Variable Difficulty Model): Only the difficulty of suspected DIF questions varies with the population.

[0105] M3 (Fully Variable Model): The difficulty and discrimination of suspected DIF questions vary with the population.

[0106] 2) Calculate the log-likelihood ratio statistic. and If the result is significant (p<0.05), then DIF is determined to exist.

[0107] To test whether there are group differences in the difficulty of DIF questions, the log-likelihood ratio statistic was used. To further test whether there are additional group differences in the discrimination of DIF items, the log-likelihood ratio statistic is used, where p is a measure of "...". / The probability index of "statistical extremity" represents the observed current (or more extreme) situation under the premise that "the null hypothesis of the model holds (i.e., no DIF)". / The probability of a value.

[0108] In one embodiment, the scenario involves analyzing the DIF (Discretionary Index) of math questions in the PISA dataset to determine whether a particular question is biased against a specific group of students.

[0109] step:

[0110] 1) Data input: Select student answer records from 10 groups and mark suspected DIF questions.

[0111] 2) Model training:

[0112] Student encoder generates initial knowledge distribution .

[0113] The autoregressive attention layer adjusts the knowledge distribution and captures the relevance of students' knowledge within the group.

[0114] 3) DIF detection:

[0115] like The significant difference indicates that there is a group-specific variation in the difficulty of the questions (Uniform DIF).

[0116] like The significant difference indicates that there are also differences in the discriminatory power of the questions (Non-uniform DIF).

[0117] 4) Interpretation of results: For example, a probability question involving the "Blackjack" scenario may result in a DIF (Different Result) among students unfamiliar with the game.

[0118] In one embodiment, the key formula and optimization objective, variational lower bound (ELBO) are:

[0119] ;

[0120] in, This is an auxiliary latent variable introduced in the "hierarchical variational model" of this invention, used to characterize the "knowledge dependency structure among students in a group" (such as the synergy and correlation of knowledge point mastery formed by students in the same class due to a shared teaching environment). Let be the set of student i's answers to all questions, consisting of the answers to individual questions. ( ∈{0,1}, where 0 = wrong answer and 1 = right answer constitutes the structure, i.e. , (m is the total number of test questions), corresponds to "all interaction records of student i" in "interaction quadruple set L of student and exercise". This is the group affiliation label for student i, used to clarify which of the L disjoint groups (such as class, grade, teaching group) student i belongs to, i.e. (G is the set of all groups). This is a global set of test question parameters generated by the "Question Encoder," integrating the core characteristic parameters of all test questions (including anchor questions and suspected DIF questions), i.e. (m is the total number of test items), where the parameter of a single test item j is... It includes "difficulty and discrimination". This is the set of parameters for the variational distribution in the "variational inference framework," encompassing the network weights (e.g., ...) of all encoders (student encoder, swarm encoder, problem encoder). ), bias terms (such as ) and parameters of the latent variable distribution (such as The distribution parameter), i.e., ϕ, is the parameter that controls the variational distribution. "The term refers to all parameters of the form."

[0121] Knowledge distribution KL divergence:

[0122] ;

[0123] in, To traverse the knowledge state vector of student i The dimensions are indexed, with each dimension corresponding to a "knowledge point that student i needs to master". In the formula, j=1 to j= The summation is achieved by iterating through all knowledge point dimensions associated with student i's knowledge state. Let i be the knowledge state vector. The total number of dimensions is essentially equal to "the total number of knowledge points associated with student i's answers to the questions" (for example, if student i answered 5 questions associated with 3 knowledge points, then...). =3). In the formula, j=1 to j= The upper limit of the summation is defined as "the range of knowledge points for which differences need to be calculated". The prior distribution variance set for student i's knowledge state corresponds to the "benchmark diffusion degree of knowledge distribution under ideal conditions" (if set to a small fixed value by default, it represents "the prior assumption that students' knowledge states are relatively concentrated and without extreme bias"). In the formula... The difference is used to calculate the "posterior distribution" (or "posterior distribution"). ) and prior distribution (in) The core term is the difference in variance over the mean square term. The standard deviation vector of the initial knowledge state distribution of student i is output by the student encoder, with each dimension corresponding to the uncertainty of student i's mastery of a certain knowledge point. The larger the value, the higher the uncertainty (e.g., fewer answer data leads to ambiguous model judgments). The smaller the value, the lower the uncertainty (e.g., sufficient answer data leads to accurate model judgment).

[0124] The specific algorithm is as follows:

[0125]

[0126] In one embodiment, such as Figure 3As shown, the performance improvement of this invention is as follows: AUC is improved by 3-5% on the PISA and MATH datasets. DIF detection: accurately identifies question bias, such as question unfairness caused by cultural background.

[0127] In one embodiment, this invention improves the accuracy of knowledge point prediction in group knowledge diagnosis. This invention proposes a hierarchical variational model that explicitly models the covariance structure of students' knowledge states within a group (such as the correlation between students' mastery of "quadratic equations" and "factorization" knowledge points in the same class) through an autoregressive attention network, capturing shared knowledge patterns within the group (such as the influence of teaching progress and textbook version). On publicly available knowledge diagnosis datasets (such as Math2015 and PISA-Math), the AUC of predicted student knowledge point mastery is improved by 3-5%, solving the fitting error problem caused by the "independence assumption" in traditional methods.

[0128] In one embodiment, this invention improves the objectivity and accuracy of DIF (Discretionary Index) detection. This invention introduces a stratified variational likelihood ratio (LLR) test, which quantifies the probability of DIF (p < 0.05 is considered significant) by comparing the likelihood values ​​of "invariant model (M1: all item parameters are consistent across groups), difficulty-variable model (M2: only suspected DIF items have variable difficulty), and fully parameter-variable model (M3: suspected DIF items have variable difficulty + discrimination)". The false positive rate of DIF detection is reduced to below 8%, and sensitivity is improved by 20%, providing a standardized detection tool for item fitting in knowledge diagnosis.

[0129] In one embodiment, this invention reduces the KL divergence in knowledge state distribution modeling, thereby improving inference efficiency. This invention improves the autoregressive attention encoder by fitting the "collective knowledge state full covariance matrix" through Cholesky decomposition, generating a more flexible posterior distribution. The KL divergence of the collective knowledge state distribution is reduced to below 0.3, and the model training convergence speed is improved by 30%, providing a more accurate parameter estimation basis for DIF detection.

[0130] In one embodiment, this invention is a group dependency modeling method based on hierarchical variational inference. It proposes a group-level latent variable *zi* to characterize the common features of group knowledge, and establishes the relationship between "student knowledge state - group commonality" through a conditional probability distribution *p(θs|zi)*. A hierarchical variational distribution is designed. The algorithm employs a Gaussian mixture model for q(zi) and an autoregressive neural network for q(θs|zi). A variational inference algorithm for group perception is developed, achieving efficient training by alternately optimizing group and student parameters.

[0131] In one embodiment, this invention integrates a variational approximation method based on an autoregressive attention mechanism. An autoregressive transformation layer based on linear attention is designed, dynamically adjusting student knowledge through attention weights. A randomized sorting training strategy is developed, randomly shuffling the order of students in each batch of training to enhance model robustness. An analytical formula for calculating the transformed Jacobian determinant is derived, enabling efficient computation of the KL divergence.

[0132] In one embodiment, this invention provides a scalable knowledge diagnostic model integration framework. It designs a standardized model interface that supports plug-and-play functionality for six mainstream knowledge diagnostic models, including IRT, DINA, and NeuralCD. A modular encoder-decoder architecture is constructed, with the encoder uniformly processing group features and the decoder adapting to different diagnostic models. A dynamic parameter sharing mechanism is implemented, allowing basic parameters to be shared across models while unique parameters are optimized independently.

[0133] In one embodiment, the present invention provides a multi-level DIF detection and interpretation method. A three-stage progressive testing process is constructed: M1 (completely unchanged), M2 (variable difficulty), and M3 (variable parameters), progressively locating the source of DIF (difficulty / knowledge point correlation); a dynamic sharing mechanism for test item parameters is designed, anchoring global sharing of test item parameters to ensure benchmark consistency, and grouping and optimizing parameters of suspected DIF test items to capture group differences; a degree-of-freedom calculation scheme based on the number of knowledge points is developed to automatically adapt to the detection needs of test items of different complexity.

[0134] In one embodiment, this invention focuses on optimizing the objective function for group knowledge diagnosis. It derives an ELBO expression incorporating autoregressive transformation, adds a Jacobian correction term to address the shortcomings of traditional ELBO in modeling knowledge dependencies, and designs a hierarchical KL divergence decomposition algorithm to split the total divergence into "group knowledge layer divergence" and "student knowledge layer divergence," optimizing them separately to improve modeling accuracy. It implements end-to-end differentiation of variational parameters, supporting automatic differentiation in TensorFlow / PyTorch, reducing the difficulty of model implementation.

[0135] While the invention has been described herein with reference to specific embodiments, it should be understood that these embodiments are merely examples of the principles and applications of the invention. Therefore, it should be understood that many modifications can be made to the exemplary embodiments, and other arrangements can be designed without departing from the spirit and scope of the invention as defined by the appended claims. It should be understood that different dependent claims and features described herein can be combined in ways different from those described in the original claims. It is also understood that features described in conjunction with individual embodiments can be used in other described embodiments.

Claims

1. A test question parameter consistency detection method for group knowledge diagnosis, characterized in that, Comprising, Step S1, obtaining L disjoint groups, students, test questions, knowledge points and interaction data, constructing exercise to concept relationship Q matrix, marking anchor questions and DIF questions, removing abnormal data, dividing training set, verification set and test set, Step S2, a student encoder composed of a 3-layer fully connected neural network is constructed to map the student answer records into the variational parameters and generate the initial knowledge state distribution , Step S3, according to the second layer feature value of all students in the group in the student encoder, generating the group knowledge common feature through linear transformation and sigmoid activation, as the initial input of autoregressive attention, adjusting the student knowledge state distribution, obtaining the final knowledge distribution fused with the group correlation, Step S4, constructing a question encoder including question parameters, which include but are not limited to difficulty and discrimination, Step S5, constructing an interaction encoder, calculating the probability of student i answering question j correctly based on the knowledge diagnosis model, Step S6, updating the model parameters iteratively based on the optimized knowledge state, and outputting the diagnosis result, Step S7, DIF detection is performed, an invariant model M1 of all question parameters consistent across groups is constructed, a difficulty variable model M2 of only the difficulty of DIF questions changing with groups is constructed, and a fully variable model M3 of the difficulty and discrimination of DIF questions both changing with groups is constructed, log likelihood values of M1, M2 and M3 are respectively calculated based on test set data, and LR u is calculated n , a statistical quantity, a p value, and the DIF type and significance of a test question are determined according to the p value, wherein LR u is a log likelihood ratio statistical quantity for testing whether there is a group difference in the difficulty of a DIF question, LR n is a log likelihood ratio statistical quantity for further testing whether there is an additional group difference in the discrimination of a DIF question, and p is a probability index for measuring the extremity of LR u / LR n n, representing the probability of observing the current or more extreme LRu / LRn value under the premise of no DIF.

2. The group knowledge diagnosis test question parameter consistency detection method according to claim 1, wherein the step S3 comprises: The characteristic is that, in step S2, the student encoder records each student's answers using a 3-layer neural network. Mapping to variational parameters Generate the initial knowledge state distribution Where N is a normal distribution, Let i represent the knowledge state vector of student i. ； ； ； W1-W3 are network weight matrices, b1-b3 are bias terms, sigmoid is an activation function to ensure the output is in the interval [0,1], f1 is the output feature of the first layer neural network of the student encoder, which is the deep intermediate feature vector processed by the sigmoid activation function after the first layer full connection neural network f2 is the output feature of the second layer neural network of the student encoder, which is the deep intermediate feature vector processed by the sigmoid activation function after the first layer output f1 is input into the second layer full connection neural network, is the mean parameter of the initial knowledge state distribution of the student, is the standard deviation parameter of the initial knowledge state distribution of the student, and is the output of the third layer neural network of the student encoder.

3. The group knowledge diagnosis test question parameter consistency detection method according to claim 1, wherein the step S3 comprises: In the step S3, the group overall knowledge representation h is calculated as the initial input of autoregressive attention, ； wherein the second layer network output for the i-th student, The dependency of student knowledge is modeled by a linear attention mechanism, outputting an adjusted knowledge distribution , ； Wherein, Qi is the knowledge query vector of student i, i is a student in the group, j is a student other than the current student i in the group, that is, other students providing knowledge correlation reference for student i, Kj is the knowledge key vector of student j.

4. The method of claim 1, wherein the test item parameter consistency of the group knowledge diagnosis is detected. In step S4, anchor questions use shared embedding layer We to ensure that question parameters are consistent across groups, and DIF questions: additional group-specific embedding layer We' is introduced to allow question parameters to vary with groups, In step S5, the answer probability calculation formula is , is the set of knowledge points associated with test question j, is the mastery degree of student i on knowledge point c, r ij is the core variable describing the interaction result of student i and test question j, taking values 0 or 1, is the parameter set vector generated by the question encoder, describing test question j.

5. The method of claim 1, wherein the detecting of the item parameter consistency of the group knowledge diagnosis is characterized by, Step S6, the variational lower bound ELBO, ； wherein, is an auxiliary latent variable that is a hierarchical variational model, is a set of answer results of student i on all test questions, which is composed of answer results of individual test questions i.e. m is the total number of test questions, is a group membership label of student i, is a set of global test parameters generated by the test question encoder, which integrates the core characteristic parameters of all test questions, is a total set of parameters of the variational distribution in the variational inference framework, The knowledge distribution KL divergence, ； where, is the knowledge point that student i needs to master, is the knowledge state vector of student i is the total dimension number of the knowledge state, is the prior distribution variance of the knowledge state of student i, is the standard deviation vector of the initial knowledge state distribution of student i output by the student encoder.

6. A test item parameter consistency detection system for group knowledge diagnosis, characterized by, Comprising, The data acquisition module obtains L disjoint groups, students, test questions, knowledge points and interaction data, constructs exercise to concept relationship Q matrix, marks anchor questions and DIF questions, removes abnormal data, and divides training set, verification set and test set, The student encoder module constructs a student encoder composed of 3 layers of fully connected neural networks, maps student answer records into variational parameters, and generates an initial knowledge state distribution, The group encoder module generates a group knowledge common feature through linear transformation and sigmoid activation according to the second layer feature value of all students in the student encoder, as the initial input of autoregressive attention, adjusts the student knowledge state distribution, and obtains the final knowledge distribution fused with the group correlation, The question encoder module constructs a question encoder including question parameters, which include but are not limited to difficulty and discrimination, The interaction encoder module constructs an interaction encoder, and calculates the probability of student i answering question j correctly based on the knowledge diagnosis model, An optimization and diagnosis module, aiming at maximizing the variational lower bound ELBO, and minimizing the knowledge distribution KL divergence, iteratively updates the model parameters, and outputs the diagnosis results based on the optimized knowledge states, DIF detection module, DIF detection, construct all the question parameters across the population consistent invariant model M1, only DIF question difficulty with population change difficulty variable model M2, and DIF question difficulty and discrimination all change with population variable model M3, based on test set data, calculate the log likelihood value of M1, M2, M3, calculate LR u and LR n statistic, p value, according to the p value to determine the test question DIF type and the degree of significance, wherein, LR u is the log likelihood ratio statistic for testing whether the difficulty of the DIF question has population difference, LR n is the log likelihood ratio statistic for further testing whether the discrimination of the DIF question has additional population difference, and p is the probability index for measuring the extreme degree of LR u / LR n statistic, represents the probability of observing the current or more extreme LRu / LRn value under the premise of no DIF.

7. The test question parameter consistency detection system for group knowledge diagnosis according to claim 6, characterized in that the group encoder module calculates the group overall knowledge representation h as the initial input of the self-attention. The application is characterized in that the student encoder module maps each student's answer record to a variational parameter through a 3-layer neural network The application is characterized in that the student encoder module maps each student's answer record to a variational parameter through a 3-layer neural network The application is characterized in that the student encoder module maps each student's answer record to a variational parameter through a 3-layer neural network The application is characterized in that the student encoder module maps each student's answer record to a variational parameter through a 3-layer neural network The application is characterized in that the student encoder module maps each student's answer record to a variational parameter through ； ； ； W1-W3 are network weight matrices, b1-b3 are bias terms, sigmoid is an activation function to ensure the output is in the interval [0,1], f1 is the output feature of the first layer neural network of the student encoder, which is the deep intermediate feature vector processed by the sigmoid activation function after the first layer full connection neural network f2 is the output feature of the second layer neural network of the student encoder, which is the deep intermediate feature vector processed by the sigmoid activation function after the first layer output f1 is input into the second layer full connection neural network, is the mean parameter of the initial knowledge state distribution of the student, is the standard deviation parameter of the initial knowledge state distribution of the student, and is the output of the third layer neural network of the student encoder.

8. The test question parameter consistency detection system for group knowledge diagnosis according to claim 6, characterized in that the group encoder module calculates the group overall knowledge representation h as the initial input of the self-attention. Qi is the knowledge query vector of student i, i is a student in the group, j is a student other than the current student i in the group, that is, other students who provide knowledge association reference for student i, and Kj is the knowledge key vector of student j. ； wherein, is the second layer network output for the i-th student, The dependency of student knowledge is modeled by a linear attention mechanism, outputting an adjusted knowledge distribution , ； wherein, An optimization and diagnosis module, aiming at maximizing the variational lower bound ELBO, and minimizing the knowledge distribution KL divergence, 9. The system for detecting item parameter consistency of group knowledge diagnosis according to claim 6, wherein, The topic encoder module, the anchor topic uses a shared embedding layer We to ensure that the topic parameters are consistent across groups, and the DIF topic additionally introduces a group-specific embedding layer We' to allow the topic parameters to vary across groups. In the interaction encoder module, the formula for calculating the probability of a correct answer is , is the set of knowledge points associated with the test question j, is the degree of mastery of knowledge point c by student i, ij is the core variable that describes the interaction between student i and test question j, taking a value of 0 or 1, is the parameter set vector generated by the topic encoder, which describes test question j.

10. The system for detecting item parameter consistency of group knowledge diagnosis according to claim 6, wherein, The knowledge distribution KL divergence, ； wherein, is an auxiliary latent variable that is a hierarchical variational model, is a set of answer results of student i on all test questions, which is composed of answer results of individual test questions i.e. m is the total number of test questions, is a group membership label of student i, is a set of global test parameters generated by the test question encoder, which integrates the core characteristic parameters of all test questions, is the total set of parameters of the variational distribution in the variational inference framework, ​ ； where, is the knowledge point that student i needs to master, is the knowledge state vector of student i is the total dimension number of the knowledge state of student i, is the prior distribution variance set for the knowledge state of student i, is the standard deviation vector of the initial knowledge state distribution of student i output by the student encoder.

Images