Computerized adaptive test depolarization method based on selective mixing

Abstract

The invention relates to the technical field of wisdom education, and discloses a computerized adaptive test depolarization method based on selective mixing, which comprises the following steps: constructing an interaction data set based on obtained historical answer records of examinees, and encoding to obtain answer state vectors; obtaining an ability vector representing the cognitive state of the examinee according to the answer state vector; defining a deviation alignment sample, an unbiased sample and a deviation conflict sample; selective mixing is carried out, and diversified problem characterization is generated by carrying out linear interpolation on problem parameters, so that deviation conflict samples are expanded, and a synthetic sample set is obtained; constructing final training loss including empirical loss and synthetic sample loss, and optimizing the selection network to relieve selection deviation; through cross-attribute retrieval and selective mixing, a small amount of deviation conflict samples can be fully utilized and expanded to cope with cognitive diagnosis challenges caused by unbalanced data distribution, so that more accurate cognitive diagnosis is realized, and more excellent test experience is brought to examinees.

Description

Technical Field

[0001] This invention relates to the field of smart education technology, specifically to a computerized adaptive test debiasing method based on selective hybridization. Background Technology

[0002] In the field of smart education, computer-adaptive testing is a fundamental and important task. It aims to gradually select appropriate questions for test takers based on their answering status, thereby obtaining an accurate estimate of their cognitive state during the test even when they have answered fewer questions.

[0003] While early computer adaptive testing relied on heuristic strategies such as maximum Fisher information, recent research has shifted towards data-driven approaches that learn the question selection module from large-scale interactive data, significantly improving diagnostic performance. However, biases arising from data imbalance or adaptive selection can lead to the systematic assignment of easier or harder questions, distorting ability estimation and compromising fairness. Although previous research mitigated bias by recalibrating question parameters, it neglected the impact of biased training data on the question selection module itself. The roots and consequences of selection bias remain insufficiently explored, leaving a significant gap in the development of fair and robust computer adaptive testing. Summary of the Invention

[0004] This invention provides a computerized adaptive test debiasing method based on selective hybridization, which can address the problem of bias in the question selection module, thereby enabling a more accurate diagnosis of the candidate's performance.

[0005] To solve the above-mentioned technical problems, the present invention adopts the following technical solution: A computerized adaptive test debiasing method based on selective hybridization includes: An interactive dataset is constructed based on the acquired candidates' historical answer records, and divided into a support set and a metaset. The answer state vector of the candidates at time step t is then encoded. ; Based on the answer state vector Using selection networks from support sets Select the question and pass through the The inner layer optimization minimizes the prediction loss to obtain the ability vector representing the examinee's cognitive state. ; Metaset based on the percentage of correct answers The candidates were categorized into three subsets with attributes A, B, and C, and biased aligned samples, unbiased samples, and biased conflict samples were defined. For candidates with attributes A or C... Based on the ability vector, retrieve the candidate with the most similar ability from the candidates with attribute B. ; Candidates Answer records and candidates The bias conflict samples are selectively mixed, and the problem parameters are linearly interpolated to generate diverse problem representations to expand the bias conflict samples and obtain a synthetic sample set. Constructing with experience loss and synthetic sample loss Final training loss Optimize the selection network to mitigate selection bias.

[0006] In one embodiment, an interactive dataset is constructed based on the acquired candidates' historical answer records and divided into a support set and a metaset, specifically including: Define the set of candidates as The problem set is ;in, Indicates the first One candidate, Indicates the first One question, Indicates the number of test takers. Indicates the number of questions. , ; Based on the candidates Questions answered by the candidates and the corresponding tags Building an interactive dataset , When, it indicates the first The candidates answered the questions. Incorrect answer. When, it indicates the first The candidates answered the questions. Correct answer; Interaction dataset of test takers Divided into support sets based on candidates Heyuanji Supports collection Used for inner-layer optimization of selected networks; metaset Used for outer layer optimization of the selected network.

[0007] In one embodiment, the candidate's answer state vector at time step t is encoded. Specifically, it includes: Let the answer state matrix of N candidates for M answer records be denoted as , elements in When, it indicates the first One candidate For the One question Incorrect answer; , indicating the first One candidate For the One question No answer; , indicating the first One candidate For the One question Correct answer; The i-th row vector in This represents the answer state vector of the i-th candidate at time step t. This generally refers to the answer state vector of each candidate at time step t.

[0008] In one embodiment, the step of basing the answer state vector Using selection networks from support sets Select the question and pass through the The inner layer optimization minimizes the prediction loss to obtain the ability vector representing the examinee's cognitive state. Specifically, it includes: Network selection is based on the answer state vector. Choose a question for the current test taker. ; ; in, Indicates network selection. Indicates the parameters for selecting the network; Perform inner-layer optimization by supporting sets The ability vector of the test taker is estimated by minimizing the prediction loss of the answer; ; in, This is the capability vector after inner-layer optimization using cross-entropy loss. This indicates that the i-th candidate's opinion is correct. The answer results This represents a cognitive diagnostic model that obtains the probability of a candidate answering the current question correctly. This represents the candidate's ability vector before the update. These are the problem parameters in the cognitive diagnostic model. This represents the network parameters in the cognitive diagnostic model. Represents cross-entropy loss, This represents the total number of time steps.

[0009] In one embodiment, the proportion of correct answer records is used to set the metaset. The test takers were categorized into three subsets with attributes A, B, and C, specifically including: According to the meta-set The percentage of correct answers recorded by test takers will be... Divide into three subsets with attributes A, B, and C: , , The attribute is subset of In the middle, the proportion of candidates who answered questions correctly was located at Within the range; attribute is subset of In the middle, the proportion of candidates who answered the questions correctly was at... Within the range; attribute is subset of In the middle, the proportion of candidates who answered correctly was at Within the range.

[0010] In one embodiment, the definition of bias-aligned samples, unbiased samples, and bias-conflicted samples specifically includes: Deviation alignment samples are from The tag is The sample or from The tag is The unbiased samples are from The tag is The sample or label is The sample; the biased conflict sample is from The tag is The sample or from The tag is The sample is labeled as follows: a sample is labeled as 0, indicating that the candidate in the sample answered the question in the sample incorrectly; a sample is labeled as 1, indicating that the candidate in the sample answered the question in the sample correctly.

[0011] In one embodiment, the statement refers to candidates with attribute A or C. Based on the ability vector, retrieve the candidate with the most similar ability from the candidates with attribute B. Specifically, it includes: By calculating each attribute candidates Ability Vector , and has attributes Candidates capability vector The similarity between them was used to retrieve the results. The most similar abilities ; and Similarity between for: ; For Euclidean distance.

[0012] In one embodiment, the candidate Answer records and candidates The conflicting bias samples are selectively mixed, and the problem parameters are linearly interpolated to generate diverse problem representations, thereby expanding the conflicting bias samples and obtaining a synthetic sample set, which specifically includes: With the test takers Candidates with the most similar abilities Choose one question from your answer history. ,exist The problem of biased conflicting sample sampling with the same label Linear interpolation is performed on the parameters of these two problems to generate diverse problem representations. : ; Among them, the weighting coefficient , Controlling the interpolation intensity, for The parameters, for The parameters, This represents the Beta distribution.

[0013] In one embodiment, the construction includes empirical loss. and synthetic sample loss Final training loss Optimize network selection, specifically including: Constructing experience loss :

[0014] This indicates the total number of test takers; Represents cross-entropy loss, Indicates the first The candidates' questions The answer results This represents a cognitive diagnostic model that obtains the probability of a candidate answering the current question correctly. For the question The parameters, This represents the network parameters in the cognitive diagnostic model; Constructing synthetic sample loss :

[0015] Indicates the problem and the problem The problem of synthesis; Indicates a composite sample set; Construct the final outer layer optimization loss : ; express The coefficient.

[0016] Compared with the prior art, the beneficial technical effects of the present invention are: This invention addresses the data imbalance problem by introducing a cross-attribute candidate retrieval strategy. By retrieving the most similar reference object to the biased candidate from the balanced candidate group, it successfully compares the biased candidate with the balanced data, thus optimizing the distribution of the training data.

[0017] This invention enhances the diversity of training data by introducing a selective hybrid regularization strategy and smooths the decision boundary of the model by generating bias conflict samples. This strategy effectively reduces the overfitting of the model to bias patterns and improves the generalization ability and robustness of the problem selection module. Attached Figure Description

[0018] Figure 1 This is a flowchart illustrating the selective hybridization method for removing biases in computerized adaptive testing, as described in this invention. Detailed Implementation

[0019] A preferred embodiment of the present invention will now be described in detail with reference to the accompanying drawings.

[0020] This embodiment presents a computerized adaptive test bias removal method based on selective mixing, inspired by the mixing sample strategy in invariant learning. It learns to generate biased conflicting samples through selective mixing and aligns them with underrepresented group data through cross-attribute candidate retrieval and label consistency constraints, thereby effectively alleviating the data imbalance problem.

[0021] This method enhances the diversity of training data and improves the generalization ability of the question selection module by smoothing the decision boundary, thereby improving the accuracy of cognitive diagnosis and the robustness of the model. First, by acquiring candidates' historical answer records and constructing heterogeneous data, after encoding candidate states, questions are selected from the support set for inner-layer optimization. Next, candidates in the meta-set are classified and cross-attribute candidate retrieval is performed. Then, the retrieved interactions are selectively mixed with other interactions to expand the sample of biased conflict. Finally, a training loss is constructed and question selection is optimized. This method effectively addresses the challenges of cognitive diagnosis caused by data imbalance, improving the accuracy of cognitive diagnosis and the testing experience.

[0022] Specifically, such as Figure 1 As shown, a computerized adaptive test bias removal method based on selective hybridization in this embodiment is performed according to the following steps: Step 1: Obtain the candidate's historical answer records and construct heterogeneous data to encode the candidate's status. : Define the set of candidates as The problem set is ;in, Indicates the first One candidate, Indicates the first One question, Indicates the number of test takers. Indicates the number of questions. , .

[0023] Interaction dataset of test takers Divided into support sets based on candidates Heyuanji These two sections are used to select the questions the candidate will answer and to optimize the question selection, respectively. The time step records the candidate's answer status matrix for each question as follows: ,in, , 0、 They represent the first One candidate For the One question Incorrect answer, no answer, correct answer; the candidate's current answer state vector is , the answer state vector Decompose into binary pairs Set, let Any pair of tuples Indicates the candidate's answer to the question. One question The corresponding answer tags .

[0024] Interactive dataset of test takers It is for students A set of units, where each student The interactive data is determined by the questions it answers. and the corresponding tags The set of sequences that are formed together When, it indicates the first The candidates answered the questions. Incorrect answer. When, it indicates the first The candidates answered the questions. The answer is correct.

[0025] Step 2: Based on the answer state vector From support set Select a question, perform inner-layer optimization, and update the candidate's cognitive state: Step 2.1: Based on equation (1), select the network according to the answer state vector. Select appropriate questions for the current test takers : (1) in, Indicates network selection. This indicates the parameters for selecting the network.

[0026] In this embodiment, the selection network first maps the input state to the latent space through a linear layer, then through two fully connected layers (with a Tanh activation function in between), and finally outputs the problem selection probability. The problem with the highest probability is selected as the appropriate problem. .

[0027] Step 2.2: Perform inner layer optimization. According to equation (2), by optimizing the support set... Minimize the prediction loss of the answer to estimate the candidate's ability vector. : (2) in, The inner layer's updated capability vector is optimized using cross-entropy loss. Indicates the first For each candidate The answer results When, it indicates the first The candidates answered the questions. Incorrect answer. When, it indicates the first The candidates answered the questions. Correct answer; This indicates that a cognitive diagnostic model can be used to obtain the probability of a candidate answering the current question correctly. This represents the candidate's ability vector before the update. These are the problem parameters in the cognitive diagnostic model. This represents the network parameters in the cognitive diagnostic model.

[0028] Step 3, for the metaset Middle school students are categorized, and cross-attribute student searches are performed: Step 3.1, based on the metaset The percentage of correct answers recorded by test takers was divided into three subsets: , , These subsets respectively reflect different distributions within the meta-set. Let... This represents a collection of attribute categories. The attributes are... The percentage of candidates who answered correctly was located at Within the range, incorrect answers predominated; the attribute was... The percentage of candidates who answered correctly was located at Within the range, the number of correct and incorrect answers is roughly balanced; the attribute is The percentage of candidates who answered correctly was located at Within the range, the correct answers predominate.

[0029] Step 3.2: Define bias-aligned samples, unbiased samples, and bias-conflicted samples: Defining bias-aligned samples as samples that follow a dominant attribute pattern, such as those from... The tag is The sample or from The tag is The unbiased samples are from The tag is The sample or label is The samples; biased conflict samples are those that deviate from the dominant attribute pattern and are more difficult to learn, such as those from... The tag is The sample or from The tag is The sample.

[0030] Step 3.3: Perform cross-attribute candidate retrieval: In real-world computer adaptive testing scenarios, training data is often affected by selection bias. For example, in Candidates in the middle are more likely to answer incorrectly, while... Test takers tend to answer correctly in this context. This leads to bias-aligned interactions dominating, which, while easy to fit, are prone to shortcut learning and reduce the model's robustness. In contrast, bias-conflicting cases (e.g., from...) The correct answer or from Incorrect answers (though rare) are crucial for achieving fairness and the model's generalization ability. To address this issue, this invention utilizes data from... By leveraging the candidate data and their balanced response patterns as a reference, this effectively enhanced the representation of these underrepresented, biased, and conflicting samples, thus promoting the implementation of selective mixed design. Specifically, for each candidate with attributes... Candidates Based on the ability vector estimated by the cognitive diagnostic model Retrieve from attributes Candidates It possesses [something related to the examinee] The ability vector with the most similar abilities The similarity of abilities among candidates is calculated according to formula (3): (3) Step 4: To further mitigate selection bias, this invention proposes a selective fusion enhancement strategy. Unlike traditional training methods that rely solely on natural samples, fusion generates synthetic interactions through interpolation instances, significantly enriching the diversity of biased conflict data. This is particularly important in computer adaptive testing, as the scarcity of biased conflict samples often leads the problem selection module to overfit bias alignment patterns and ignore a few interactions. Fusion offers several advantages in this case: it smooths the decision boundary, reduces overfitting, and improves the model's generalization ability when there are differences between the training and test distributions. Furthermore, fusion encourages the model to learn more balanced and robust representations by diluting spurious correlations in the original data. Through this targeted enhancement, the problem selection module can move towards fairer, more consistent, and more reliable predictions, thereby enhancing robustness and generalization ability. Specifically, the data obtained in Step 3... Interaction and Selective mixing of interactions to expand biased conflict samples; from A problem in the answer record ,Give The problem of biased conflicting sample sampling with the same true label Based on equation (4), linear interpolation is performed on the parameters of these two problems to generate diverse problem representations. ; (4) Among them, the weighting coefficient , Control the interpolation intensity.

[0031] Step 5: Construct the training loss: Step 5.1: Construct the experience loss based on equation (5). : (5) Step 5.2: Construct the synthetic sample loss according to equation (6). : (6) in, This represents the synthesized sample set.

[0032] Step 5.3: Construct the final outer layer optimization loss according to equation (7). : (7) in Indicates loss The coefficient.

[0033] Step 6: Train the selected network using the Adam optimizer and minimize the loss function. This is done by updating the selection network parameter set until convergence, thereby mitigating selection bias in the computer adaptive testing model, improving the generalization of the selection network, and obtaining more accurate estimates of the candidates' abilities.

[0034] Example: To verify the effectiveness of this method, experiments were conducted on two commonly used datasets in smart education: ASSIST0910 and NIPS-EDU. Table 1 summarizes the statistical information of these datasets, including the number of examinees, the number of questions, the number of knowledge points, and the number of interactions. In the experiment, for ASSIST0910, records with missing values ​​were removed, and examinees with fewer than 40 interactions were excluded. The results are summarized in Table 1 to ensure that each examinee has sufficient data for diagnostic analysis.

[0035] Table 1. Statistics from Matmat, Junyi, and NIPS 2020EC:

[0036] To evaluate the model's effectiveness in predicting and correcting student scores, this invention employs two complementary metrics: worst-case group accuracy and overall average accuracy. The worst-case group accuracy measures the model's robustness by focusing on the most unfavorable group, while the overall average accuracy characterizes the overall predictive performance. The experimental setup includes multiple representative baseline methods and comprehensively evaluates the model across two types of cognitive diagnostic models and two data-driven computerized adaptive testing frameworks.

[0037] Table 2. Prediction results of test taker performance using our method and the comparison method on the ASSIST0910 dataset:

[0038] Table 3. Prediction results of our method and the comparison method on the NIPSEDU dataset for test taker performance.

[0039] The results in Tables 2 and 3 demonstrate that this invention primarily enhances robustness by improving the performance of disadvantaged groups while maintaining competitive overall accuracy. This aligns with the main design goal of the bias reduction framework, which is to reduce systematic differences between different groups. Experimental results show that the method proposed in this invention significantly outperforms numerous comparative methods in most cases, thus fully demonstrating the feasibility of the proposed method.

[0040] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the invention. The terms “comprising,” “including,” etc., as used herein indicate the presence of the stated features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0041] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0042] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or essential characteristics. Therefore, the embodiments should be considered in all respects as exemplary and non-limiting, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within the present invention, and no reference numerals in the claims should be construed as limiting the scope of the claims.

[0043] Furthermore, it should be understood that although this specification describes embodiments, not every embodiment contains only one independent technical solution. This narrative style is merely for clarity. Those skilled in the art should consider the specification as a whole, and the technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.

Claims

1. A method for bias reduction in a computerized adaptive test based on selective mixing, characterized in that, The method comprises the following steps: Based on the obtained historical answering records of the examinee, an interaction dataset is constructed, and is divided into a support set and a meta set, and an answering state vector of the examinee at a tth time step is obtained by encoding ; According to the answer state vector , the question is selected from the support set by using a selection network, and the inner layer optimization is performed by minimizing the prediction loss on to obtain the ability vector characterizing the cognitive state of the examinee Metaset based on the percentage of correct answers The candidates were categorized into three subsets with attributes A, B, and C, and biased aligned samples, unbiased samples, and biased conflict samples were defined. For candidates with attributes A or C... Based on the ability vector, retrieve the candidate with the most similar ability from the candidates with attribute B. ; Candidates Answer records and candidates The bias conflict samples are selectively mixed, and the problem parameters are linearly interpolated to generate diverse problem representations to expand the bias conflict samples and obtain a synthetic sample set. Constructing a final training loss comprising an experience loss and a synthetic sample loss Optimizing the selection network to mitigate selection bias ​ 2. The method as claimed in claim 1, wherein, Based on the obtained historical answer records of the examinee, an interaction dataset is constructed and divided into a support set and a meta set, specifically including: Define the set of examinees as , the set of questions as ; where represents the th examinee, represents the th question, represents the number of examinees, represents the number of questions, , ; Based on the examinee , the question answered by the examinee , and the corresponding label , , indicates that the th examinee answered the question incorrectly, , indicates that the th examinee answered the question correctly; The examinee interaction dataset is divided into a support set and a meta set by examinee and a meta set , the support set is used for inner-layer optimization of the selection network; the meta set is used for outer-layer optimization of the selection network.

3. The method of debiasing a computerized adaptive test based on selective mixing as claimed in claim 2, wherein, The encoding obtains the examinee's answer state vector at the tth time step , specifically comprising: The answer state matrix of N examinees to M answer records is denoted as , The element in the matrix indicates that the th examinee answers the th question incorrectly; The element in the matrix indicates that the th examinee does not answer the th question; The element in the matrix indicates that the th examinee answers the th question correctly; the ith row vector in denotes the answer state vector of the ith examinee at the tth time step, denotes the answer state vector of each examinee at the tth time step.

4. The method of claim 1, wherein, The answer state vector , selecting a question from the support set using a selection network, and performing inner loop optimization by minimizing the prediction loss on to obtain the ability vector characterizing the cognitive state of the examinee, specifically comprising: Selecting a network based on answer status vector Selecting a question for a current examinee ; ； wherein represents a selection network, represents a parameter of the selection network; The inner layer optimizes by minimizing the predicted loss of answers in the support set to estimate the ability vector of the examinee; ； wherein, is the ability vector after inner optimization by cross-entropy loss, denotes the answer result of the i-th examinee examinee to denotes the cognitive diagnosis model obtaining the probability of the examinee answering the current question correctly, denotes the ability vector of the examinee before updating, is the question parameter in the cognitive diagnosis model, denotes the network parameter in the cognitive diagnosis model, denotes the cross-entropy loss, denotes the total number of time steps.​ 5. The method as claimed in claim 1, wherein, The metaset based on the proportion of correct answer records The test takers were categorized into three subsets with attributes A, B, and C, specifically including: According to the proportion of correct answers of the examinees in the meta-set , it is divided into three subsets with attributes A, B and C: , , , , wherein the proportion of correct answers of the examinees in the subset with attribute is located in the interval , the proportion of correct answers of the examinees in the subset with attribute is located in the interval , and the proportion of correct answers of the examinees in the subset with attribute is located in the interval .

6. The method as claimed in claim 5, wherein, The definition bias alignment sample, unbiased sample and bias conflict sample, specifically including: The bias alignment sample is a sample from with a label of or a sample from with a label of ; the unbiased sample is a sample from with a label of or a sample with a label of ; the bias conflict sample is a sample from with a label of or a sample from with a label of ; wherein the label of the sample is 0, indicating that the examinee in the sample answers the question in the sample incorrectly; the label of the sample is 1, indicating that the examinee in the sample answers the question in the sample correctly.

7. The method as claimed in claim 5, wherein, The examinee whose attribute is A or C Retrieving the examinee with the most similar ability from the examinees whose attribute is B according to the ability vector Specifically, the method comprises the steps of: The ability vector of the examinee with attributes is calculated The ability vector of the examinee with attributes is calculated The similarity between the ability vector of the examinee with attributes and the ability vector of the examinee with attributes is calculated The examinee with the most similar ability is retrieved ; The similarity between is calculated ​ ； is the Euclidean distance.

8. The method as claimed in claim 1, wherein, The test taker The test taker The test taker, the deviation conflict sample is selectively mixed, the diversified problem representation is generated by linear interpolation on the problem parameter, the deviation conflict sample is expanded, and the synthetic sample set is obtained, specifically including: In the exam The examinee with the most similar ability Select one question from the answer record of the examinee , in The bias conflict sample sampling has the same label question Linear interpolation is performed on the parameters of the two questions to generate a diversified question representation : ； where the weighting coefficients , control the strength of the interpolation, is a parameter of the Beta distribution, is a parameter of the Beta distribution, is a parameter of the Beta distribution, is a parameter of the Beta distribution, denotes the Beta distribution.

9. The method of claim 1, wherein, The construction comprises an experience loss and a synthetic sample loss of final training loss , optimizing the selection network, specifically comprising: Constructing experience loss : represents the total number of examinees; represents a cross-entropy loss, represents the answer result of the th examinee to the question , represents a cognitive diagnosis model that obtains a probability of the examinee answering the current question correctly, is a parameter of the question , represents a network parameter in the cognitive diagnosis model; Constructing synthetic sample loss : representing a problem and a problem synthetic problems; representing a synthetic sample set; constructing the final outer layer optimization loss : ; denotes a coefficient of .

Images