Computer adaptive testing method

Abstract

The application discloses a computer adaptive testing method based on EAP algorithm, and mainly comprises the following steps: step one, a user starts to do the questions; step two, first layer question extraction, N questions are needed to be extracted; step three, whether N questions have been extracted is judged, if yes, step four is turned to, if not, step two is turned to; step four, i layer question extraction, Ki questions are needed to be extracted; step five, EAP algorithm is used to calculate the trait value; step six, M questions with the minimum absolute value of the difference between the difficulty value and the trait value are searched; step seven, data information of the M questions is calculated; step eight, an exposure coefficient is calculated, and the question with the minimum exposure coefficient is selected. The application can improve the detection uniformity, and meanwhile, the detection efficiency and the detection precision are considered, the performance is high, the information function value of each question does not need to be calculated through traversal, the resource of the central processing unit is greatly saved, and the average return time is 600 milliseconds when 1000 people simultaneously answer in 1 second.

Description

Technical Field

[0001] This invention relates to a detection method, and more particularly to a computer adaptive testing method based on the EAP algorithm. Background Technology

[0002] Currently, the most common adaptive testing methods include the maximum information method (MAB), alpha stratification, and shadow question banks. The MAB calculates the information function value of each item in the question bank based on the trait values ​​calculated from the test takers' responses, and then selects the item with the highest information function value. Alpha stratification is an algorithm that overcomes the limitation of the MAB in selecting low-discrimination items. Alpha stratification stratifies items based on their discrimination ability, and then selects items for each or several strata using either the MAB or the minimum absolute difference method between difficulty and trait values. Shadow question banks are a method that selects the most suitable items from a question bank for each test taker and then uses this current question bank for adaptive testing.

[0003] Existing popular adaptive testing methods all have certain drawbacks. The maximum information method (MAB) boasts the highest testing efficiency and accuracy, but suffers from poor test uniformity. Furthermore, calculating the information function for each question in the question bank leads to poor system performance. While stratification improves the test uniformity of MAB, the improvement is limited, and calculating the information function for each layer incurs significant CPU overhead. Shadow question banks pre-emptively eliminate unsuitable questions, forming a comprehensive question bank that greatly improves test uniformity while maintaining high accuracy and efficiency. However, the objective function and constraints of shadow question banks are complex, leading to poor system performance in real-world applications. This invention relates to parameter estimation and question selection algorithms for computer adaptive testing. The goal of this invention is to achieve high-efficiency and high-performance adaptive testing by calculating the next most suitable question for each subject's answer for each test. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to provide a computer adaptive testing method based on the EAP algorithm, which can improve the uniformity of detection, while taking into account the detection efficiency and detection accuracy, and has high performance.

[0005] The present invention solves the above-mentioned technical problems through the following technical solution: a computer adaptive testing method based on the EAP algorithm, characterized in that the computer adaptive testing method includes the following steps:

[0006] Step 1: The user starts taking the test. The user opens the testing system and prepares to take the test.

[0007] Step 2, First layer of question selection: The computer interface provides the user with a question bank and selects N questions. All the questions in the first layer are divided into N piles according to their difficulty value from smallest to largest. Then, one question is selected from each pile to form the first layer of questions. N is a natural number.

[0008] Step 3: The testing system determines whether the user has completed the first level of N questions. If yes, proceed to step 4; otherwise, proceed to step 2.

[0009] Step 4: When the testing system determines that the first layer of questions has been answered, it moves to the i-th layer to draw questions. Ki questions need to be drawn. The value of Ki needs to be set in advance, where i is a natural number.

[0010] Step 5: Based on the previous answer scores, the testing system uses the EAP algorithm to calculate the subject's trait values ​​using the scores and test parameters.

[0011] Step 6: The testing system searches for the M questions with the smallest absolute value of the difference between the difficulty value and the trait value of the i-th level, where M is a natural number;

[0012] Step 7: The testing system calculates the information function values ​​for these M questions, forming a vector of information function values;

[0013] Step 8: The testing system calculates the exposure count of these M questions, and the total exposure count of these M questions;

[0014] Step 9: The testing system calculates the exposure coefficient vector of the M questions. The exposure coefficient is the ratio of the number of times a question is exposed to the product of the information function value of the question and the total number of exposures of the M questions. Then, the smallest element is selected from the exposure coefficient vector.

[0015] Step 10: The testing system determines whether all questions for all levels have been drawn. If yes, proceed to step 12; otherwise, proceed to step 11.

[0016] Step 11: The testing system determines whether all Ki questions have been drawn. If yes, i = i + 1 and then proceed to step 4; otherwise, proceed directly to step 4.

[0017] Step 12, End.

[0018] Preferably, step seven includes the following steps:

[0019] Calculate the information function value for each of the M questions;

[0020] Iterate through the problems to get the number of times each problem has been used;

[0021] Calculate the total number of times all M questions are used.

[0022] Preferably, the computer adaptive testing method adopts an asynchronous non-blocking solution. Non-blocking means that when a thread encounters an I / O operation, it does not block and wait for the I / O operation to complete or for data to return; instead, it simply sends the I / O request to the operating system and continues executing the next statement. When the operating system completes the I / O operation, it notifies the thread that performed the I / O operation in the form of an event. The thread will then process this event at a specific time. Asynchronous means that the application does not need to poll and process each task; it only needs to pass the data to the application through a signal or callback after the I / O is completed. The biggest advantage of asynchronous non-blocking is that it can respond to large-scale I / O requests, making it very suitable for large-scale computer adaptive testing applications.

[0023] Preferably, the computer adaptive testing method employs the EAP algorithm, which is the average of the posterior probability function values ​​of the IRT model. In step five, the EAP algorithm is used to calculate the subject's trait values ​​using scores and entity parameters, eliminating the need for iteration and reducing computational load. Its advantages include no need for iteration, no occurrence of abnormal situations, and low computational load, making it particularly suitable for large-scale computer adaptive testing applications.

[0024] The positive and progressive effects of this invention are as follows: This invention can improve the uniformity of detection, while taking into account both detection efficiency and detection accuracy. It has high performance, does not require traversing and calculating the information function value of each test question, greatly saves the resources of the central processing unit, and the average return time for 1000 people answering simultaneously in 1 second is 600 milliseconds. Attached Figure Description

[0025] Figure 1 This is a flowchart of the algorithm of the present invention. Detailed Implementation

[0026] The preferred embodiments of the present invention are given below with reference to the accompanying drawings to illustrate the technical solution of the present invention in detail.

[0027] like Figure 1 As shown, the computer adaptive testing method based on the EAP algorithm of this invention mainly includes the following steps:

[0028] Step 1: The user starts taking the test. The user opens the testing system and prepares to take the test.

[0029] Step 2, First layer of question selection: The computer interface provides the user with a question bank and selects N questions. All the questions in the first layer are divided into N piles according to their difficulty value from smallest to largest. Then, one question is selected from each pile to form the first layer of questions. N is a natural number.

[0030] Step 3: The testing system determines whether the user has completed the first level of N questions. If yes, proceed to step 4; otherwise, proceed to step 2.

[0031] Step 4: When the testing system determines that the first layer of questions has been answered, it moves to the i-th layer to draw questions. Ki questions need to be drawn. The value of Ki needs to be set in advance, where i is a natural number.

[0032] Step 5: Based on the previous answer scores, the testing system uses the EAP algorithm to calculate the subject's trait values ​​using the scores and test parameters.

[0033] Step 6: The testing system searches for the M questions with the smallest absolute value of the difference between the difficulty value and the trait value of the i-th level, where M is a natural number;

[0034] Step 7: The testing system calculates the information function values ​​for these M questions, forming a vector of information function values;

[0035] Step 8: The testing system calculates the exposure count of these M questions, and the total exposure count of these M questions;

[0036] Step 9: The testing system calculates the exposure coefficient vector of the M questions. The exposure coefficient is the ratio of the number of times a question is exposed to the product of the information function value of the question and the total number of exposures of the M questions. Then, the smallest element is selected from the exposure coefficient vector.

[0037] Step 10: The testing system determines whether all questions for all levels have been drawn. If yes, proceed to step 12; otherwise, proceed to step 11.

[0038] Step 11: Determine if all Ki questions have been drawn. If yes, then i = i + 1 and go to step 4. Otherwise, go directly to step 4.

[0039] Step 12, End.

[0040] Step seven includes the following steps:

[0041] Calculate the information function value for each of the M questions;

[0042] Iterate through the problems to get the number of times each problem has been used;

[0043] Calculate the total number of times all M questions are used. Based on the number of times questions are used, combined with stratification (a), the maximum information function, and the shadow question bank, solve for the minimum exposure coefficient by establishing an objective function and constraints, placing the number of times questions are used and the information function value on the same dimension.

[0044] The computer adaptive testing method described employs an asynchronous non-blocking solution. Non-blocking means that when a thread encounters an I / O operation, it does not block and wait for the I / O operation to complete or for data to return; instead, it simply sends the I / O request to the operating system and continues executing the next statement. When the operating system completes the I / O operation, it notifies the thread that performed the I / O operation as an event, and the thread handles this event at a specific time. Asynchronous means that the application does not need to poll and process each task; it only needs to pass the data to the application via a signal or callback after the I / O is completed. The biggest advantage of asynchronous non-blocking is its ability to respond to large-scale I / O requests, making it very suitable for large-scale computer adaptive testing applications. This invention pre-loads the question bank into a cache to ensure uninterrupted I / O.

[0045] The computer adaptive testing method described uses the EAP algorithm, which is the average of the posterior probability function values ​​of the IRT model. Its advantages include no need for iteration, no occurrence of outliers, and low computational cost, making it particularly suitable for large-scale computer adaptive testing applications. The EAP algorithm is twice as fast as the maximum likelihood method.

[0046] The working principle of this invention is as follows: Based on stratification (a), questions are selected by combining the number of times the questions are used, a shadow question bank, and the maximum information method. The first stratum randomly selects questions of uniform difficulty. For example, if the difficulty distribution of the first stratum is from -3 to 3, and 3 questions need to be randomly selected, then this invention randomly selects questions with difficulty levels of [-3, -1], [-1, 1], and [1, 3] (corresponding to step two). Other strata select questions based on the subject's trait values. First, a question bank is formed by selecting several questions with the smallest absolute difference between the subject's trait values ​​and the difficulty values ​​(the default is 30 questions). Then, the total number of times the questions in the question bank are used is calculated. Next, the total number of times the questions are used, along with the number of times each question is used and the information function of each question, are substituted into the constraint condition (this condition is the ratio of the number of times the question is exposed to the product of the question's information function value and the total number of times the M questions are exposed), to find the exposure coefficient with the smallest objective function. Then, the question with the smallest exposure coefficient is selected (corresponding to steps seven, eight, and nine). The ending rule is a fixed length. For example, if 3 questions are drawn from the first layer, 3 from the second layer, 3 from the third layer, and 3 from the fourth layer, the test ends after 12 questions have been answered.

[0047] In summary, this invention improves detection uniformity while balancing detection efficiency and accuracy. It boasts high performance, eliminates the need to calculate the information function value for each question, significantly saving central processing unit resources. The average return time for 1000 simultaneous responses per second is 600 milliseconds. Applying the Monte Carlo method to a random question bank of 300 questions with a discrimination factor of unif(1,3) and a difficulty factor of norm(0,1), the various testing metrics of this algorithm are shown in Table 1.

[0048] Table 1. Test Indicators for Computer Adaptive Testing Method Based on Monte Carlo Method

[0049] detection indicators Indicator value Detection of uniformity 0.99 Detection efficiency 9.7 Detection overlap rate 0.02

[0050] The results in Table 1 show that the experiment conforms to the theoretical analysis. The computer adaptive testing method based on the EAP algorithm has high detection uniformity and takes into account both detection efficiency and detection accuracy.

[0051] The specific embodiments described above further illustrate the technical problems solved, the technical solutions, and the beneficial effects of the present invention. It should be understood that the above descriptions are merely specific embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A computer-adaptive testing method based on the EAP algorithm, characterized in that, The computer-adaptive testing method includes the following steps: Step 1: The user starts taking the test. The user opens the testing system and prepares to take the test. Step 2, First layer of question selection: The computer interface provides the user with a question bank and selects N questions. All the questions in the first layer are divided into N piles according to their difficulty value from smallest to largest. Then, one question is selected from each pile to form the first layer of questions. N is a natural number. Step 3: The testing system determines whether the user has completed the first level of N questions. If yes, proceed to step 4; otherwise, proceed to step 2. Step 4: When the testing system determines that the first layer of questions has been answered, it moves to the i-th layer to draw questions. Ki questions need to be drawn. The value of Ki needs to be set in advance, where i is a natural number. Step 5: Based on the previous answer scores, the testing system uses the EAP algorithm to calculate the subject's trait value using the scores and test parameters. Step 6: The testing system searches for the M questions with the smallest absolute value of the difference between the difficulty value and the trait value of the i-th level, where M is a natural number; Step 7: The testing system calculates the information function values ​​for these M questions, forming a vector of information function values; Step 8: The testing system calculates the exposure count of these M questions, and the total exposure count of these M questions; Step nine: For the M candidate questions determined in step six, the testing system calculates the exposure coefficient for each candidate question, forming an exposure coefficient vector E, where the i-th vector element e of vector E is... i Let N be the exposure coefficient of the i-th candidate question; the exposure coefficient of the i-th candidate question is the number of times the candidate question is exposed, N. i Information function value I of the candidate question i The ratio of the product of the exposure times of the M candidate questions; the testing system selects the vector element e_min with the smallest exposure coefficient from the exposure coefficient vector E, and extracts the candidate question corresponding to the smallest vector element as the next question to be extracted; Step 10: The testing system determines whether all questions for all levels have been drawn. If yes, proceed to step 12; otherwise, proceed to step 11. Step 11: Determine if all Ki questions have been drawn. If yes, then i = i + 1 and go to step 4. Otherwise, go directly to step 4. Step 12, End.

2. The computer-adaptive testing method as described in claim 1, characterized in that, Step seven includes the following steps: Calculate the information function value for each of the M questions; Iterate through the problems to get the number of times each problem has been used; Calculate the total number of times all M questions are used.

3. The computer-adaptive testing method as described in claim 1, characterized in that, The computer adaptive testing method described above employs an asynchronous non-blocking solution. Non-blocking means that when a thread encounters an I / O operation, it does not block and wait for the I / O operation to complete or for data to return; instead, it simply sends the I / O request to the operating system and continues executing the next statement. When the operating system completes the I / O operation, it notifies the thread that performed the I / O operation as an event, and the thread handles this event at a specific time. Asynchronous means that the application does not need to poll and process each task; it only needs to pass the data to the application via a signal or callback after the I / O is completed. The biggest advantage of asynchronous non-blocking is its ability to respond to large-scale I / O requests, making it very suitable for large-scale computer adaptive testing applications.

4. The computer adaptive testing method as described in claim 1, characterized in that, The computer adaptive testing method uses the EAP algorithm, which is the average of the posterior probability function values ​​of the IRT model. In step five, the EAP algorithm is used to calculate the subject's trait values ​​through scores and entity parameters, which eliminates the need for iteration and reduces computational load.

Images