Test paper difficulty evaluation method, system, device and medium based on parameter correction model

Abstract

The embodiment of the application provides an evaluation method, system, device and medium for test paper difficulty based on a parameter correction model, and belongs to the field of examination evaluation. The method comprises the following steps: constructing a test question estimated difficulty correction model based on historical data; correcting the estimated difficulty of the test questions used in the test by using the correction model; setting the virtual discrimination degree and guessing degree parameters of the test questions according to the experience and theoretical general numerical values of the item writers; and simulating the probability distribution model of the examinee ability according to the examinee ability distribution in the past years. The Monte Carlo simulation method is adopted to generate virtual examinee answer data based on the 3PL model to evaluate the whole test passing rate. The quality parameters of the test questions estimated by the item writers are corrected, the general numerical values are used to assign values to the test question quality parameters, the examinee ability distribution is simulated according to the historical data, the virtual examinee answer data is generated by using the Monte Carlo simulation, and the whole test passing rate is calculated, so that the accurate evaluation of the whole test passing rate is realized, and the scientificity of test paper setting and the fairness of the test are improved.

Description

Technical Field

[0001] This invention relates to the field of examination and assessment technology, and specifically to a method, system, device, and medium for assessing the difficulty of test papers based on a parameter correction model. Background Technology

[0002] In large-scale standardized tests, multiple sets of test papers are usually administered simultaneously to prevent cheating and ensure fairness. These test papers strive to be consistent in question structure, knowledge point distribution, and overall difficulty. However, due to factors such as the high proportion of new questions during the actual question setting, test paper compilation, and examination process, as well as the potential bias in expert subjective judgment, there may be slight differences in difficulty between different test papers. This can lead to uneven pass rates across different papers, affecting the fairness and validity of the examination.

[0003] Currently, the quality assessment of test items and test papers primarily relies on Classical Test Theory (CTT). The core indicators of CTT include the pass rate and discrimination of test items, calculated based on real test-taker response data. For example, the pass rate of a particular item... The CTT method is often used as an estimate of the difficulty of test questions. However, the CTT method has obvious limitations: its parameters are heavily dependent on the ability level of a specific group of test takers. This group dependence makes it impossible to separate the "question quality parameter" from the "test taker ability parameter". When the ability values ​​of the test takers change, the pass rate P-value of the same test question will also change, resulting in a distortion of the question quality assessment.

[0004] To address the issue of pre-exam quality assessment for new test questions and papers, expert prediction methods are typically used to make preliminary judgments on the difficulty of the questions. However, expert prediction methods are inherently subjective and their errors are difficult to quantify. Current technology lacks a systematic mechanism for correcting test parameters, particularly for effectively correcting deviations in the difficulty predictions made by test-setting experts, leading to significant discrepancies between simulation results and actual exam performance. Furthermore, the prediction of the overall pass rate relies solely on a simple average of expert estimates of test difficulty, lacking systematic data support. Summary of the Invention

[0005] The purpose of this invention is to provide a method, system, device, and medium for assessing the difficulty of a test paper based on a parameter correction model. This model comprehensively considers the model supported by systematic simulation data of test paper prediction parameter correction and overall paper pass rate prediction, thereby improving the accuracy and reliability of the overall paper pass rate assessment.

[0006] To achieve the above objectives, embodiments of the present invention provide a method for assessing the difficulty of a test paper based on a parameter correction model, comprising:

[0007] The difficulty of historical test questions is estimated, and the actual difficulty is determined based on historical answer data. Based on the estimated difficulty and the actual difficulty, a test question difficulty correction model is constructed.

[0008] The initial difficulty of the test questions used in this exam is estimated, and the initial difficulty is corrected using the test question estimated difficulty correction model to obtain the corrected test question difficulty parameters;

[0009] Based on expert experience in setting test questions or commonly used theoretical values, set virtual discrimination parameters and guessing parameters for each test question;

[0010] Obtain the ability value distribution data of candidates in previous years on the test questions used in this exam, fit the probability distribution model of the candidates' ability level based on the ability value distribution data, and randomly select multiple candidates' ability values ​​to construct a set of virtual candidates' abilities.

[0011] For each virtual test taker, based on the corrected question difficulty parameters, virtual discrimination parameters and guessing parameters of each question, and the set of abilities of the virtual test taker, the Monte Carlo simulation method is used to simulate the virtual test taker's probability of answering each question correctly based on the 3PL item response theory model.

[0012] The probability of each virtual candidate answering each question correctly is calculated to determine the overall pass rate, and the difficulty of the exam paper used in this exam is evaluated based on the overall pass rate.

[0013] Optionally, based on the estimated difficulty and the actual difficulty, a test difficulty correction model is constructed, including:

[0014] Based on the number of historical test questions and the linear or non-linear trend between the estimated difficulty and the actual difficulty, a test question difficulty correction model is constructed to establish the correspondence between the expert's estimated difficulty and the actual test difficulty. The test question difficulty correction model includes at least: a linear regression model, a multinomial regression model, a piecewise linear regression model, or a machine learning model.

[0015] Optionally, based on the number of historical test questions and the linear or non-linear trend between the estimated difficulty and the actual difficulty, a test question difficulty correction model can be constructed, including:

[0016] If the number of historical test questions is less than the first preset threshold, a linear regression model or a quadratic polynomial regression model will be selected. If the number of historical test questions is less than the first preset threshold and there is a non-linear trend between the estimated difficulty and the actual difficulty, a quadratic polynomial regression model will be selected.

[0017] If the number of historical test questions is greater than the first preset threshold but less than the second preset threshold, a cubic multinomial regression model or a piecewise linear regression model will be selected. If the number of historical test questions is greater than the first preset threshold and there is a multi-stage deviation between the estimated difficulty and the actual difficulty, a piecewise linear regression model will be selected.

[0018] If the number of historical test questions exceeds the second preset threshold, a machine learning model will be selected.

[0019] Optionally, the initial difficulty is corrected using the predicted difficulty correction model to obtain the corrected difficulty, including:

[0020] If the number of history questions is less than the first preset threshold, the difficulty level of the questions will be adjusted using the following formula:

[0021] ;

[0022] In the formula, and These are the slope and intercept parameters, respectively. Let be the estimation error for the i-th question. Let be the estimated difficulty of the i-th question. The measured difficulty of the i-th question;

[0023] If the number of historical test questions is less than the first preset threshold and there is a non-linear trend between the estimated difficulty and the actual difficulty, the following formula is used to correct the test difficulty value:

[0024] ;

[0025] In the formula, , , For coefficients;

[0026] If the number of history questions is greater than the first preset threshold but less than the second preset threshold, the difficulty level of the questions will be adjusted using the following formula:

[0027] ;

[0028] In the formula, , , , For coefficients;

[0029] If the number of history test questions exceeds the first preset threshold and there are multiple stages of deviation between the estimated difficulty and the actual difficulty, then the difficulty value of the test questions will be corrected using the following formula:

[0030] ;

[0031] In the formula, , Two breakpoints are used to distinguish between function intervals with different forms of change. , , Indicates the slope. , , Indicates the intercept;

[0032] If the number of history questions exceeds the second preset threshold, the difficulty level of the questions will be adjusted using the following formula:

[0033] ;

[0034] ;

[0035] In the formula, T represents the number of training rounds. Let be the prediction function for round t. For multi-round prediction functions The average value, where X is the input vector of test item attributes. The prediction function is the result of combining various input vectors, where M is the number of feature categories. Here, "Type" represents the question type, and "Knowledge" represents the knowledge points covered in the question. This is an indicator function that determines whether the input x belongs to a feature region of class m.

[0036] Optionally, the difficulty level of the test questions can be obtained by Probit transformation to obtain the corresponding difficulty parameter. The Probit transformation formula is as follows:

[0037] ;

[0038] In the formula, For difficulty parameters, It is the inverse function of the standard normal cumulative distribution function. This represents the difficulty level of the test questions.

[0039] Optionally, for each virtual test taker, based on the corrected question difficulty parameters, the virtual discrimination parameters and guessing parameters of each question, and the virtual test taker's ability set, a Monte Carlo simulation method is used to simulate the virtual test taker's probability of answering each question correctly based on the 3PL item response theory model. This includes calculating the probability of answering each question correctly according to the following formula:

[0040] ;

[0041] In the formula, Let j be the ability value of the j-th virtual candidate. This represents the discrimination index of the i-th question. This indicates the difficulty level of the i-th question. This represents the guess probability of the i-th question. Let represent the probability that the j-th virtual candidate answers the i-th question correctly.

[0042] Optionally, the method for assessing the difficulty of the test paper may also include:

[0043] The calculated overall pass rate is compared with the preset target pass rate to obtain the deviation. If the deviation exceeds the set threshold, feedback is provided to adjust the combination of questions that make up the test paper, and the difficulty is reassessed until the overall pass rate meets the requirements.

[0044] Secondly, the present invention also provides a test paper difficulty assessment system based on a parameter correction model, comprising:

[0045] The model building unit is used to predict the difficulty of historical test questions, determine the actual difficulty based on historical answer data of historical test questions, and build a test question prediction difficulty correction model based on the predicted difficulty and the actual difficulty.

[0046] The correction unit is used to estimate the initial difficulty of the test questions used in this exam, and to correct the initial difficulty using the test question difficulty estimation correction model to obtain the corrected test question difficulty parameters.

[0047] The parameter setting unit is used to set the virtual discrimination parameter and guessing parameter for each test item based on expert test-setting experience or theoretically common values.

[0048] The fitting unit is used to obtain the distribution data of the ability values ​​of candidates in previous years on the test questions used in this exam, and to fit a probability distribution model of the candidates' ability levels based on the ability value distribution data, and to randomly select multiple candidates' ability values ​​to construct a set of virtual candidates' abilities.

[0049] The probability calculation unit is used to calculate the probability of answering each question correctly for each virtual candidate based on the corrected question difficulty parameters, the virtual discrimination parameters and guessing parameters of each question, and the ability set of the virtual candidate, using the Monte Carlo simulation method. The calculation is based on the 3PL item response theory model.

[0050] The evaluation unit is used to calculate the probability of each virtual candidate answering each question correctly, determine the overall pass rate, and evaluate the difficulty of the exam paper based on the overall pass rate.

[0051] Thirdly, the present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the above-described method for evaluating the difficulty of a test paper based on a parameter correction model.

[0052] Fourthly, the present invention also provides a storage medium storing a computer program thereon, wherein the computer program, when executed by a processor, implements the steps of the above-described method for evaluating the difficulty of a test paper based on a parameter correction model.

[0053] By employing the above technical solutions, the system can accurately assess the overall pass rate of the exam paper by correcting the predicted quality parameters of the test questions by the test question setting experts, assigning values ​​to the test question quality parameters using the test question setting experience or theoretically universal values, simulating the distribution of candidates' abilities based on historical data, generating virtual candidate answer data using Monte Carlo simulation, and calculating the overall pass rate. This improves the scientific nature of the test paper and the fairness of the exam.

[0054] Other features and advantages of the embodiments of the present invention will be described in detail in the following detailed description section. Attached Figure Description

[0055] The accompanying drawings are provided to further illustrate embodiments of the present invention and form part of the specification. They are used together with the following detailed description to explain the embodiments of the present invention, but do not constitute a limitation thereof. In the drawings:

[0056] Figure 1 This is a flowchart of a test paper difficulty assessment method based on a parameter correction model provided in an embodiment of this application;

[0057] Figure 2 This is a schematic diagram of a Monte Carlo simulation process provided in an embodiment of this application;

[0058] Figure 3 This is a schematic diagram of the structure of a test paper difficulty assessment system based on a parameter correction model provided in an embodiment of this application;

[0059] Figure 4 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0060] Various embodiments of this disclosure will be described more fully in the following detailed description. This disclosure may have various embodiments, and adjustments and changes may be made therein. However, it should be understood that there is no intention to limit the various embodiments of this disclosure to the specific embodiments disclosed herein, but rather this disclosure should be understood to cover all adjustments, equivalents, and / or alternatives falling within the spirit and scope of the various embodiments of this disclosure.

[0061] In the following, the terms “comprising” or “may include”, which may be used in various embodiments of this disclosure, indicate the presence of the disclosed functions or operations and do not limit the addition of one or more functions or operations. Furthermore, as used in various embodiments of this disclosure, the terms “comprising,” “having,” and their cognates are intended only to indicate a specific feature, number, step, operation, or combination of the foregoing and should not be construed as primarily excluding the presence of one or more other features, numbers, steps, operations, or combinations of the foregoing, or the possibility of adding one or more features, numbers, steps, operations, or combinations of the foregoing.

[0062] In various embodiments of this disclosure, the expression "or" or "at least one of A and / or B" includes any combination or all combinations of the words listed simultaneously. For example, the expression "A or B" or "at least one of A and / or B" may include A, may include B, or may include both A and B.

[0063] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0064] See Figure 1 The diagram shows a flowchart of a test paper difficulty assessment method based on a parameter correction model in a specific embodiment, including the following execution steps:

[0065] Step 100: Estimate the difficulty of historical test questions, determine the actual difficulty based on historical answer data, and construct a test question difficulty correction model based on the estimated difficulty and the actual difficulty.

[0066] For example, 28 multiple-choice questions from a previous professional qualification exam were analyzed, including expert-estimated difficulty and actual exam difficulty, both derived from classical measurement theory. Anonymized data samples are shown in Table 1 below; this is for illustrative purposes only and has no actual meaning.

[0067] Table 1. Examples of anonymized data from a professional qualification examination in previous years.

[0068]

[0069] This document contains data on 28 multiple-choice questions from a professional qualification exam, including expert estimates of the question's difficulty. Anonymized data samples are shown in Table 2 below; this information is for illustrative purposes only and has no substantive meaning.

[0070] Table 2. Sample anonymized data from this professional qualification examination

[0071]

[0072] Specifically, based on the estimated difficulty and the actual difficulty, a test question estimated difficulty correction model is constructed, including: based on the number of historical test questions and the linear or non-linear trend between the estimated difficulty and the actual difficulty, a test question estimated difficulty correction model is constructed to establish the correspondence between the expert's estimated difficulty and the actual test difficulty. The test question estimated difficulty correction model includes at least: a linear regression model, a multinomial regression model, a piecewise linear regression model, or a machine learning model.

[0073] More specifically, based on the number of historical test questions and the non-linear trend between the estimated difficulty and the actual difficulty, a test question difficulty correction model is constructed, including the following three scenarios:

[0074] Case 1: If the number of historical test questions is less than the first preset threshold, then a linear regression model or a quadratic polynomial regression model shall be selected. If the number of historical test questions is less than the first preset threshold and there is a non-linear trend between the estimated difficulty and the actual difficulty, then a quadratic polynomial regression model shall be selected.

[0075] Scenario 2: If the number of historical test questions is greater than the first preset threshold but less than the second preset threshold, then a cubic multinomial regression model or a piecewise linear regression model shall be selected. If the number of historical test questions is greater than the first preset threshold and there is a multi-stage deviation between the estimated difficulty and the actual difficulty, then a piecewise linear regression model shall be selected.

[0076] Scenario 3: If the number of historical test questions exceeds the second preset threshold, then a machine learning model will be selected.

[0077] For example, select an appropriate correction model based on the amount of historical data. For datasets with fewer than 50 questions, choose a linear regression model or a quadratic polynomial regression model with fewer parameters and stronger interpretability to avoid overfitting that may occur with highly complex models on small datasets. Linear regression models have advantages such as fewer parameters and stronger interpretability, and the simplest model can capture the predicted difficulty. and actual difficulty The basic trend between them. And regarding the estimated difficulty... Compared with actual measurement difficulty The non-linear trend that may occur means that expert judgments may not change uniformly, especially for easy and difficult questions, where the accuracy is even greater. This "biased at both ends, accurate in the middle" phenomenon is represented by a U-shaped curve on the graph. With a small amount of data, quadratic polynomial regression can be used to introduce higher-order terms of the variables to capture the non-linear deviation pattern between the two. By introducing a squared term to bend the model curve, it can more realistically reflect the actual response patterns of "extreme and more biased" questions. When the amount of data is sufficient, there is enough statistical power to identify and stably estimate higher-order relationships, i.e., S-shaped relationships with multiple inflection points on the graph. Therefore, for data between 50 and 500 questions, a cubic polynomial regression model or a piecewise linear regression model can be used. The cubic polynomial regression model can capture the S-shaped non-linear deviation characteristics between the estimated difficulty and the actual difficulty. When a multi-stage deviation phenomenon occurs, i.e., "biased at both ends, accurate in the middle," a piecewise linear regression model can be used to fit the regression curve piecewise, establishing a linear mapping relationship independently in each interval. In scenarios with large datasets, machine learning models with strong expressive power, such as random forest models, are more likely to capture complex nonlinear, interactive, or multimodal features that may arise between expert-predicted difficulty and actual testing difficulty. They can also support the addition of auxiliary features such as question type and knowledge points to enhance model capabilities. Model building and validity validation: 70% of the data is used for model building, and 30% is used for model validity validation. Validation metrics include... RMSE, MAE.

[0078] Step 101: Estimate the initial difficulty of the test questions used in this exam, and use the test question difficulty estimation correction model to correct the initial difficulty to obtain the corrected test question difficulty parameters.

[0079] In one embodiment, the initial difficulty is corrected using the test question prediction difficulty correction model to obtain the corrected test question difficulty, including:

[0080] If the number of history questions is less than the first preset threshold, the difficulty level of the questions will be adjusted using the following formula:

[0081] ;

[0082] In the formula, and These are the slope and intercept parameters, respectively. Let be the estimation error for the i-th question. Let be the estimated difficulty of the i-th question. The measured difficulty of the i-th question;

[0083] If the number of historical test questions is less than the first preset threshold and there is a non-linear trend between the estimated difficulty and the actual difficulty, the following formula is used to correct the test difficulty value:

[0084] ;

[0085] In the formula, , , For coefficients;

[0086] If the number of history questions is greater than the first preset threshold but less than the second preset threshold, the difficulty level of the questions will be adjusted using the following formula:

[0087] ;

[0088] In the formula, , , , For coefficients;

[0089] If the number of history test questions exceeds the first preset threshold and there are multiple stages of deviation between the estimated difficulty and the actual difficulty, then the difficulty value of the test questions will be corrected using the following formula:

[0090] ;

[0091] In the formula, , Two breakpoints are used to distinguish between function intervals with different forms of change. , , Indicates the slope. , , Indicates the intercept;

[0092] If the number of history questions exceeds the second preset threshold, the difficulty level of the questions will be adjusted using the following formula:

[0093] ;

[0094] ;

[0095] In the formula, T represents the number of training rounds. Let be the prediction function for round t. For multi-round prediction functions The average value of X is the input vector containing test item attributes, etc. The prediction function is the result of combining various input vectors, where M is the number of feature categories. Here, "Type" represents the question type, and "Knowledge" represents the knowledge points covered in the question. This is an indicator function that determines whether the input x belongs to a feature region of class m.

[0096] For example, the difficulty value of a test question is transformed using a Probit transform to obtain the corresponding difficulty parameter. The Probit transform formula is as follows:

[0097] ;

[0098] In the formula, For difficulty parameters, It is the inverse function of the standard normal cumulative distribution function. This represents the difficulty level of the test questions.

[0099] Step 102: Based on expert experience in setting test questions or theoretically common values, set the virtual discrimination parameter and guessing parameter for each test question.

[0100] In some implementations, the virtual discrimination parameter α is set by test-setting experts based on experience according to the question type. For multiple-choice questions, α ranges from [0.8, 2.5]; for fill-in-the-blank questions, it ranges from [1.2, 2.5]; and for subjective questions, it ranges from [1.5, 3.0]. Theoretically, α's range is (0, +∞), but in practice it is often [0.5, 3.0]. When α ≥ 1.7, the question can theoretically effectively distinguish the ability value θ within […]. -1, For candidates between +1], the multiple-choice questions in this embodiment use a theoretically universal discrimination index α=1.2.

[0101] In some implementations, the ideal value of the guessing parameter c is set according to the question type and the number of options. The ideal value of the guessing parameter c for single-choice questions is 0.25, the ideal value of the guessing parameter c for multiple-choice questions is 0.08, the ideal value of the guessing parameter c for fill-in-the-blank questions is 0.05, and the ideal value of the guessing parameter c for subjective questions is close to 0.00. The guessing parameter c can also be set by experts based on their question-setting experience. In this embodiment, the theoretically common guessing value of 0.25 is used for multiple-choice questions.

[0102] Step 103: Obtain the ability value distribution data of candidates in previous years on the test questions used in this exam, fit the probability distribution model of the candidates' ability level based on the ability value distribution data, and randomly select multiple candidates' ability values ​​to construct a set of virtual candidates' abilities;

[0103] In some implementations, the probability distribution model of the candidate's ability level θ is a normal distribution N(μ, ), where μ and δ can be determined based on the mean and standard deviation of the candidates' ability values ​​θ over the past five years. The formula for calculating the mean is:

[0104] ;

[0105] In the formula, n represents the total number of test takers, and the standard deviation δ describes the dispersion of the score data around the mean. Since the sample size of test takers in previous years was large, the biased estimation formula is:

[0106] .

[0107] This embodiment assumes that the probability distribution model of the simulated candidate's ability level θ is a standard normal distribution, and is randomly sampled according to N(0,1).

[0108] Step 104: For each virtual candidate, based on the corrected question difficulty parameters, virtual discrimination parameters and guessing parameters of each question, and the set of abilities of the virtual candidate, the Monte Carlo simulation method is used to simulate the virtual candidate's probability of answering each question correctly based on the 3PL item response theory model;

[0109] Specifically, the probability of answering each question correctly is calculated using the following formula:

[0110] ;

[0111] In the formula, Let j be the ability value of the j-th virtual candidate. This represents the discrimination index of the i-th question. This indicates the difficulty level of the i-th question. This represents the guess probability of the i-th question. Let represent the probability that the j-th virtual candidate answers the i-th question correctly.

[0112] For example, see Figure 2 The diagram shown is a schematic of a Monte Carlo simulation process provided in an embodiment of this application. The first step is based on the capability... Difficulty Discrimination Speculation calculate The second step is to generate a random value u ~ [0, 1]; the third step is to determine the size, if u < Record "Correct" if the answer is correct, otherwise "Incorrect". Calculate the candidate's score.

[0113] Step 105: Calculate the probability of each virtual candidate answering each question correctly, determine the overall pass rate, and assess the difficulty of the exam paper based on the overall pass rate.

[0114] In some implementations, the overall pass rate assessment is completed, and the IRT model difficulty parameter β after being corrected by the estimated difficulty correction model and transformed by Probit is shown in Table 3 below:

[0115] Table 3. Difficulty parameter β after Probit transformation

[0116]

[0117] The experimental calculation results obtained according to the method of this application are shown in Table 4 below:

[0118] Table 4 Experimental Calculation Results

[0119]

[0120] In some implementations, the method for assessing the difficulty of the test paper further includes: comparing the calculated overall pass rate with the preset target pass rate to obtain the deviation; if the deviation exceeds a set threshold, then adjusting the combination of questions constituting the test paper and reassessing the difficulty until the overall pass rate meets the requirements.

[0121] In this embodiment, by correcting the test quality parameters predicted by the test-setting experts, assigning values ​​to the test quality parameters using theoretically universal values, simulating the distribution of candidates' abilities based on historical data, generating virtual candidate answer data using Monte Carlo simulation, and calculating the overall pass rate, the system achieves an accurate assessment of the overall pass rate, thereby improving the scientific nature of the test paper and the fairness of the examination.

[0122] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0123] like Figure 3 As shown, the following are embodiments of the test paper difficulty assessment system based on the parameter correction model provided in this disclosure. It belongs to the same inventive concept as the test paper difficulty assessment method based on the parameter correction model in the above embodiments. For details not described in detail in the embodiments of the test paper difficulty assessment system, please refer to the embodiments of the test paper difficulty assessment method based on the parameter correction model described above.

[0124] A test difficulty assessment system based on a parameter correction model includes:

[0125] The model building unit is used to predict the difficulty of historical test questions, determine the actual difficulty based on historical answer data of historical test questions, and build a test question prediction difficulty correction model based on the predicted difficulty and the actual difficulty.

[0126] The correction unit is used to estimate the initial difficulty of the test questions used in this exam, and to correct the initial difficulty using the test question difficulty estimation correction model to obtain the corrected test question difficulty parameters.

[0127] The parameter setting unit is used to set the virtual discrimination parameter and guessing parameter for each test item based on expert test-setting experience or theoretically common values.

[0128] The fitting unit is used to obtain the distribution data of the ability values ​​of candidates in previous years on the test questions used in this exam, and to fit a probability distribution model of the candidates' ability levels based on the ability value distribution data, and to randomly select multiple candidates' ability values ​​to construct a set of virtual candidates' abilities.

[0129] The probability calculation unit is used to calculate the probability of answering each question correctly for each virtual candidate based on the corrected question difficulty parameters, the virtual discrimination parameters and guessing parameters of each question, and the ability set of the virtual candidate, using the Monte Carlo simulation method. The calculation is based on the 3PL item response theory model.

[0130] The evaluation unit is used to calculate the probability of each virtual candidate answering each question correctly, determine the overall pass rate, and evaluate the difficulty of the exam paper based on the overall pass rate.

[0131] Figure 4 This is a schematic diagram of the hardware structure of an electronic device that implements various embodiments of the present invention.

[0132] The test paper difficulty assessment method provided in this application embodiment can be applied to electronic devices. Those skilled in the art will understand that the electronic device structure involved in the embodiments of this invention does not constitute a limitation on the electronic device. An electronic device may include more or fewer components than illustrated, or combine certain components, or have different component arrangements. In the embodiments of this invention, the electronic device includes, but is not limited to, laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the embodiments of this application described and / or claimed herein.

[0133] Electronic devices may include processors, external memory interfaces, internal memory, universal serial bus (USB) interfaces, charging management modules, power management modules, batteries, wireless communication modules, audio modules, speakers, microphones, sensor modules, buttons, cameras, displays, and SIM card interfaces, etc.

[0134] It is understood that the structures illustrated in the embodiments of this application do not constitute a specific limitation on the electronic device. In other embodiments of this application, the electronic device may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0135] A processor may include one or more processing units, such as: a central processing unit (CPU), an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural network processing unit (NPU). Different processing units may be independent devices or integrated into one or more processors.

[0136] The processor can serve as the nerve center and command center of an electronic device. The controller can generate operation control signals based on the instruction opcode and timing signals to control the fetching and execution of instructions.

[0137] The processor may also include memory for storing instructions and data. In some embodiments, the memory in the processor is a cache memory. This memory can store instructions or data that the processor has just used or that are used repeatedly. If the processor needs to use the instruction or data again, it can retrieve it directly from this memory. This avoids repeated accesses, reduces processor latency, and thus improves system efficiency.

[0138] An external storage interface (ESI) can be used to connect external memory cards, such as microSD cards, to expand the storage capacity of electronic devices. The external memory card communicates with the processor through the ESI to perform data storage functions, such as saving music and video files on the external memory card.

[0139] Internal memory can be used to store computer executable program code, which includes instructions. The processor executes various functional applications and data processing of electronic devices by running the instructions stored in internal memory. Internal memory can include a program storage area and a data storage area. Internal memory can include high-speed random access memory, and can also include non-volatile memory, such as at least one disk storage device, flash memory device, universal flash storage (UFS), etc.

[0140] Wireless communication functionality in electronic devices can be achieved through antennas, wireless communication modules, modem processors, and baseband processors.

[0141] Wireless communication modules can provide solutions for wireless communication applications in electronic devices, including wireless local area networks (WLANs) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), and infrared (IR) technologies.

[0142] Electronic devices can implement audio functions through audio modules, speakers, receivers, microphones, headphone jacks, and application processors.

[0143] Electronic devices can achieve shooting functions through ISPs, cameras, video codecs, GPUs, displays, and application processors.

[0144] Electronic devices can achieve display functions through GPUs, displays, and application processors.

[0145] A GPU is a microprocessor for image processing, connected to the display screen and application processor. GPUs are used to perform mathematical and geometric calculations for graphics rendering. A processor may include one or more GPUs, which execute program instructions to generate or modify display information.

[0146] A display screen is used to display images, videos, etc. A display screen includes a display panel.

[0147] The storage medium provided in this application stores a program product capable of implementing an assessment method for the difficulty of a test paper.

[0148] The methods for assessing the difficulty of the exam paper include: estimating the difficulty of historical exam questions, determining the actual difficulty based on historical answer data, and constructing a question difficulty correction model based on the estimated and actual difficulty; estimating the initial difficulty of the questions used in this exam, and correcting the initial difficulty using the question difficulty correction model to obtain the corrected question difficulty parameters; setting virtual discrimination parameters and guessing parameters for each question based on expert question-setting experience or theoretically common values; obtaining the ability value distribution data of candidates in previous years on the questions used in this exam, and based on the... The probability distribution model of the candidates' ability levels is fitted to the ability value distribution data, and multiple candidates' ability values ​​are randomly selected to construct a set of virtual candidates' abilities. For each virtual candidate, based on the corrected question difficulty parameters, virtual discrimination parameters and guessing parameters of each question, and the set of virtual candidates' abilities, the Monte Carlo simulation method is used to simulate the virtual candidate's probability of answering each question based on the 3PL item response theory model. The probability of answering each question correctly for all virtual candidates is statistically analyzed to determine the overall pass rate, and the difficulty of the test paper used in this exam is evaluated based on the overall pass rate.

[0149] In some possible implementations, the subject matter of this disclosure, the method and system for assessing the difficulty of test papers, can be implemented as a program product comprising program code that, when run on a terminal device, causes the terminal device to perform the steps described in the "Exemplary Methods" section of this specification according to various exemplary embodiments of this disclosure.

[0150] The storage medium disclosed herein may be any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (a non-exhaustive list) of readable storage media include: an electrical connection having one or more wires, a portable disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof.

[0151] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for assessing test paper difficulty based on a parameter correction model, characterized in that, include: The difficulty of historical test questions is estimated, and the actual difficulty is determined based on historical answer data. Based on the estimated difficulty and the actual difficulty, a test question difficulty correction model is constructed. The initial difficulty of the test questions used in this exam is estimated, and the initial difficulty is corrected using the test question estimated difficulty correction model to obtain the corrected test question difficulty parameters; Based on expert experience in setting test questions or commonly used theoretical values, set virtual discrimination parameters and guessing parameters for each test question; Obtain the ability value distribution data of candidates in previous years on the test questions used in this exam, fit the probability distribution model of the candidates' ability level based on the ability value distribution data, and randomly select multiple candidates' ability values ​​to construct a set of virtual candidates' abilities. For each virtual test taker, based on the corrected question difficulty parameters, virtual discrimination parameters and guessing parameters of each question, and the set of abilities of the virtual test taker, the Monte Carlo simulation method is used to simulate the virtual test taker's probability of answering each question correctly based on the 3PL item response theory model. The probability of each virtual candidate answering each question correctly is calculated to determine the overall pass rate, and the difficulty of the exam paper used in this exam is evaluated based on the overall pass rate. The initial difficulty is corrected using the predicted difficulty correction model to obtain the corrected difficulty, including: If the number of history questions is less than the first preset threshold, the difficulty level of the questions will be adjusted using the following formula: ； In the formula, and These are the slope and intercept parameters, respectively. Let be the estimation error for the i-th question. Let be the estimated difficulty of the i-th question. The measured difficulty of the i-th question; If the number of historical test questions is less than the first preset threshold and there is a non-linear trend between the estimated difficulty and the actual difficulty, the following formula is used to correct the test difficulty value: ； In the formula, , , For coefficients; If the number of history questions is greater than the first preset threshold but less than the second preset threshold, the difficulty level of the questions will be adjusted using the following formula: ； In the formula, , , , For coefficients; If the number of history test questions exceeds the first preset threshold and there are multiple stages of deviation between the estimated difficulty and the actual difficulty, then the difficulty value of the test questions will be corrected using the following formula: ； In the formula, , Two breakpoints are used to distinguish between function intervals with different forms of change. , , Indicates the slope. , , Indicates the intercept; If the number of history questions exceeds the second preset threshold, the difficulty level of the questions will be adjusted using the following formula: ； ； In the formula, T represents the number of training rounds. Let be the prediction function for round t. For multi-round prediction functions The average value, where X is the input vector of test item attributes. The prediction function is the result of combining various input vectors, where M is the number of feature categories. Here, "Type" represents the question type, and "Knowledge" represents the knowledge points covered in the question. This indicates whether the input x belongs to the m-class feature region; The difficulty level of the test questions is obtained by Probit transformation, and the Probit transformation formula is as follows: ； In the formula, For difficulty parameters, It is the inverse function of the standard normal cumulative distribution function. This represents the difficulty level of the test questions.

2. The method for assessing test paper difficulty based on a parameter correction model according to claim 1, characterized in that, For each virtual test taker, based on the corrected question difficulty parameters, virtual discrimination parameters and guessing parameters for each question, and the virtual test taker's ability set, the Monte Carlo simulation method is used to simulate the virtual test taker's probability of answering each question correctly based on the 3PL item response theory model. This includes calculating the probability of answering each question correctly using the following formula: ； In the formula, Let j be the ability value of the j-th virtual candidate. This represents the discrimination parameter of the i-th test item. This represents the difficulty parameter of the i-th question. This represents the guessing power parameter for the i-th question. Let represent the probability that the j-th virtual candidate answers the i-th question correctly.

3. The method for assessing test paper difficulty based on a parameter correction model according to claim 1, characterized in that, The methods for assessing the difficulty of the exam paper also include: The calculated overall pass rate is compared with the preset target pass rate to obtain the deviation. If the deviation exceeds the set threshold, feedback is provided to adjust the combination of questions that make up the test paper, and the difficulty is reassessed until the overall pass rate meets the requirements.

4. A test paper difficulty assessment system based on a parameter correction model, applicable to the test paper difficulty assessment method based on a parameter correction model as described in any one of claims 1-3, characterized in that, include: The model building unit is used to predict the difficulty of historical test questions, determine the actual difficulty based on historical answer data of historical test questions, and build a test question prediction difficulty correction model based on the predicted difficulty and the actual difficulty. The correction unit is used to estimate the initial difficulty of the test questions used in this exam, and to correct the initial difficulty using the test question difficulty estimation correction model to obtain the corrected test question difficulty parameters. The parameter setting unit is used to set the virtual discrimination parameter and guessing parameter for each test item based on expert test-setting experience or theoretically common values. The fitting unit is used to obtain the distribution data of the ability values ​​of candidates in previous years on the test questions used in this exam, and to fit a probability distribution model of the candidates' ability levels based on the ability value distribution data, and to randomly select multiple candidates' ability values ​​to construct a set of virtual candidates' abilities. The probability calculation unit is used to calculate the probability of answering each question correctly for each virtual candidate based on the corrected question difficulty parameters, the virtual discrimination parameters and guessing parameters of each question, and the ability set of the virtual candidate, using the Monte Carlo simulation method. The calculation is based on the 3PL item response theory model. The evaluation unit is used to calculate the probability of each virtual candidate answering each question correctly, determine the overall pass rate, and evaluate the difficulty of the exam paper based on the overall pass rate.

5. An electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the test paper difficulty assessment method based on the parameter correction model as described in any one of claims 1 to 3.

6. A storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the test paper difficulty assessment method based on the parameter correction model as described in any one of claims 1 to 3.

Images