Test question judgment method and device, electronic equipment, storage medium and product

Abstract

The invention provides a test question judgment method and device, electronic equipment, a storage medium and a product, and the method comprises the steps: carrying out the test question judgment of pre-collected test question data through a pre-trained test question judgment model, obtaining a judgment result corresponding to the test question data, and enabling the judgment result to comprise a score rate and a judgment label which represents whether the test question data can be judged or not; if the evaluation label shows that the evaluation label can be evaluated, taking the score rate as an evaluation output result corresponding to the test question data; and if the evaluation label shows that the test question data cannot be evaluated, taking preset rejection prompt information as an evaluation output result corresponding to the test question data. By adopting the technical scheme, whether the test question can be evaluated or not can be predicted while the score rate of the test question is predicted, the score rate of the test question is output only when the test question can be evaluated, and test question rejection is realized when the test question cannot be evaluated, so that the test question misjudgment rate is reduced, and the test question evaluation accuracy is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer vision and artificial intelligence, and in particular to a test question evaluation method and device, an electronic device, a storage medium and a product. BACKGROUND

[0002] With the application of artificial intelligence in the field of education, intelligent test question evaluation technology in the education artificial intelligence system has attracted widespread attention. Most current automatic evaluation models can only output a score or score rate prediction. When encountering abnormal answers such as unclear logical expression, language confusion, and incomplete formula, the model will still score according to the default scoring rules, resulting in an increase in test question misjudgment rate and affecting test question evaluation accuracy. SUMMARY

[0003] Based on the above needs, the present application provides a test question evaluation method, device, electronic device, storage medium and product, which can predict whether a test question is evaluable while predicting the score rate of the test question. Only when the test question is evaluable, the score rate of the test question is output. When the test question is not evaluable, the test question is rejected for evaluation, thereby reducing the test question misjudgment rate and improving the test question evaluation accuracy.

[0004] To achieve the above object, the present application provides the following technical solutions:

[0005] According to a first aspect of an embodiment of the present application, a test question evaluation method is provided, comprising:

[0006] Using a pre-trained test question evaluation model, test question evaluation is performed on pre-collected test question data to obtain an evaluation result corresponding to the test question data, the evaluation result including a score rate and an evaluation label indicating whether it is evaluable;

[0007] If the evaluation label indicates that it is evaluable, the score rate is taken as the evaluation output result corresponding to the test question data;

[0008] If the evaluation label indicates that it is not evaluable, a pre-set rejection prompt information is taken as the evaluation output result corresponding to the test question data;

[0009] The test question evaluation model is obtained by performing test question evaluation on pre-collected test question samples carrying real evaluation results to obtain predicted evaluation results, and is trained with the minimum difference between the real evaluation results and the predicted evaluation results as the target.

[0010] Optionally, the evaluation result further includes a score rate confidence;

[0011] If the judgment label indicates that the test question is reviewable, the score rate is taken as the judgment output result corresponding to the test question data; if the judgment label indicates that the test question is not reviewable, preset non-review prompt information is taken as the judgment output result corresponding to the test question data, comprising:

[0012] If the judgment label indicates that the test question is reviewable, and the score rate confidence reaches a predetermined score rate confidence threshold, the score rate is taken as the judgment output result corresponding to the test question data.

[0013] If the judgment label indicates that the test question is not reviewable, or the score rate confidence does not reach the predetermined score rate confidence threshold, the preset non-review prompt information is taken as the judgment output result corresponding to the test question data.

[0014] Optionally, the training process of the test question judgment model comprises:

[0015] The first training sample carrying the real score rate is input into the pre-constructed large language model, the predicted score rate corresponding to the first training sample is predicted, and the large language model is trained in the first stage with the minimum difference between the predicted score rate and the real score rate as the target, to obtain an initial test question judgment model.

[0016] The second training sample carrying the real score rate and the real judgment label is input into the initial test question judgment model, the predicted score rate and the predicted judgment label corresponding to the second training sample are predicted, and the initial test question judgment model is trained in the second stage with the minimum difference between the predicted score rate and the real score rate and the minimum difference between the predicted judgment label and the real judgment label as the target, to obtain a test question judgment model.

[0017] Optionally, after the first training sample carrying the real score rate is input into the pre-constructed large language model, the predicted score rate corresponding to the first training sample is predicted, and the large language model is trained in the first stage with the minimum difference between the predicted score rate and the real score rate as the target, to obtain an initial test question judgment model, the method further comprises:

[0018] The second training sample carrying the real score rate is input into the initial test question judgment model, and the predicted score rate corresponding to the second training sample is predicted.

[0019] The real judgment label corresponding to the second training sample is determined based on the comparison result between the difference between the predicted score rate and the real score rate and the preset error threshold.

[0020] Optionally, the determination process of the score rate confidence threshold comprises:

[0021] The test question evaluation model is used to evaluate test samples carrying real evaluation results that are collected in advance, to obtain a predicted evaluation result and a classification confidence corresponding to a predicted evaluation label in the predicted evaluation result;

[0022] Based on the predicted evaluation label and the classification confidence, the test samples are divided into an evaluatable sample set and a non-evaluatable sample set;

[0023] Based on a pre-set evaluatable proportion of test questions and a predicted score rate confidence in the predicted evaluation result, the score rate confidence threshold is determined.

[0024] Optionally, based on the pre-set evaluatable proportion of test questions and the predicted score rate confidence in the predicted evaluation result, the score rate confidence threshold is determined, including:

[0025] Based on the pre-set evaluatable proportion of test questions and a total number of test samples, a number of evaluatable test questions is determined;

[0026] Test samples in the number of evaluatable test questions are extracted from the evaluatable sample set in descending order of the predicted score rate confidence, as final evaluatable samples;

[0027] A minimum value in the predicted score rate confidence corresponding to the final evaluatable samples is taken as the score rate confidence threshold.

[0028] According to a second aspect of an embodiment of the present application, a test question evaluation device is provided, including:

[0029] An evaluation module is configured to use a pre-trained test question evaluation model to evaluate test question data collected in advance, to obtain an evaluation result corresponding to the test question data, the evaluation result including a score rate and an evaluation label indicating whether the test question data is evaluatable;

[0030] A result determination module is configured to, if the evaluation label indicates that the test question data is evaluatable, take the score rate as an evaluation output result corresponding to the test question data.

[0031] The result determination module is further configured to, if the evaluation label indicates that the test question data is not evaluatable, take pre-set non-evaluation prompt information as the evaluation output result corresponding to the test question data.

[0032] The test question evaluation model is obtained by evaluating test question samples carrying real evaluation results that are collected in advance, to obtain a predicted evaluation result, and is trained with a target of minimizing a difference between the real evaluation results and the predicted evaluation results.

[0033] According to a third aspect of an embodiment of the present application, an electronic device is provided, including a memory and a processor.

[0034] The memory is connected with the processor, and is configured to store a program;

[0035] The processor is configured to realize the test question evaluation method by running the program in the memory.

[0036] According to a fourth aspect of the embodiments of the present application, a storage medium is provided, and the storage medium has a computer program stored thereon. When the computer program is executed by a processor, the test question evaluation method is realized.

[0037] According to a fifth aspect of the embodiments of the present application, a computer program product is provided, and the computer program product includes computer program instructions. When the computer program instructions are executed by a processor, the processor realizes the test question evaluation method.

[0038] The test question evaluation method provided in the present application uses a pre-trained test question evaluation model to evaluate pre-collected test question data, to obtain an evaluation result corresponding to the test question data. The evaluation result includes a score rate and an evaluation label indicating whether the test question data can be evaluated. If the evaluation label indicates that the test question data can be evaluated, the score rate is taken as the evaluation output result corresponding to the test question data. If the evaluation label indicates that the test question data cannot be evaluated, pre-set non-evaluation prompt information is taken as the evaluation output result corresponding to the test question data. The test question evaluation model is obtained by evaluating test question samples carrying real evaluation results, obtaining predicted evaluation results, and being trained with the minimum difference between the real evaluation results and the predicted evaluation results as the target. The technical solution of the present application can predict whether a test question can be evaluated while obtaining the score rate of the test question. Only when the test question can be evaluated, the score rate of the test question is output. When the test question cannot be evaluated, the test question is non-evaluated, thereby reducing the test question misjudgment rate and improving the test question evaluation accuracy. BRIEF DESCRIPTION OF DRAWINGS

[0039] In order to more clearly illustrate the technical solutions of the embodiments of the present application or the prior art, the drawings needed in the embodiments or the prior art description will be briefly introduced. Obviously, the drawings in the following description are only embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor on the basis of the provided drawings.

[0040] FIG. 1 A flowchart of a test question evaluation method provided by an embodiment of the present application is shown in the figure.

[0041] FIG. 2 A flowchart of another test question evaluation method provided by an embodiment of the present application is shown in the figure.

[0042] FIG. 3 A processing flowchart of a test question evaluation model training method provided by an embodiment of the present application is shown in the figure.

[0043] FIG. 4 Another processing flow schematic diagram for training a test question evaluation model provided by an embodiment of the present application;

[0044] FIG. 5 A processing flow schematic diagram for determining a score rate confidence threshold provided by an embodiment of the present application;

[0045] FIG. 6 A structural schematic diagram of a test question evaluation device provided by an embodiment of the present application;

[0046] FIG. 7 A structural schematic diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0047] The technical solution of the embodiment of the present application is applicable to the application scenario of an educational artificial intelligence system, and is specifically used in the application scenario of test question evaluation of artificial intelligence. By using the technical solution of the embodiment of the present application, the score rate of a test question can be predicted, and it is determined whether the test question can be evaluated. Only when the test question can be evaluated, the score rate of the test question is output. When the test question cannot be evaluated, the test question is rejected, thereby reducing the test question misjudgment rate and improving the test question evaluation accuracy.

[0048] With the deep penetration of artificial intelligence technology in the education field, the educational artificial intelligence system based on natural language processing (NLP), knowledge graph and deep learning is gradually reconstructing the traditional teaching evaluation system. Among them, the intelligent test question evaluation technology, as a key link connecting teaching data and learning feedback, has become a core breakthrough of the intelligent transformation of education.

[0049] Most of the current automatic evaluation models intelligently output a score or score rate prediction. However, when facing ambiguous, incorrect or incomplete student answers, the model will still score according to the default scoring rules, often causing "overconfidence" scoring, affecting the reliability of the test question evaluation model and the teaching credibility, and leading to an increase in the test question misjudgment rate and affecting the test question evaluation accuracy. For example, in the mainstream mathematical homework evaluation process, the student answers are first extracted by OCR (Optical Character Recognition), and then analyzed and scored by a language model. However, OCR itself has the risk of recognition errors, such as confusion of numbers and variables, omission of symbols, etc. Once such errors of recognition are received by the language model, it may cause incorrect scoring. That is, even if there is significant noise or defects in the input, the model will still give a deterministic score, further amplifying the error, thereby leading to an increase in the test question misjudgment rate and affecting the test question evaluation accuracy.

[0050] Based on this, the application provides a test question evaluation method. The technical solution can predict the scoring rate of a test question and whether the test question can be evaluated. Only when the test question can be evaluated, the scoring rate of the test question is output. When the test question cannot be evaluated, the test question is rejected. Thus, the problem of high test question misjudgment rate and low test question evaluation accuracy in the prior art is solved.

[0051] The technical solutions in the embodiments of the application will be clearly and completely described below with reference to the drawings in the embodiments of the application. Obviously, the described embodiments are only part of the embodiments of the application, rather than all the embodiments of the application. Based on the embodiments in the application, all other embodiments obtained by those skilled in the art without creative work fall within the protection scope of the application.

[0052] Exemplary method

[0053] Referring to FIG. 1 The application provides a test question evaluation method. The method comprises the following steps.

[0054] In S101, a pre-trained test question evaluation model is used to evaluate pre-collected test question data, to obtain an evaluation result corresponding to the test question data.

[0055] In the embodiment, the test question evaluation model is pre-built and trained. When the test question evaluation is performed, the test question data to be evaluated is collected, and then the test question evaluation model is used to evaluate the test question data, to obtain an evaluation result corresponding to the test question data. The test question data comprises a test question stem, a standard answer and a student answer, and the evaluation result comprises a scoring rate and an evaluation label indicating whether the test question can be evaluated. Specifically, when the student answer in the test question data is clear and complete, the logic expression is clear, the collected test question data is clear and complete, and there is no test question recognition error, it is indicated that the test question data can be evaluated. When the student answer in the test question data is abnormal, such as fuzzy, incomplete, unclear logic expression, language confusion, or the collected test question data is fuzzy and incomplete (for example, number, variable confusion, symbol omission), there is a test question recognition error, it is indicated that the test question data cannot be evaluated.

[0056] In this embodiment, the test question data can be collected by image recognition. The images of the test question stems, standard answers and student answers are collected, and then the image recognition technology (e.g., OCR) is used to recognize the test question stem information from the image of the test question stem, the standard answer information from the image of the standard answer corresponding to the test question, and the student answer information from the image of the student answer corresponding to the test question. The test question stem information, the standard answer information and the student answer information corresponding to the test question are spliced to obtain the test question data corresponding to the test question. In addition, the test question stems, standard answers and student answers can also be scanned by text scanning to determine the test question data corresponding to the test question.

[0057] In this embodiment, the test question evaluation model preferably uses a large language model. To train the test question evaluation model, test question samples for training the model are first collected, wherein the test question samples carry real evaluation results, which can be evaluation results manually labeled by professional evaluators (e.g., teachers of subjects corresponding to the test questions or personnel with a certain basis for the subjects corresponding to the test questions, etc.). The test question samples include the test question stem, the standard answer and the student answer corresponding to the sample test question, and the real evaluation results carried by the test question samples include the real score rate and the real evaluation label of the sample test question. The real evaluation label indicates whether the sample test question is evaluable. Then, the test question samples are input into a pre-constructed test question evaluation model (e.g., a large language model), which predicts the score rate and the evaluation label of the test question samples to obtain a predicted evaluation result, which includes a predicted score rate and a predicted evaluation label. Finally, based on the difference between the predicted evaluation result and the real evaluation result, the model parameters of the test question evaluation model are adjusted until the difference between the predicted evaluation result and the real evaluation result is within a preset threshold range. That is, the model parameters of the test question evaluation model are adjusted to minimize the difference between the predicted score rate and the real score rate, and to minimize the difference between the predicted evaluation label and the real evaluation label.

[0058] Specifically, in the training of the test question evaluation model, the test question samples carrying the real score rate and the real evaluation label can be directly used to train the test question evaluation model in one step, or the test question samples carrying only the real score rate can be used to train the test question evaluation model in the first stage, and then the test question samples carrying the real score rate and the real evaluation label can be used to train the test question evaluation model in the first stage in the second stage.

[0059] S102, if the evaluation label indicates that the test question is evaluable, the score rate is taken as the evaluation output result corresponding to the test question data.

[0060] In this embodiment, if the evaluation label corresponding to the test question data predicted by the test question evaluation model indicates that the test question is evaluable, it indicates that the student's answer in the test question data is clear and complete, the logical expression is clear, and the collected test question data is clear and complete, and there is no test question recognition error. At this time, the scoring rate evaluation of the test question data has credibility, therefore, the scoring rate corresponding to the test question data predicted by the test question evaluation model is taken as the evaluation output result corresponding to the test question data, and the evaluation output result is output to the user for viewing.

[0061] S103, if the evaluation label indicates that the test question is not evaluable, the pre-set evaluation rejection prompt information is taken as the evaluation output result corresponding to the test question data.

[0062] In this embodiment, if the evaluation label corresponding to the test question data predicted by the test question evaluation model indicates that the test question is not evaluable, it indicates that the student's answer in the test question data is abnormal, such as fuzzy, incomplete, unclear logical expression, language confusion, or the collected test question data is fuzzy and incomplete (for example, numbers, variables are confused, symbols are missing), and there is a test question recognition error. At this time, the scoring rate evaluation of the test question data may have low credibility due to the abnormal situation of the test question data, therefore, in order to avoid the model predicting an uncertain scoring rate, the pre-set evaluation rejection prompt information is taken as the evaluation output result corresponding to the test question data, and the evaluation output result is output to the user for viewing. The pre-set evaluation rejection prompt information can be a sentence indicating the rejection of test question evaluation (for example, "I am not sure", "I cannot evaluate the test question", etc.), or a prompt sentence indicating that the test question data has low clarity and completeness.

[0063] As can be seen from the above introduction, the test question evaluation method proposed in the embodiments of the present application uses a pre-trained test question evaluation model to evaluate the pre-collected test question data, and obtains the evaluation result corresponding to the test question data. The evaluation result includes: a scoring rate and an evaluation label indicating whether the test question is evaluable; if the evaluation label indicates that the test question is evaluable, the scoring rate is taken as the evaluation output result corresponding to the test question data; if the evaluation label indicates that the test question is not evaluable, the pre-set evaluation rejection prompt information is taken as the evaluation output result corresponding to the test question data; wherein the test question evaluation model is obtained by evaluating the pre-collected test question samples carrying the real evaluation result, obtaining the predicted evaluation result, and training the test question evaluation model with the minimum difference between the real evaluation result and the predicted evaluation result as the target. The technical solution of the embodiment can predict whether the test question is evaluable while predicting the scoring rate of the test question. Only in the case that the test question is evaluable, the scoring rate of the test question is output, and in the case that the test question is not evaluable, the test question is rejected, thereby reducing the test question misjudgment rate and improving the test question evaluation accuracy.

[0064] As an optional implementation, the embodiments of the present application also propose a test question evaluation method. Referring to FIG. 8, FIG. 2 the method comprises:

[0065] S201, using a pre-trained test question evaluation model, performing test question evaluation on pre-collected test question data to obtain an evaluation result corresponding to the test question data.

[0066] In this embodiment, the pre-trained test question evaluation model is used to perform test question evaluation on pre-collected test question data to obtain an evaluation result corresponding to the test question data. The evaluation result includes a score rate and an evaluation label indicating whether it is evaluable, and can also include a score rate confidence. The test question evaluation model is capable of calculating the confidence of the output result. When predicting the result, the confidence of the output result is calculated.

[0067] S202, if the evaluation label indicates that it is evaluable, and the score rate confidence reaches a pre-determined score rate confidence threshold, the score rate is taken as the evaluation output result corresponding to the test question data.

[0068] In this embodiment, the score rate confidence threshold is pre-determined. When determining the evaluation output result, it is not only necessary to determine whether the evaluation label indicates that it is evaluable, but also necessary to determine whether the score rate confidence reaches the pre-determined score rate confidence threshold. If the evaluation label indicates that it is evaluable, and the score rate confidence reaches the pre-determined score rate confidence threshold, the score rate is taken as the evaluation output result corresponding to the test question data.

[0069] In this embodiment, the score rate confidence threshold can be set artificially according to the experience of professional evaluators, or can be determined according to a pre-set test question evaluable proportion (i.e. the proportion between evaluable test questions and all test questions). That is, a batch of test question samples are input into the test question evaluation model for test question evaluation, and then the number of evaluable test questions and the number of rejected test questions are controlled to meet the test question evaluable proportion. That is, if the pre-determined number of evaluable test questions is large, the corresponding number of evaluable test questions is extracted according to the test question evaluable proportion, and other evaluable test questions are classified as rejected test questions. Then, the minimum value of the score rate confidence corresponding to each test question in all evaluable test questions is taken as the score rate confidence threshold.

[0070] S203, if the evaluation label indicates that it is not evaluable, or the score rate confidence does not reach the pre-determined score rate confidence threshold, a pre-set rejection prompt information is taken as the evaluation output result corresponding to the test question data.

[0071] Only according to the evaluation label to determine whether the test question is evaluable, the evaluation basis may be simple. Therefore, in this embodiment, on the basis of determining whether the test question is evaluable by using the evaluation label, the score rate confidence is added as a judgment basis, which can improve the accuracy of whether the test question is evaluable, further reduce the test question misjudgment rate, and improve the test question evaluation accuracy.

[0072] As an optional implementation, seeFIG. 3 In another embodiment of the present application, the training process of the test question evaluation model is disclosed, and the specific steps are as follows:

[0073] S301, input the first training sample carrying the real score rate collected in advance into the large language model constructed in advance, predict the predicted score rate corresponding to the first training sample, and perform first-stage training on the large language model with the minimum difference between the predicted score rate and the real score rate as the target, to obtain an initial test question evaluation model.

[0074] The embodiment can train the large language model in a two-stage supervised fine-tuning manner to obtain the final test question evaluation model. In the first-stage training, first, the first training sample is collected, which includes the test question stem, the standard answer and the student answer of the sample test question, and carries the real score rate of the sample test question, which can be manually annotated. Then, the first training sample is input into the large language model constructed in advance, and the large language model is used to evaluate the test question of the first training sample to predict the predicted score rate corresponding to the first training sample. Finally, based on the difference between the predicted score rate of the first training sample and the real score rate carried by the first training sample, the model parameters of the large language model are adjusted to obtain the initial test question evaluation model after the first-stage training. That is, the loss function between the predicted score rate of the first training sample and the real score rate carried by the first training sample is calculated, and the model parameters of the large language model are adjusted with the minimum loss function as the target. The loss function between the predicted score rate of the first training sample and the real score rate carried by the first training sample can use mean square error (MSE) loss function or cross-entropy loss function, etc.

[0075] In an example training process, a cosine learning rate decay strategy (initial learning rate 2e-5, minimum learning rate 1e-6) is used, and the batch size is set to 32, and 4 epochs are trained to fully fit the task characteristics. Here is only an example, and the embodiment does not limit the above content.

[0076] S302, input the second training sample carrying the real score rate and the real evaluation label collected in advance into the initial test question evaluation model, predict the predicted score rate and the predicted evaluation label corresponding to the second training sample, and perform second-stage training on the initial test question evaluation model with the minimum difference between the predicted score rate and the real score rate, and the minimum difference between the predicted evaluation label and the real evaluation label as the target, to obtain the test question evaluation model.

[0077] After the first-stage training of the large language model is completed to obtain the initial test question evaluation model, the model still needs to be trained in the second stage. First, second training samples are collected, which include the test question stems, standard answers and student answers of the sample test questions, and carry the real score rates and real evaluation labels of the sample test questions. The real score rates and real evaluation labels can be manually annotated or predicted by the initial test question evaluation model. Then, the second training samples are input into the initial test question evaluation model obtained by the first-stage training, and the initial test question evaluation model is used to evaluate the test questions of the second training samples to predict the predicted score rates and predicted evaluation labels corresponding to the second training samples. Finally, based on the difference between the predicted score rates of the second training samples and the real score rates carried by the second training samples, and the difference between the predicted evaluation labels of the second training samples and the real evaluation labels carried by the second training samples, the model parameters of the initial test question evaluation model are adjusted to obtain the test question evaluation model after the second-stage training, which is the final trained test question evaluation model. That is, the first loss function between the predicted score rates of the second training samples and the real score rates carried by the second training samples, and the second loss function between the predicted evaluation labels of the second training samples and the real evaluation labels carried by the second training samples are calculated, and the value of the weighted sum of the first loss function and the second loss function is minimized to adjust the model parameters of the initial test question evaluation model. The loss function between the predicted score rates of the second training samples and the real score rates carried by the second training samples can use mean square error (MSE) loss function or cross-entropy loss function, and the loss function between the predicted evaluation labels of the second training samples and the real evaluation labels carried by the second training samples can use cross-entropy loss function. The calculation formula of the loss function is as follows:

[0078] L = λ1·MSE(s^,s*) + λ2·CE(c^,c*)

[0079] Wherein, L represents the sum of the loss functions of the second stage training, s^ represents the real score rate, s* represents the predicted score rate, c^ represents the real evaluation label, c* represents the probability distribution of the predicted evaluation label, MSE(s^,s*) represents the mean square error loss function between the predicted score rate and the real score rate, ·CE(c^,c*) represents the cross-entropy loss function between the predicted evaluation label and the real evaluation label, λ1 and λ2 represent the weighting coefficients for balancing the importance of the predicted score rate and the predicted evaluation label.

[0080] Further, in the embodiment, the pre-constructed large language model can adopt a plurality of large language models of different scales, such as Spark-13B (13B parameters), Qwen-14B (14B parameters), Qwen-8B (8B parameters), and Spark-7B (7B parameters). It can be seen that the test question evaluation model constructed in the embodiment has good transferability.

[0081] As an optional implementation, referring to FIG. 4 In another embodiment of the present application, another training process of the test question evaluation model is disclosed, and the specific steps are as follows:

[0082] S401, input the pre-acquired first training sample carrying the real score rate into the pre-constructed large language model, predict the predicted score rate corresponding to the first training sample, and perform first-stage training on the large language model with the minimum difference between the predicted score rate and the real score rate as the target, to obtain an initial test question evaluation model.

[0083] S402, input the pre-acquired second training sample carrying the real score rate into the initial test question evaluation model, and predict the predicted score rate corresponding to the second training sample.

[0084] After the first-stage training of the large language model is completed to obtain the initial test question evaluation model, the second training sample carrying the real score rate needs to be collected first, wherein the second training sample includes the test question stem, the standard answer and the student answer of the sample test question, and the second training sample currently only carries the real score rate, which can be manually annotated. Then, the second training sample is input into the initial test question evaluation model, and the initial test question evaluation model is used to evaluate the sample test question of the second training sample to predict the predicted score rate corresponding to the second training sample.

[0085] S403, based on the comparison result between the difference between the predicted score rate and the real score rate and the preset error threshold, determine the real evaluation label corresponding to the second training sample.

[0086] The embodiment pre-sets an error threshold, that is, a preset error threshold. After the initial test question evaluation model predicts the predicted score rate corresponding to the second training sample, the difference between the predicted score rate corresponding to the second training sample and the real score rate carried by the second training sample is calculated, and the difference is compared with the preset error threshold to determine a comparison result. Based on the comparison result, the real evaluation label corresponding to the second training sample is determined. Specifically, since the initial test question evaluation model has completed the first-stage training and has the prediction performance of the score rate reaching the standard, if the comparison result indicates that the difference between the predicted score rate and the real score rate exceeds the preset error threshold, it is indicated that the difference between the predicted score rate and the real score rate is large at this time due to abnormal conditions such as unclear and incomplete test question data and unclear logical expression, and then it is determined that the real evaluation label of the second training sample indicates unevaluation. If the comparison result indicates that the difference between the predicted score rate and the real score rate does not exceed the preset error threshold, it is indicated that the difference between the predicted score rate and the real score rate is not large at this time, the difference between the prediction result and the real label result is not large, and the test question data is clear and complete, and then it is determined that the real evaluation label of the second training sample indicates evaluation. In the embodiment, the preset error threshold is preferably set to 0.2. The greater the value of the preset error threshold, the greater the probability that the real evaluation label is determined as evaluation, the higher the evaluation rate of the test question, and vice versa.

[0087] In addition, after the initial test question evaluation model predicts the predicted score rate corresponding to the second training sample, the value of the real score rate carried by the second training sample can be replaced with the value of the predicted score rate, so that the consistency of the model evaluation capability can be maintained, and the score distribution deviation caused by the introduction of the real score rate can be avoided. When the data quality of the second training sample is high, the original real score rate can be more appropriate.

[0088] S404, inputting the pre-collected second training sample carrying the real score rate and the real evaluation label into the initial test question evaluation model, predicting a predicted score rate and a predicted evaluation label corresponding to the second training sample, taking the minimum difference between the predicted score rate and the real score rate and the minimum difference between the predicted evaluation label and the real evaluation label as the target, performing second-stage training on the initial test question evaluation model, and obtaining a test question evaluation model.

[0089] The specific execution manners of steps S401 and S404 in the embodiment are the same as those of steps S301 and S302 in the above-described embodiment, and the embodiment will not be described herein again.

[0090] As an optional implementation, refer to FIG. 5As shown, in another embodiment of the present application, the determination process of the score rate confidence threshold is disclosed, which specifically includes the following steps:

[0091] S501, using the test question evaluation model, the test sample carrying the real evaluation result is evaluated, the predicted evaluation result and the classification confidence corresponding to the predicted evaluation label in the predicted evaluation result are obtained.

[0092] The present embodiment first needs to collect test samples carrying real evaluation results, wherein the test samples include the test question stems, standard answers and student answers of sample test questions, and the real evaluation results carried by the test samples include real score rates and real evaluation labels. Then the test sample is input into the pre-trained test question evaluation model, the test sample is evaluated by using the test question evaluation model, the predicted score rate and the predicted evaluation label are obtained, and the classification confidence corresponding to the predicted evaluation label is also obtained.

[0093] In the present embodiment, the evaluation result prediction of the test question evaluation model is generated token by token, for example, the predicted evaluation result "0.75, is <end>" is by '0', '.', '7', '5', ',', 'is', ' <end>The tokens consist of. Let the test item judgment model generate a predicted judgment result string Y = (y1, y2,..., y T ) of length T tokens when given an input sequence X (i.e., the sequence after concatenating the test stem, standard answer, and student answer of a sample test item in the test sample). For example, for the output "0.75", it can correspond to the token sequence: y1 = '0', y2 = '.', y3 = '7', y4 = '5', y5 = ',', y6 = 'is', y7 = ' <end>For each token y t , the conditional probability given by the test question evaluation model is: P(y t |y <t , X). In the above manner, the test question evaluation model can determine the classification confidence corresponding to the predicted evaluation label.

[0094] S502, based on the predicted evaluation label and the classification confidence, the test sample is divided into an evaluatable sample set and a non-evaluatable sample set.

[0095] The embodiment divides the test sample into an evaluatable sample set and a non-evaluatable sample set according to the predicted evaluation label of the test sample predicted by the test question evaluation model and the classification confidence corresponding to the predicted evaluation label.

[0096] Specifically, the embodiment divides the test sample into the evaluatable sample set and the non-evaluatable sample set according to the predicted evaluation label and the classification confidence corresponding to the predicted evaluation label. The test sample whose predicted evaluation label indicates that it is evaluatable and whose classification confidence corresponding to the predicted evaluation label reaches the evaluatable confidence threshold is divided into the evaluatable sample set. The test sample whose predicted evaluation label indicates that it is evaluatable but whose classification confidence corresponding to the predicted evaluation label does not reach the evaluatable confidence threshold is divided into the non-evaluatable sample set. The test sample whose predicted evaluation label indicates that it is non-evaluatable and whose classification confidence corresponding to the predicted evaluation label reaches the non-evaluatable confidence threshold is divided into the non-evaluatable sample set. The test sample whose predicted evaluation label indicates that it is non-evaluatable but whose classification confidence corresponding to the predicted evaluation label does not reach the non-evaluatable confidence threshold is divided into the evaluatable sample set. In the embodiment, the evaluatable confidence threshold is preferably set to 0.9, and the non-evaluatable confidence threshold is preferably set to 0.5. The evaluatable confidence threshold and the non-evaluatable confidence threshold can be adjusted according to actual conditions, and the embodiment does not limit them.

[0097] S503, based on the pre-set test question evaluatable proportion and the predicted score rate confidence in the predicted evaluation result, a score rate confidence threshold is determined.

[0098] In the embodiment, the test question evaluatable proportion, that is, the ratio between the number of evaluatable test questions and the total number of all test questions, is pre-set. Moreover, the confidence of the predicted result determined by the test question evaluation model in the embodiment is the ability of the large language model. When the result is predicted, the confidence of the predicted result will be calculated. Therefore, the test question evaluation model in the embodiment can determine the predicted score rate confidence corresponding to the predicted score rate of the test sample when predicting the score rate of the test sample.

[0099] In order to ensure that the ratio of the evaluable and non-evaluable in the predicted evaluation label of the test question evaluation model in the test question evaluation meets the test question evaluable ratio, the test samples in the evaluable sample set and the non-evaluable sample set need to be divided again after the evaluable sample set and the non-evaluable sample set are determined. That is, if the number of test samples in the evaluable sample set is large, the corresponding number of test samples are extracted from the evaluable sample set as evaluable test samples according to the test question evaluable ratio, and the other test samples in the evaluable sample set are classified as non-evaluable test samples. In the extraction of the corresponding number of test samples from the evaluable sample set according to the test question evaluable ratio, the predicted score rate confidence of the test sample can be used as a basis, and the test sample with a higher predicted score rate confidence is preferably extracted until the corresponding number is reached. Then the minimum value of the score rate confidence corresponding to all evaluable test samples is taken as the score rate confidence threshold. Through the setting of the score rate confidence threshold, in the actual application of the test question evaluation model, it can be ensured that the percentage of the test questions represented by the predicted evaluation label that are evaluable in the total number of test questions evaluated by the test question evaluation model is close to the test question evaluable ratio.

[0100] In one specific embodiment, the present step comprises:

[0101] Firstly, based on the pre-set test question evaluable ratio and the total number of test samples, the number of evaluable test questions is determined.

[0102] In the embodiment, the number of evaluable test questions is determined by calculating the product of the pre-set test question evaluable ratio and the total number of test samples.

[0103] Secondly, the test samples of the number of evaluable test questions are extracted from the evaluable sample set as final evaluable samples in the order of the predicted score rate confidence from large to small.

[0104] In the embodiment, the test samples in the evaluable sample set are sorted in the order of the predicted score rate confidence from large to small, and then the test samples of the number of evaluable test questions are extracted from the evaluable sample set as final evaluable samples according to the sorting result. That is, the test sample with a higher predicted score rate confidence is preferably extracted as a final evaluable sample.

[0105] Thirdly, the minimum value of the predicted score rate confidence corresponding to the final evaluable sample is taken as the score rate confidence threshold.

[0106] Exemplary apparatus

[0107] Correspondingly, the present application also provides a test question evaluation device, as shown in FIG. 6 The device comprises:

[0108] The judgment module 100 is configured to perform test question judgment on the pre-collected test question data by using the pre-trained test question judgment model, to obtain a judgment result corresponding to the test question data, the judgment result including a score rate and a judgment label indicating whether the test question is assessable or not;

[0109] The result determination module 110 is configured to, if the judgment label indicates that the test question is assessable, take the score rate as the judgment output result corresponding to the test question data.

[0110] The result determination module 110 is further configured to, if the judgment label indicates that the test question is not assessable, take the pre-set non-assessment prompt information as the judgment output result corresponding to the test question data.

[0111] The test question judgment model is obtained by performing test question judgment on pre-collected test question samples carrying real judgment results, obtaining predicted judgment results, and training to minimize the difference between the real judgment results and the predicted judgment results.

[0112] As can be seen from the above description, the test question judgment device provided in the embodiments of the present application can predict whether a test question is assessable while obtaining the score rate of the test question. Only when the test question is assessable, the score rate of the test question is output. When the test question is not assessable, non-assessment of the test question is realized. Thus, the test question misjudgment rate is reduced, and the test question judgment accuracy is improved.

[0113] As an optional implementation, in another embodiment of the present application, it is disclosed that the judgment result further includes a score rate confidence; and the result determination module 110 is specifically configured to:

[0114] If the judgment label indicates that the test question is assessable, and the score rate confidence reaches a pre-determined score rate confidence threshold, the score rate is taken as the judgment output result corresponding to the test question data.

[0115] If the judgment label indicates that the test question is not assessable, or the score rate confidence does not reach the pre-determined score rate confidence threshold, the pre-set non-assessment prompt information is taken as the judgment output result corresponding to the test question data.

[0116] As an optional implementation, in another embodiment of the present application, it is disclosed that the test question judgment device further includes a first-stage training module and a second-stage training module.

[0117] The first-stage training module is configured to input a pre-collected first training sample carrying a real score rate into a pre-constructed large language model, predict a predicted score rate corresponding to the first training sample, perform first-stage training on the large language model to minimize the difference between the predicted score rate and the real score rate, and obtain an initial test question judgment model.

[0118] The second stage training module is configured to input the second training sample carrying the real score rate and the real judgment label into the initial test question judgment model, predict the predicted score rate and the predicted judgment label corresponding to the second training sample, and minimize the difference between the predicted score rate and the real score rate and minimize the difference between the predicted judgment label and the real judgment label.

[0119] As an optional implementation, in another embodiment of the present application, the test question judgment device further includes a score rate prediction module and a judgment label determination module.

[0120] The score rate prediction module is configured to input the second training sample carrying the real score rate into the initial test question judgment model and predict the predicted score rate corresponding to the second training sample.

[0121] The judgment label determination module is configured to determine the real judgment label corresponding to the second training sample based on a comparison result between the difference between the predicted score rate and the real score rate and the preset error threshold.

[0122] As an optional implementation, in another embodiment of the present application, the test question judgment device further includes a test sample judgment module, a test sample division module, and a confidence threshold determination module.

[0123] The test sample judgment module is configured to perform test question judgment on the test sample carrying the real judgment result by using the test question judgment model to obtain the predicted judgment result and the classification confidence corresponding to the predicted judgment label in the predicted judgment result.

[0124] The test sample division module is configured to divide the test sample into an assessable sample set and a non-assessable sample set based on the predicted judgment label and the classification confidence.

[0125] The confidence threshold determination module is configured to determine the score rate confidence threshold based on the preset assessable proportion of the test question and the predicted score rate confidence in the predicted judgment result.

[0126] As an optional implementation, in another embodiment of the present application, the confidence threshold determination module is specifically configured to:

[0127] determine the number of assessable test questions based on the preset assessable proportion of the test question and the total number of the test sample;

[0128] extract the test sample of the number of assessable test questions from the assessable sample set as the final assessable sample in the order of the predicted score rate confidence from large to small;

[0129] The minimum value in the score rate confidence corresponding to the final reviewable sample is taken as a score rate confidence threshold.

[0130] The test question evaluation device provided by the embodiment belongs to the same application concept as the test question evaluation method provided by the above-mentioned embodiments of the application, can execute the test question evaluation method provided by any of the above-mentioned embodiments of the application, and has the corresponding function modules and beneficial effects of executing the test question evaluation method. Technical details not described in detail in the embodiment can be referred to the specific processing content of the test question evaluation method provided by the above-mentioned embodiments of the application, which will not be described here.

[0131] Exemplary electronic device

[0132] Another embodiment of the application also provides an electronic device, referring to FIG. 7 The device comprises:

[0133] a memory 200 and a processor 210;

[0134] The memory 200 is connected with the processor 210, and is configured to store a program.

[0135] The processor 210 is configured to realize the test question evaluation method disclosed in any of the above-mentioned embodiments by running the program stored in the memory 200.

[0136] Specifically, the electronic device can further comprise a bus, a communication interface 220, an input device 230 and an output device 240.

[0137] The processor 210, the memory 200, the communication interface 220, the input device 230 and the output device 240 are connected with each other through the bus.

[0138] The bus can comprise a path for transmitting information between various components of the computer system.

[0139] The processor 210 can be a general-purpose processor, such as a general-purpose central processing unit (CPU), a microprocessor, etc., or can be an application-specific integrated circuit (ASIC), or one or more integrated circuits for controlling the program execution of the present application scheme. It can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a ready-to-use programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component.

[0140] The processor 210 can include a main processor, and can also include a baseband chip, a modem, etc.

[0141] The memory 200 stores programs for implementing the technical solutions of the present application, and can also store an operating system and other key services. Specifically, the programs can include program codes, which include computer operation instructions. More specifically, the memory 200 can include a read-only memory (ROM), other types of static storage devices that can store static information and instructions, a random access memory (RAM), other types of dynamic storage devices that can store information and instructions, a disk memory, a flash, and the like.

[0142] The input device 230 can include devices that receive data and information input by a user, such as a keyboard, a mouse, a camera, a scanner, a light pen, a voice input device, a touch screen, a pedometer, or a gravity sensor, and the like.

[0143] The output device 240 can include devices that allow information to be output to a user, such as a display screen, a printer, a speaker, and the like.

[0144] The communication interface 220 can include devices using any transceiver to communicate with other devices or communication networks, such as an Ethernet, a radio access network (RAN), a wireless local area network (WLAN), and the like.

[0145] The processor 210 executes programs stored in the memory 200 and calls other devices, which can be used to implement each step of any one of the test question evaluation methods provided by the above-described embodiments of the present application.

[0146] Exemplary computer program product and storage medium

[0147] In addition to the above methods and devices, the embodiments of the present application can also be computer program products, which include computer program instructions that, when executed by a processor, cause the processor to perform the steps of the test question evaluation methods according to various embodiments of the present application described in the above "Exemplary Methods” section of the present specification.

[0148] The computer program product can be written in any combination of one or more programming languages to perform the operations of the embodiments of the present application, including an object-oriented programming language, such as Java, C++, and the like, and a conventional procedural programming language, such as "C” language or similar programming languages. The program code can be executed entirely on a user computing device, partially on a user device, as an independent software package, partially on a user computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0149] In addition, an embodiment of the present application can also be a storage medium having stored thereon a computer program which, when executed by a processor, performs the steps of the test question evaluation method according to various embodiments of the present application described in the above "Exemplary Method" section of the specification.

[0150] For each of the above method embodiments, in order to simply describe, each is described as a combination of a series of actions, but those skilled in the art should know that the present application is not limited to the order of the actions described, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0151] It should be noted that each of the embodiments in the specification is described in a progressive manner, and each embodiment focuses on the difference from other embodiments, and the same or similar parts between the embodiments can be referred to each other. For the device embodiment, since it is basically similar to the method embodiment, it is described relatively simply, and the relevant parts are referred to the part of the method embodiment.

[0152] The steps in the method of each embodiment of the present application can be adjusted, combined and reduced in sequence according to actual needs, and the technical features recorded in each embodiment can be replaced or combined.

[0153] The modules and sub-modules in the device and terminal of each embodiment of the present application can be combined, divided and reduced according to actual needs.

[0154] In several embodiments provided by the present application, it should be understood that the disclosed terminal, device and method can be implemented by other ways. For example, the terminal embodiments described above are only schematic, and the division of the modules or sub-modules is only a logical function division, and there can be another division way in actual implementation, for example, a plurality of sub-modules or modules can be combined or integrated into another module, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the shown or discussed mutually can be indirect coupling or communication connection through some interfaces, devices or modules, and can be electrical, mechanical or other forms.

[0155] The modules or sub-modules described as separate components can or can not be physically separate, and the components of the modules or sub-modules can or can not be physical modules or sub-modules, that is, they can be located in one place, or can be distributed on a plurality of network modules or sub-modules. Some or all of the modules or sub-modules can be selected according to actual needs to achieve the purpose of the embodiment scheme.

[0156] In addition, each functional module or sub-module in each embodiment of the present application can be integrated in one processing module, or each module or sub-module can exist physically alone, or two or more modules or sub-modules can be integrated in one module. The integrated module or sub-module can be realized in the form of hardware or in the form of a software functional module or sub-module.

[0157] Those skilled in the art will further appreciate that the functions or steps of the examples described in connection with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or any combination thereof. To clearly illustrate the interchangeability of hardware and software, various examples have been described generally in terms of their functionality or steps. Whether such functionality or steps are implemented as hardware or software depends on the particular application and design constraints imposed on the overall system. Skilled persons can implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.

[0158] The steps of a method or algorithm described in connection with the embodiments disclosed herein can be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. A software module can reside in random access memory (RAM), memory, read-only memory (ROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0159] Finally, it should be noted that the terms "first" and "second" and the like in this text are used only to distinguish one entity or operation from another, and do not necessarily require or imply any actual such relationship or order between the entities or operations. Also, the terms "comprises", "comprising", or any other variant thereof are intended to cover non-exclusive inclusions, so that a process, method, article, or apparatus that comprises a list of elements does not only include those elements, but also includes other elements not expressly listed, or other elements inherent in such a process, method, article, or apparatus. Without more limitations, an element defined by the phrase "comprising a" does not exclude the presence of additional identical elements in the process, method, article, or apparatus that includes the element.

[0160] The above description of disclosed embodiments enables one of ordinary skill in the art to make and use the application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein can be applied to other embodiments without departing from the spirit or scope of the application. Thus, the present application is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.< / end> < / end> < / end>

Claims

1. A method for evaluating test questions, characterized in that, include: Using a pre-trained test item evaluation model, test items are evaluated on pre-collected test item data to obtain evaluation results corresponding to the test item data. The evaluation results include: score rate and evaluation labels indicating whether the test item is evaluable. If the evaluation label indicates that it is evaluable, then the score rate is used as the evaluation output result corresponding to the test question data; If the evaluation label indicates that it cannot be evaluated, then the pre-set rejection message will be used as the evaluation output result corresponding to the test question data. The test item evaluation model is obtained by evaluating test item samples with real evaluation results collected in advance, obtaining predicted evaluation results, and training the model with the goal of minimizing the difference between the real evaluation results and the predicted evaluation results.

2. The method according to claim 1, characterized in that, The evaluation results also include: score confidence level; If the evaluation label indicates that the test item is eligible for evaluation, then the score rate is used as the evaluation output result corresponding to the test item data. If the evaluation label indicates that the test item is not eligible for evaluation, then the pre-set rejection message is used as the evaluation output result corresponding to the test item data, including: If the evaluation label indicates that the test is acceptable, and the confidence level of the score rate reaches a predetermined confidence level threshold, then the score rate is used as the evaluation output result corresponding to the test data. If the evaluation label indicates that it cannot be evaluated, or if the confidence level of the score rate does not reach the predetermined confidence level threshold, then the pre-set rejection message will be used as the evaluation output result corresponding to the test question data.

3. The method according to claim 1, characterized in that, The training process of the test item evaluation model includes: The first training sample carrying the true score rate is pre-collected and input into the pre-built large language model to predict the predicted score rate corresponding to the first training sample. The large language model is trained in the first stage with the goal of minimizing the difference between the predicted score rate and the true score rate to obtain the initial test item evaluation model. The second training sample, which carries the true score rate and the true evaluation label, is pre-collected and input into the initial test item evaluation model. The predicted score rate and predicted evaluation label corresponding to the second training sample are predicted. The second stage of training is carried out on the initial test item evaluation model with the goal of minimizing the difference between the predicted score rate and the true score rate, and minimizing the difference between the predicted evaluation label and the true evaluation label, so as to obtain the test item evaluation model.

4. The method according to claim 3, characterized in that, The first training sample, carrying the actual score rate, is pre-collected and input into a pre-constructed large language model to predict the predicted score rate corresponding to the first training sample. The large language model is then trained in the first stage with the objective of minimizing the difference between the predicted score rate and the actual score rate. After obtaining the initial test item evaluation model, the training further includes: The second training sample, which carries the true score rate, is pre-collected and input into the initial test item evaluation model to predict the predicted score rate corresponding to the second training sample. Based on the difference between the predicted score rate and the actual score rate, and the comparison result with the preset error threshold, the true evaluation label corresponding to the second training sample is determined.

5. The method according to claim 2, characterized in that, The process of determining the score confidence threshold includes: Using the aforementioned test item evaluation model, test items are evaluated on pre-collected test samples carrying real evaluation results, and the predicted evaluation results and the classification confidence corresponding to the predicted evaluation labels in the predicted evaluation results are obtained. Based on the predicted evaluation labels and the classification confidence, the test samples are divided into an evaluable sample set and a rejected sample set; The score rate confidence threshold is determined based on the pre-set proportion of test items that can be evaluated and the confidence level of the predicted score rate in the predicted evaluation results.

6. The method according to claim 5, characterized in that, Based on a pre-set proportion of assessable questions and the confidence level of the predicted score rate in the predicted evaluation results, the confidence threshold of the score rate is determined, including: The number of assessable items is determined based on the pre-set proportion of assessable items and the total number of test samples; The test samples of the number of items that can be evaluated are extracted from the set of evaluable samples in descending order of the confidence level of the predicted score rate as the final evaluable samples. The minimum value among the predicted score confidence scores corresponding to the final evaluable samples is used as the score confidence score threshold.

7. A test item evaluation device, characterized in that, include: The evaluation module is used to evaluate the pre-collected test question data using a pre-trained test question evaluation model, and obtain the evaluation result corresponding to the test question data. The evaluation result includes: score rate and evaluation label indicating whether it can be evaluated. The result determination module is used to take the score rate as the evaluation output result corresponding to the test data if the evaluation label indicates that it is evaluable. The result determination module is also used to, if the evaluation label indicates that the test item cannot be evaluated, use the pre-set rejection prompt information as the evaluation output result corresponding to the test item data. The test item evaluation model is obtained by evaluating test item samples with real evaluation results collected in advance, obtaining predicted evaluation results, and training the model with the goal of minimizing the difference between the real evaluation results and the predicted evaluation results.

8. An electronic device, characterized in that, include: Memory and processor; The memory is connected to the processor and is used to store programs; The processor is configured to implement the test item evaluation method as described in any one of claims 1 to 6 by running the program in the memory.

9. A storage medium, characterized in that, The storage medium stores a computer program, which, when executed by a processor, implements the test question evaluation method as described in any one of claims 1 to 6.

10. A computer program product, characterized in that, It includes computer program instructions that, when executed by a processor, cause the processor to implement the test item evaluation method as described in any one of claims 1 to 6.

Images