Test question correcting method and device, computer equipment and readable storage medium

Abstract

The invention provides a test question correcting method and device, computer equipment and a readable storage medium, and relates to the field of artificial intelligence. The method comprises the steps of obtaining test question information of test questions; based on the test question information, performing question type classification on the test questions, and determining question types of the test questions, the question types of the test questions including at least one of semantic similar questions, open questions and objective questions; and correcting the test questions according to the correction modes corresponding to the question types of the test questions.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence, and in particular to a test question grading method, apparatus, computer equipment and readable storage medium. Background Art

[0002] With the development of educational technology, manual marking of homework is not only time-consuming and labor-intensive, but also easily affected by subjective factors, resulting in inconsistent scores and delayed feedback. Automated assessment systems play an increasingly important role in modern teaching. In related technologies, image recognition is used to mark homework. The test image is sent to the recognition model to obtain the content of each question, and then the question is judged based on the content of the question. However, this method is not targeted at different question types, resulting in low marking accuracy. Summary of the invention

[0003] In view of this, the present application provides a test question grading method, apparatus, computer device and readable storage medium, which solves the problem in the related art that there is no targeted grading for different question types, resulting in low grading accuracy.

[0004] In a first aspect, an embodiment of the present application provides a method for correcting test questions, comprising:

[0005] Get the test information of the test questions;

[0006] Classifying the test questions based on the test question information to determine the test question type, wherein the test question type includes at least one of a semantically similar question, an open question, and an objective question;

[0007] The test questions are corrected according to the correction mode corresponding to the question type of the test questions.

[0008] In a second aspect, an embodiment of the present application provides a test question grading device, comprising:

[0009] An information acquisition module, used to acquire test information of test questions;

[0010] A question type classification module, used to classify the question type of the question based on the question information, and determine the question type of the question, wherein the question type of the question includes at least one of a semantically similar question, an open question, and an objective question;

[0011] The marking module is used to mark the test questions according to the marking mode corresponding to the question type of the test questions.

[0012] In a third aspect, an embodiment of the present application provides a computer device, which includes a processor and a memory, wherein the memory stores programs or instructions that can be executed on the processor, and when the programs or instructions are executed by the processor, the steps of the method of the first aspect are implemented.

[0013] In a fourth aspect, an embodiment of the present application provides a readable storage medium, on which a program or instruction is stored, and when the program or instruction is executed by a processor, the steps of the method of the first aspect are implemented.

[0014] In an embodiment of the present application, the test information of the test question is obtained, and the test information includes the stem of the test question, the user answer generated by the user answering the test question, and the standard answer corresponding to the test question. The test questions are classified into question types according to the test information, and different question types correspond to their own correction modes. After determining the question type of the test question, the test question can be corrected according to the correction mode corresponding to the question type. In the embodiment of the present application, it is possible to achieve targeted correction of test questions of different question types, thereby improving the accuracy of test question correction. In particular, the embodiment of the present application can correct semantically similar questions and open questions, reduce the difficulty of correcting semantically similar questions and open questions, and improve the efficiency of test question correction.

[0015] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0017] Figure 1 One of the flowcharts of the test question correction method according to an embodiment of the present application is shown;

[0018] Figure 2 A schematic diagram showing the process flow of the dictionary and the model working together in an embodiment of the present application;

[0019] Figure 3 A schematic diagram of a large language binary classification model according to an embodiment of the present application is shown;

[0020] Figure 4 The second flowchart of the test question correction method according to the embodiment of the present application is shown;

[0021] Figure 5 A structural block diagram of a test question grading device according to an embodiment of the present application is shown;

[0022] Figure 6 A structural block diagram of a computer device according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0023] The following will be combined with the drawings in the embodiments of the present application to clearly describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the present application belong to the scope of protection of this application.

[0024] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0025] The following, in conjunction with the accompanying drawings, describes in detail the test question grading method, apparatus, computer device and readable storage medium provided in the embodiments of the present application through specific embodiments and their application scenarios. In the absence of conflict, the following embodiments and features in the embodiments may be combined with each other.

[0026] The present application embodiment provides a method for correcting test questions, such as Figure 1 As shown, the method includes:

[0027] Step 101, obtaining the test information of the test questions.

[0028] In this step, the test information of the test question is obtained, and the test information includes the stem of the test question, the user answer generated by the user in answering the test question, and the standard answer corresponding to the test question. The user answer is the answer submitted by the received user, and the standard answer is the correct answer pre-stored in the database.

[0029] Step 102, classifying the test questions based on the test question information to determine the test question type, wherein the test question type includes at least one of a semantically similar question, an open question, and an objective question.

[0030] In this step, the test questions are classified into question types according to the test question information, for example, the question type is determined according to the test question stem. In the embodiment of the present application, the question types of the test questions include but are not limited to semantic similarity questions, open questions and objective questions, wherein semantic similarity questions refer to questions whose answers are not unique and are considered correct as long as the semantics of the user's answer is similar to the standard answer, open questions refer to questions that do not set a unique, fixed answer and the semantics of the user's answer may not be similar to the semantics of the standard answer, and objective questions refer to questions whose answers are clear and unique.

[0031] Step 103, correcting the test questions according to the correction mode corresponding to the question type.

[0032] In this step, different question types have their own corresponding grading modes. After the question type is determined, the question can be graded according to the grading mode corresponding to the question type.

[0033] The embodiment of the present application can realize targeted correction of test questions of different types, thereby improving the accuracy of test question correction. In particular, the embodiment of the present application can correct semantically similar questions and open questions, thereby reducing the difficulty of correcting semantically similar questions and open questions and improving the efficiency of test question correction.

[0034] In one embodiment of the present application, the test questions are graded according to the grading mode corresponding to the question type, including:

[0035] If the test question is a semantically similar question, it will be graded according to the marking mode for semantically similar questions;

[0036] If the test question is an open-ended question, it will be graded according to the open-ended question grading mode;

[0037] If the test question is an objective question, it will be marked according to the objective question marking model.

[0038] In this embodiment, the types of test questions include semantic similarity questions, open questions and objective questions, and correspondingly, the grading modes include semantic similarity question grading mode, open question grading mode and objective question grading mode. Different grading modes are used to grade different types of test questions, so that the grading logic of the test questions is more targeted and the accuracy of the grading is guaranteed.

[0039] In one embodiment of the present application, if the test question is a semantically similar question, the test question is corrected according to the semantically similar question correction mode, including:

[0040] The user answers and standard answers of the test questions are segmented to obtain the first keyword of the user answers and the second keyword of the standard answers;

[0041] Traversing the first keywords of the user's answers, for any first keyword, performing a semantic similarity comparison with each second keyword of the standard answer to obtain the semantic similarity, and determining whether the first keyword is a valid answer based on the semantic similarity;

[0042] Count the number of first keywords that are valid answers, and calculate the accuracy of the user's answer based on the number and the total number of second keywords;

[0043] If the answer accuracy of the user's answer is greater than or equal to the second preset threshold, the user's answer is determined to be correct.

[0044] In one embodiment, a method for determining whether a first keyword is a valid answer based on semantic similarity includes: determining a target semantic similarity based on the semantic similarity between the first keyword and each second keyword; if the target semantic similarity is less than a first preset threshold, determining that the answer with the first keyword is invalid; if the target semantic similarity is greater than or equal to the first preset threshold, determining that the first keyword is a valid answer.

[0045] In this embodiment, the semantic similarity technology is based on natural language processing and machine learning algorithms, which can effectively understand and compare the meaning of text content, rather than just superficial vocabulary matching. This technology can determine the degree of similarity between the user's answer and the standard answer by analyzing the semantic relationship between the two, thereby achieving automatic correction. The correction mode for semantic similarity questions includes the following steps:

[0046] Step 1: Enter the user answer and the standard answer.

[0047] Step 2: Segment the user answer and the standard answer separately, remove stop words, and the remaining words are keywords, obtaining the first keyword of the user answer and the second keyword of the standard answer.

[0048] Step 3, traverse the first keyword in the user's answer, take out the current first keyword, compare the first keyword with each second keyword in the standard answer, obtain the semantic similarity between the first keyword and the second keyword, and determine whether the first keyword is a valid answer based on the semantic similarity. Specifically, according to the semantic similarity between the first keyword and each second keyword, determine the target semantic similarity, the target semantic similarity can be the maximum semantic similarity, take the second keyword corresponding to the target semantic similarity to match the answer to the first keyword, and determine whether the target semantic similarity is greater than or equal to the first preset threshold. If it is greater than or equal to the first preset threshold, the first keyword is determined to be a valid answer, that is, the user answered a keyword correctly; otherwise, the first keyword answer is determined to be invalid, that is, the first keyword is an invalid answer.

[0049] It should be noted that the matched second keyword will no longer participate in the subsequent comparison with other first keywords to prevent multiple first keywords from being counted multiple times when they match the same second keyword of the standard answer.

[0050] Exemplarily, the current first keyword of the user's answer is UW, and the second keyword list of the standard answer is AW1, AW2, AW3...AWn. The similarity between UW and AW1 is calculated, assuming S1; the similarity between UW and AW2 is calculated, assuming S2; the similarity between UW and AW3 is calculated, assuming S3; the similarity between UW and AWn is calculated, assuming Sn. Select the maximum value from S1, S2, S3...Sn, assuming Si, and determine whether Si is greater than the first preset threshold. If it is less than the first preset threshold, it means that the current first keyword UW is an invalid answer. If it is greater than or equal to the first preset threshold, the current first keyword UW matches the i-th second keyword in the standard answer, indicating that the current first keyword is a valid answer, and the i-th second keyword in the standard answer no longer participates in subsequent comparisons.

[0051] Step 4, repeat the operation in step 3 until the first keyword in all user answers is traversed.

[0052] Step 5: Count the number of first keywords that are valid answers and the total number of second keywords in the standard answers, and calculate the accuracy of the user's answer. For example, if the user correctly answers N first keywords, the number of second keywords in the standard answers is M, and the accuracy of the user's answer is R, then R = N / M. Further, determine whether the accuracy of the user's answer is greater than or equal to a second preset threshold. If it is greater than or equal to the second preset threshold, the user's answer is determined to be correct, otherwise the user's answer is determined to be incorrect.

[0053] In the embodiment of the present application, for semantically similar questions, a semantic similarity comparison of keywords is performed between the user's answer and the standard answer, thereby achieving automatic correction of semantically similar questions.

[0054] In one embodiment of the present application, for any first keyword, a semantic similarity comparison is performed with each second keyword of the standard answer to obtain the semantic similarity, including:

[0055] Determine whether the first keyword exists in the key value of the synonym dictionary, wherein the key value of the synonym dictionary is the keyword, and the value of the synonym dictionary is the synonym list corresponding to the keyword;

[0056] If the first keyword exists in the key value of the synonym dictionary, then check whether the second keyword exists in the synonym list corresponding to the first keyword; if the second keyword exists, the first keyword and the second keyword are combined into a synonym pair;

[0057] The semantic similarity of the keyword pair is searched in the similarity dictionary, wherein the key value of the similarity dictionary is the synonym pair, and the value of the synonym dictionary is the semantic similarity corresponding to the synonym pair.

[0058] In this embodiment, a method of calculating the semantic similarity between the first keyword of the user answer and the second keyword of the standard answer is defined.

[0059] Prepare a synonym word library in advance and store it in a dictionary data structure. The key value of the dictionary is the keyword, and the value is the synonym list corresponding to the keyword (i.e., the key value). Each synonym in the synonym list is swapped with the keyword corresponding to the key value to form a new key value and value pair. In this way, a synonym dictionary can be formed, and each word in the synonym dictionary will appear once as a key value.

[0060] Enter the first keyword of the user's answer, and determine whether the first keyword appears in the key value of the synonym dictionary. If the key value of the synonym dictionary contains the first keyword, then check whether there is a second keyword of the standard answer in the synonym list corresponding to the first keyword. If it exists, the first keyword and the second keyword are combined into a new string, that is, a synonym pair.

[0061] Search the similarity dictionary for the semantic similarity of the synonym pair. The key value of the similarity dictionary is the synonym pair, and the value value is the semantic similarity corresponding to the synonym pair (i.e., the key value). The corresponding semantic similarity of the synonym pair consisting of each key keyword in the synonym dictionary and any keyword in the value can be found in the similarity dictionary. For example, the key values ​​of quiet and tranquility in the similarity dictionary are quiet and tranquility, and the value value is the semantic similarity of the synonym pair of quiet and tranquility.

[0062] In the embodiment of the present application, a method in which a synonym dictionary and a similarity dictionary work in collaboration is used to accurately obtain the semantic similarity between the first keyword of the user's answer and the second keyword of the standard answer.

[0063] In one embodiment of the present application, for any first keyword, a semantic similarity comparison is performed with each second keyword of the standard answer to obtain the semantic similarity, further comprising:

[0064] If the first keyword does not exist in the key value of the synonym dictionary, or the semantic similarity of the keyword pair is not found in the similarity dictionary, the first keyword and the second keyword are input into the trained synonym recognition model to output the semantic similarity.

[0065] In this embodiment, if the key value of the synonym dictionary does not contain the first keyword of the user's answer, or the semantic similarity of the keyword pair is not found in the similarity dictionary, the first keyword and the second keyword are input into the trained synonym recognition model, and the similarity between the first keyword and the second keyword is output. The synonym recognition model used in the embodiment of the present application can be a StructBERT model, which is trained using the collected synonym pairs. In actual use, other models can also be used.

[0066] In the embodiment of the present application, when calculating the semantic similarity between the keywords of the user's answer and each keyword in the standard answer, first, each first keyword of the user's answer and each second keyword in the standard answer are combined into a keyword pair. Assuming that there are N second keywords in the standard answer, N keyword pairs are obtained, and then these N keyword pairs are searched in the similarity dictionary in sequence. If there is at least one keyword pair whose semantic similarity is greater than or equal to the second preset threshold, the keyword pair that is not found in the similarity dictionary will no longer be subjected to the similarity calculation of the synonym recognition model, because at this time the keyword of the user's answer has been successfully matched with the keyword in the standard answer. Otherwise, the keyword pair that is not found in the similarity dictionary needs to be subjected to the similarity calculation of the synonym recognition model to further determine whether the keyword of the user's answer is valid.

[0067] In the embodiment of the present application, by first collectively searching whether the user answer keywords and the standard answer keywords are synonyms and then performing model calculation, that is, a method in which the dictionary and the model work together, semantic similarity can be obtained and the computing efficiency of the system can be further improved.

[0068] In one embodiment of the present application, the method further includes:

[0069] When the semantic similarity output by the synonym recognition model is greater than or equal to a third preset threshold, outputting the first keyword and the second keyword to a first manual proofreading list for manual proofreading;

[0070] After the first manual proofreading information is obtained, the first keyword and the second keyword are entered into a similarity dictionary, or a synonym recognition model is adjusted according to the first keyword and the second keyword.

[0071] In this embodiment, when the semantic similarity output by the synonym recognition model is greater than or equal to a third preset threshold, the first keyword and the second keyword are output to a first manual proofreading list for manual proofreading.

[0072] After manual proofreading, it is entered into the similarity dictionary. For keyword pairs that do not appear in the similarity dictionary, when the semantic similarity of the keyword pair calculated by the synonym recognition model is greater than or equal to the third preset threshold, it is judged to be a valid answer. Through manual proofreading, it can be confirmed whether the calculation result of the synonym recognition model is correct. If the calculation is wrong, the result can be corrected by entering the similarity dictionary. The next time the keyword pair is calculated, the calculation error can be avoided, because when calculating the semantic similarity of the keyword pair, the similarity dictionary will be queried first. If the semantic similarity is obtained, the model calculation will not be performed again. This method can avoid the model from making multiple incorrect judgments on keyword pairs that are not synonyms. In other words, for keyword pairs that are not synonyms, they will only be misjudged once at most.

[0073] By entering the similarity dictionary after manual proofreading, the similarity dictionary is enriched, which can greatly save time in the subsequent determination of the semantic similarity between the first keyword of the user's answer and the second keyword of the standard answer, without having to call the synonym recognition model for calculation every time.

[0074] After manual proofreading, we can pick out the keyword pairs that are misjudged by the model, that is, they are not synonym groups but are mistakenly judged as synonym groups. These misjudged keyword pairs are collected and can be used to tune the synonym recognition model every once in a while or when a certain amount is collected, so as to improve the recognition accuracy of the synonym recognition model.

[0075] like Figure 2 As shown, in the embodiment of the present application, for each keyword pair consisting of the first keyword of the user's answer and each second keyword in the standard answer, it is determined whether the semantic similarity of the keyword pair exists in the similarity dictionary. If it exists, the semantic similarity is output; if it does not exist, the semantic similarity is calculated using the synonym recognition model, and it is determined whether the semantic similarity output by the synonym recognition model is greater than a certain threshold. If it is greater, manual proofreading is performed, and then on the one hand, it can be entered into the similarity dictionary, and on the other hand, the misjudged erroneous results can be used to tune the synonym recognition model to improve the recognition accuracy of the synonym recognition model.

[0076] In one embodiment of the present application, if the test question is an open question, the test question is corrected according to the open question correction mode, including:

[0077] Classifying the test questions into sub-question types based on the test question information to determine the sub-question types of the test questions, wherein the sub-question types of the test questions include at least one of a word formation question, a sentence formation question, and other open questions;

[0078] Correct the test questions according to the correction mode corresponding to the sub-question type of the test question.

[0079] In this embodiment, for open questions, the present application corrects them from three aspects: whether the wording is accurate, whether the sentence is fluent, and whether the semantics is consistent with the question.

[0080] If the test question is an open question, the test question is further classified into sub-question types based on the test question information to determine the sub-question type of the test question. For example, based on the stem of the open question, it is divided into word formation questions, sentence formation questions, or sentence completion questions. And the test question is corrected according to the correction mode corresponding to the sub-question type.

[0081] In the embodiment of the present application, various types of open questions are graded through different grading modes, so that the grading logic of the open questions is more targeted and the accuracy of the grading is guaranteed.

[0082] In one embodiment of the present application, if the test question is a word combination question, the test question is corrected according to the correction mode corresponding to the sub-question type of the test question, including:

[0083] Determine whether the user's answer to the test question contains the characters that need to be formed into words in the test question stem;

[0084] If not, the user's answer is considered wrong;

[0085] If it is included, determine whether the user answer exists in the word list dictionary. The key value of the word list dictionary is the word, and the value value is the right or wrong value of the word; if the user answer exists, determine whether the user answer is correct based on the right or wrong value corresponding to the user answer; if the user answer does not exist, use the large language binary classification model to determine whether the user answer is correct.

[0086] In this embodiment, if the open question is determined to be a word-forming question based on the question stem, the user's answer, that is, the words the user answered, is determined whether it contains the characters that need to be formed in the question stem of the test question. If it does not contain the characters that need to be formed, it is directly determined to be wrong. If it contains the characters that need to be formed, it is determined whether the words the user answered are in the word list dictionary. Among them, the word list dictionary is a list of Chinese words collected in advance, and the list is presented in the form of a dictionary, which is convenient for quickly finding whether the words answered by the user are correct. The key value of the word list dictionary is the word, and the value value is the right or wrong value of the word (that is, the key value).

[0087] If the word the user answered is in the word list dictionary, the corresponding value is used to determine whether the word the user answered is correct; if the word the user answered is not in the word list dictionary, the large language binary classification model is used to determine whether the word the user answered is correct, and the final judgment result of the large language binary classification model is used as the final correction result.

[0088] The embodiment of the present application can accurately and efficiently correct word combination questions through the coordinated use of a word list dictionary and a large language binary classification model.

[0089] In one embodiment of the present application, if the test question is a sentence-making question, the test question is corrected according to the correction mode corresponding to the sub-question type of the test question, including:

[0090] Determine whether the user's answer to the test question contains the words required to form a sentence in the test question stem;

[0091] If not, the user's answer is considered wrong;

[0092] If included, the sentence coherence small model is used to determine whether the user's answer is fluent; if the user's answer is fluent, the user's answer is determined to be correct; if the user's answer is not fluent, the large language binary classification model is used to determine whether the user's answer is correct.

[0093] In this embodiment, if the open question is determined to be a sentence-making question according to the question stem, the user's answer, that is, the sentence given by the user, is determined whether it contains the words required to make a sentence in the question stem of the test question. If not, the user's answer is directly determined to be wrong. If it is included, the sentence given by the user is input into a sentence coherence model for determining whether the sentence is coherent. The sentence coherence model used in this application can be a Fasttext model, and other models can also be used in actual application.

[0094] If the sentence fluency model determines that the user's answer is fluent, the user's answer is considered correct. If the sentence fluency model determines that the user's answer is not fluent, the large language binary classification model is used to make a second judgment on the user's answer to further determine whether the user's answer is accurate, and the final judgment result of the large language binary classification model is used as the final correction result.

[0095] The embodiment of the present application can accurately and efficiently correct sentence construction questions through the coordinated use of a small sentence coherence model and a large binary classification model.

[0096] In one embodiment of the present application, if the test question is other open questions, the test question is corrected according to the correction mode corresponding to the sub-question type of the test question, including:

[0097] Use the sentence coherence model to determine whether the user's answer is coherent; if the user's answer is coherent, the user's answer is correct; if the user's answer is not coherent, a preset percentage of the score will be deducted;

[0098] The semantic model is used to determine whether the user's answer is consistent with the question; if the user's answer is consistent with the question, the user's answer is determined to be correct; if the user's answer is not consistent with the question, the large language binary classification model is used to determine whether the user's answer is correct.

[0099] In this embodiment, for other open questions such as sentence completion, the user's answer, that is, the sentence the user answered, is input into a sentence coherence model for judging whether the sentence the user answered is coherent, to judge whether the sentence the user answered is coherent. If it is judged to be incoherent, a certain percentage of the score for the question will be deducted; if it is judged to be coherent, no score will be deducted. Then, the semantic model is used to judge whether the sentence the user answered is consistent with the meaning of the question. If it is determined to be consistent with the meaning of the question, the user's answer is judged to be correct; if it is determined to be inconsistent with the meaning of the question, the large language binary classification model is used for further judgment, and the result of the large language binary classification model correction is used as the final result.

[0100] The embodiment of the present application, through the coordinated use of the sentence coherence small model, the semantic model and the binary classification large model, can accurately and efficiently correct other open questions such as sentence completion.

[0101] In one embodiment of the present application, the method also includes: training a large language binary classification model; wherein the large language binary classification model includes a large language model, a feature comparison model, and a binary classification output layer, the large language model is used to determine whether the user's answer is consistent with the question, the feature comparison model is used to compare the user's answer with the standard answer, and the binary classification output layer is used to output the correct value of the user's answer.

[0102] In this embodiment, according to the open question correction scenario, the final output layer of the large language model decoder is modified, because this application uses the semantic understanding ability of the large language model to output whether the user's answer meets the question meaning, and no further output is required. Therefore, the final output of the application scenario of this application is a binary classification, and the output is the confidence level of meeting the question meaning.

[0103] The structure of the large language binary classification model of this application is as follows Figure 3 As shown in the figure, on the basis of the large language model, a feature comparison model and a binary classification output layer are added. The input of the large language model is the question stem and the user's answer. The large language model can be the Qianwen 7B model. Other models can also be used in actual applications. The input of the feature comparison model is the user's answer and the correct answer (that is, the standard answer). It is used to extract the features of the user's answer and the features of the correct answer, and compare them. During training, only the parameters of the output layer of the large language model, the feature comparison model, and the binary classification output layer need to be trained, thereby obtaining a binary classification model based on the large language model.

[0104] It should be noted that in order to make full use of the semantic understanding ability of the large language model, the parameters before the output layer of the large language model can be frozen, and the feature comparison model can be trained separately until convergence, and then merged with the large language model branch, and then jointly trained. This makes full use of the ability to compare user answers with standard answers, and the ability of the large language model to determine whether the user's answer is in line with the meaning of the question. The two are combined for binary classification to achieve more optimized results.

[0105] The embodiment of the present application provides a method for combining a large language model with a feature comparison model for binary classification, which not only improves the accuracy of the judgment, but also avoids the problem of unstable output of the large language model. Because the large language model may output different results when the same input is used for multiple forward reasoning, and sometimes it will output the intermediate process. The reason why the large language model runs relatively slowly is that it outputs more content. The more content is output, the longer it takes. There is always a failure to control the output of the large language model through input prompts or guidance information. By adding a new output layer, this problem can be solved.

[0106] In one embodiment of the present application, the method further includes: when the user answer is determined to be correct using the large language binary classification model, outputting the user answer and the correct value to a second manual proofreading list for manual proofreading; after obtaining the second manual proofreading information, entering the user answer and the correct value into a word list dictionary, or adjusting the large language binary classification model according to the user answer and the correct value.

[0107] In each of the above embodiments, after the large language binary classification model is used to determine whether the user's answer is correct, the user's answer and the correct value are output to a manual proofreading list for manual proofreading at a later time.

[0108] The user answers and correct values ​​after manual proofreading are returned to the word list dictionary. As time goes by, the word list dictionary becomes more and more complete, and fewer and fewer sentences need to be corrected by the large language binary classification model, thereby improving the efficiency of system operation and the accuracy of correction.

[0109] The manually proofread user answers and correct values ​​can also be used regularly to adjust the large language binary classification model to improve the recognition accuracy of the large language binary classification model.

[0110] In one embodiment of the present application, if the test question is an objective question, the test question is corrected according to the objective question correction mode, including:

[0111] If the test question is an objective question, the user answer and standard answer of the test question are converted into strings;

[0112] Compare the character string of the user's answer with the character string of the standard answer to correct the test questions.

[0113] In this embodiment, if the test question is an objective question, the user answer and the standard answer are uniformly converted into a string form, and interference factors are removed, such as removing extra spaces, punctuation marks, etc.

[0114] The string of the user's answer is compared with the string of the standard answer. For example, a string equality check method such as the == operator in Python may be used directly. It should be noted that when comparing, it is possible to decide whether to ignore the case difference of the string according to the requirements.

[0115] After comparison, the result of grading the objective questions can be output as a result, for example, a Boolean value or a specific identifier (such as "correct" / "wrong").

[0116] In one embodiment of the present application, if the question types include objective questions and / or semantically similar questions, the method further includes:

[0117] After the test questions are graded according to the grading mode corresponding to the question type, if the grading result is that the user's answer to the test question is wrong, the test questions are graded a second time; wherein, the method of performing a second grading on the test questions includes: performing a second grading on the test questions using a feature comparison model of a large language binary classification model.

[0118] In this embodiment, if Figure 4 As shown, for objective questions and / or semantically similar questions, if they are judged to be wrong, they will be corrected twice to prevent the situation where the correct answer is wrongly judged.

[0119] The correction results of objective questions and semantically similar questions are questions that users answered incorrectly and have low confidence. The answer comparison branch in the correction mode of open questions, that is, the feature comparison model, is used for secondary correction, and the result of the secondary correction is used as the final correction result. This method can prevent the situation of wrong judgment without increasing the running time too much.

[0120] The reason why the feature comparison model can be used for secondary correction is that before the joint training of the correction mode for open questions, the feature comparison model is trained to a convergent state using all user answers and corresponding standard answers. In this way, the feature comparison model can not only correct objective questions and semantically similar questions, but also extract the features of user answers and standard answers.

[0121] As a specific implementation of the above-mentioned test question correction method, the present application embodiment provides a test question correction device. Figure 5 As shown, the test question grading device 500 includes: an information acquisition module 501 , a question type classification module 502 and a grading module 503 .

[0122] The information acquisition module 501 is used to acquire the test information of the test questions;

[0123] The question type classification module 502 is used to classify the question types based on the question information and determine the question types of the question, wherein the question types of the question include at least one of semantically similar questions, open questions and objective questions;

[0124] The marking module 503 is used to mark the test questions according to the marking mode corresponding to the test question type.

[0125] Furthermore, the correction module 503 is specifically used for:

[0126] If the test question is a semantically similar question, it will be graded according to the marking mode for semantically similar questions;

[0127] If the test question is an open-ended question, it will be graded according to the open-ended question grading mode;

[0128] If the test question is an objective question, it will be marked according to the objective question marking model.

[0129] Furthermore, the correction module 503 is specifically used for:

[0130] The user answers and standard answers of the test questions are segmented to obtain the first keyword of the user answers and the second keyword of the standard answers;

[0131] Traversing the first keywords of the user's answers, for any first keyword, performing a semantic similarity comparison with each second keyword of the standard answer to obtain the semantic similarity, and determining whether the first keyword is a valid answer based on the semantic similarity;

[0132] Count the number of first keywords that are valid answers, and calculate the accuracy of the user's answer based on the number and the total number of second keywords;

[0133] If the answer accuracy of the user's answer is greater than or equal to the second preset threshold, the user's answer is determined to be correct.

[0134] Furthermore, the correction module 503 is specifically used for:

[0135] Determining a target semantic similarity based on the semantic similarity between the first keyword and each second keyword;

[0136] If the target semantic similarity is less than the first preset threshold, the first keyword answer is determined to be invalid;

[0137] If the target semantic similarity is greater than or equal to a first preset threshold, the first keyword is determined to be a valid answer.

[0138] Furthermore, the correction module 503 is specifically used for:

[0139] Determine whether the first keyword exists in the key value of the synonym dictionary, wherein the key value of the synonym dictionary is the keyword, and the value of the synonym dictionary is the synonym list corresponding to the keyword;

[0140] If the first keyword exists in the key value of the synonym dictionary, then check whether the second keyword exists in the synonym list corresponding to the first keyword; if the second keyword exists, the first keyword and the second keyword are combined into a synonym pair;

[0141] The semantic similarity of the keyword pair is searched in the similarity dictionary, wherein the key value of the similarity dictionary is the synonym pair, and the value of the synonym dictionary is the semantic similarity corresponding to the synonym pair.

[0142] Furthermore, the review module 503 is also used for:

[0143] If the first keyword does not exist in the key value of the synonym dictionary, or the semantic similarity of the keyword pair is not found in the similarity dictionary, the first keyword and the second keyword are input into the trained synonym recognition model to output the semantic similarity.

[0144] Furthermore, the device further comprises: an output proofreading module, configured to: when the semantic similarity output by the synonym recognition model is greater than or equal to a third preset threshold, output the first keyword and the second keyword to a first manual proofreading list for manual proofreading;

[0145] An input module, used to: after obtaining the first manual proofreading information, input the first keyword and the second keyword into a similarity dictionary;

[0146] The model training module is used to adjust the synonym recognition model according to the first keyword and the second keyword.

[0147] Furthermore, the correction module 503 is specifically used for:

[0148] Classifying the test questions into sub-question types based on the test question information to determine the sub-question types of the test questions, wherein the sub-question types of the test questions include at least one of a word formation question, a sentence formation question, and other open questions;

[0149] Correct the test questions according to the correction mode corresponding to the sub-question type of the test question.

[0150] Furthermore, the correction module 503 is specifically used for:

[0151] Determine whether the user's answer to the test question contains the characters that need to be formed into words in the test question stem;

[0152] If not, the user's answer is considered wrong;

[0153] If it is included, determine whether the user answer exists in the word list dictionary. The key value of the word list dictionary is the word, and the value value is the right or wrong value of the word; if the user answer exists, determine whether the user answer is correct based on the right or wrong value corresponding to the user answer; if the user answer does not exist, use the large language binary classification model to determine whether the user answer is correct.

[0154] Furthermore, the correction module 503 is specifically used for:

[0155] Determine whether the user's answer to the test question contains the words required to form a sentence in the test question stem;

[0156] If not, the user's answer is considered wrong;

[0157] If included, the sentence coherence small model is used to determine whether the user's answer is fluent; if the user's answer is fluent, the user's answer is determined to be correct; if the user's answer is not fluent, the large language binary classification model is used to determine whether the user's answer is correct.

[0158] Furthermore, the correction module 503 is specifically used for:

[0159] Use the sentence coherence model to determine whether the user's answer is coherent; if the user's answer is coherent, the user's answer is correct; if the user's answer is not coherent, a preset percentage of the score will be deducted;

[0160] The semantic model is used to determine whether the user's answer is consistent with the question; if the user's answer is consistent with the question, the user's answer is determined to be correct; if the user's answer is not consistent with the question, the large language binary classification model is used to determine whether the user's answer is correct.

[0161] Furthermore, the model training module is also used to: train a large language binary classification model; wherein the large language binary classification model includes a large language model, a feature comparison model and a binary classification output layer, the large language model is used to determine whether the user's answer is consistent with the question, the feature comparison model is used to compare the user's answer with the standard answer, and the binary classification output layer is used to output the correct value of the user's answer.

[0162] Furthermore, the output proofreading module is further used to: when the user answer is determined to be correct by using the large language binary classification model, output the user answer and the correct value to the second manual proofreading list for manual proofreading;

[0163] The entry module is further used to: after obtaining the second manual proofreading information, enter the user's answer and the correct value into the word list dictionary;

[0164] The model training module is also used to adjust the large language binary classification model based on user answers and correct values.

[0165] Furthermore, the correction module 503 is specifically used for:

[0166] If the test question is an objective question, the user answer and standard answer of the test question are converted into strings;

[0167] Compare the character string of the user's answer with the character string of the standard answer to correct the test questions.

[0168] Furthermore, the review module 503 is also used for:

[0169] If the test questions include objective questions and / or semantically similar questions, after the test questions are corrected according to the correction mode corresponding to the test question type, if the correction result is that the user's answer to the test question is wrong, the test questions are corrected for the second time; wherein, the method of correcting the test questions for the second time includes: using a feature comparison model of a large language binary classification model to correct the test questions for the second time.

[0170] The test question correction device 500 in the embodiment of the present application can be a computer device, or a component in the computer device, such as an integrated circuit or a chip. The computer device can be a terminal, or other devices other than the terminal. Exemplarily, the computer device can be a mobile phone, a tablet computer, a laptop computer, a handheld computer, a vehicle-mounted computer device, a mobile Internet device (Mobile Internet Device, MID), an augmented reality (Augmented Reality, AR) / virtual reality (Virtual Reality, VR) device, a robot, a wearable device, an ultra-mobile personal computer (Ultra-Mobile Personal Computer, UMPC), a netbook or a personal digital assistant (Personal Digital Assistant, PDA), etc., and can also be a server, a network attached storage (Network Attached Storage, NAS), a personal computer (Personal Computer, PC), a television (Television, TV), a teller machine or a self-service machine, etc., which is not specifically limited in the embodiment of the present application.

[0171] The test question correction device 500 provided in the embodiment of the present application can achieve Figure 1 To avoid repetition, the various processes implemented in the embodiment of the test question correction method are not described here.

[0172] The present application also provides a computer device, such as Figure 6As shown, the computer device 600 includes a processor 601 and a memory 602. The memory 602 stores programs or instructions that can be executed on the processor 601. When the program or instructions are executed by the processor 601, the various steps of the above-mentioned test question correction method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, they are not described here.

[0173] The memory 602 can be used to store software programs and various data. The memory 602 may mainly include a first storage area for storing programs or instructions and a second storage area for storing data, wherein the first storage area may store an operating system, an application program or instructions required for at least one function (such as a sound playback function, an image playback function, etc.), etc. In addition, the memory 602 may include a volatile memory or a non-volatile memory, or the memory 602 may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), a static random access memory (SRAM), a dynamic random access memory (DRAM), a synchronous dynamic random access memory (SDRAM), a double data rate synchronous dynamic random access memory (DDRSDRAM), an enhanced synchronous dynamic random access memory (ESDRAM), a synchronous link dynamic random access memory (SLDRAM) and a direct memory bus random access memory (DRRAM). The memory 602 in the embodiment of the present application includes but is not limited to these and any other suitable types of memory.

[0174] The processor 601 may include one or more processing units; optionally, the processor 601 integrates an application processor and a modem processor, wherein the application processor mainly processes operations related to an operating system, a user interface, and application programs, and the modem processor mainly processes wireless communication signals, such as a baseband processor. It is understandable that the modem processor may not be integrated into the processor 601.

[0175] The embodiment of the present application also provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, each process of the above-mentioned test question correction method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0176] It should be noted that, in this article, the terms "comprise", "include" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise one..." do not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0177] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

Claims

1. A method for correcting test questions, characterized in that: include: Get the test information of the test questions; Classifying the test questions based on the test question information to determine the test question type, wherein the test question type includes at least one of a semantically similar question, an open question, and an objective question; The test questions are corrected according to the correction mode corresponding to the question type of the test questions.

2. The method according to claim 1, characterized in that The step of correcting the test questions according to the correction mode corresponding to the question type of the test questions comprises: If the test question is a semantically similar question, then correcting the test question according to the semantically similar question correction mode; If the test question is an open question, the test question is graded according to the open question grading mode; If the test question is an objective question, the test question is corrected according to the objective question correction mode.

3. The method according to claim 2, characterized in that If the test question is a semantically similar question, the test question is corrected according to the semantically similar question correction mode, including: Segmenting the user answer and the standard answer of the test question respectively to obtain a first keyword of the user answer and a second keyword of the standard answer; Traversing the first keywords of the user's answers, for any first keyword, performing a semantic similarity comparison with each second keyword of the standard answer to obtain a semantic similarity, and determining whether the first keyword is a valid answer according to the semantic similarity; Counting the number of first keywords that are valid answers, and calculating the accuracy of the user's answer based on the number and the total number of the second keywords; If the answer accuracy of the user's answer is greater than or equal to a second preset threshold, the user's answer is determined to be correct.

4. The method according to claim 3, characterized in that The method of determining whether the first keyword is a valid answer according to the semantic similarity includes: Determining a target semantic similarity based on the semantic similarity between the first keyword and each of the second keywords; If the target semantic similarity is less than a first preset threshold, determining that the first keyword answer is invalid; If the target semantic similarity is greater than or equal to a first preset threshold, the first keyword is determined to be a valid answer.

5. The method according to claim 3, characterized in that: The step of comparing the semantic similarity of any first keyword with each second keyword of the standard answer to obtain the semantic similarity includes: Determine whether the first keyword exists in the key value of the synonym dictionary, wherein the key value of the synonym dictionary is the keyword, and the value of the synonym dictionary is the synonym list corresponding to the keyword; If the first keyword exists in the key value of the synonym dictionary, then searching whether the second keyword exists in the synonym list corresponding to the first keyword; if the second keyword exists, then forming a synonym pair with the first keyword; The semantic similarity of the keyword pair is searched in a similarity dictionary, wherein the key value of the similarity dictionary is the synonym pair, and the value value of the synonym dictionary is the semantic similarity corresponding to the synonym pair.

6. The method according to claim 5, characterized in that The step of comparing the semantic similarity of any first keyword with each second keyword of the standard answer to obtain the semantic similarity further includes: If the first keyword does not exist in the key value of the synonym dictionary, or the semantic similarity of the keyword pair is not found in the similarity dictionary, the first keyword and the second keyword are input into a trained synonym recognition model to output the semantic similarity.

7. The method according to claim 6, characterized in that The method further comprises: When the semantic similarity output by the synonym recognition model is greater than or equal to a third preset threshold, outputting the first keyword and the second keyword to a first manual proofreading list for manual proofreading; After the first manual proofreading information is obtained, the first keyword and the second keyword are entered into the similarity dictionary, or the synonym recognition model is adjusted according to the first keyword and the second keyword.

8. The method according to claim 2, characterized in that: If the test question is an open question, the test question is corrected according to the open question correction mode, including: Classifying the test question into sub-question types based on the test question information to determine the sub-question type of the test question, wherein the sub-question type of the test question includes at least one of a word formation question, a sentence formation question, and other open questions; The test question is corrected according to the correction mode corresponding to the sub-question type of the test question.

9. The method according to claim 8, characterized in that If the test question is a word combination question, the test question is corrected according to the correction mode corresponding to the sub-question type of the test question, including: Determining whether the user's answer to the test question contains a character in the question stem that needs to be formed into a word; If not, it is determined that the user's answer is wrong; If it is included, determine whether the user answer exists in the word list dictionary, the key value of the word list dictionary is the word, and the value value is the true or false value of the word; if the user answer exists, determine whether the user answer is correct based on the true or false value corresponding to the user answer; if the user answer does not exist, use the large language binary classification model to determine whether the user answer is correct.

10. The method according to claim 8, characterized in that If the test question is a sentence-making question, the test question is corrected according to the correction mode corresponding to the sub-question type of the test question, including: Determining whether the user's answer to the test question contains the words required to form a sentence in the question stem of the test question; If not, it is determined that the user's answer is wrong; If included, the sentence coherence small model is used to determine whether the user's answer is fluent; if the user's answer is fluent, it is determined that the user's answer is correct; if the user's answer is not fluent, the large language binary classification model is used to determine whether the user's answer is correct.

11. The method according to claim 8, characterized in that If the test question is other open questions, the test question is corrected according to the correction mode corresponding to the sub-question type of the test question, including: Using the sentence coherence model to determine whether the user's answer is sentence coherent; if the user's answer is sentence coherent, then the user's answer is correct; if the user's answer is not sentence coherent, then a preset percentage of the score of the test question is deducted; A semantic model is used to determine whether the user answer is consistent with the question; if the user answer is consistent with the question, the user answer is determined to be correct; if the user answer is not consistent with the question, a large language binary classification model is used to determine whether the user answer is correct.

12. The method according to any one of claims 9 to 11, characterized in that The method further comprises: Train the large language binary classification model; wherein the large language binary classification model includes a large language model, a feature comparison model and a binary classification output layer, the large language model is used to determine whether the user answer meets the requirements of the question, the feature comparison model is used to compare the user answer with the standard answer, and the binary classification output layer is used to output the correct value of the user answer.

13. The method according to any one of claims 9 to 11, characterized in that The method further comprises: When the large language binary classification model is used to determine that the user answer is correct, outputting the user answer and the correct value to a second manual proofreading list for manual proofreading; After obtaining the second manual proofreading information, the user answer and the correct value are entered into a word list dictionary, or the large language binary classification model is adjusted according to the user answer and the correct value.

14. The method according to claim 2, characterized in that If the test question is an objective question, the test question is corrected according to the objective question correction mode, including: If the test question is an objective question, converting the user answer and the standard answer of the test question into a character string; The character string of the user's answer is compared with the character string of the standard answer to correct the test question.

15. The method according to claim 1, characterized in that If the test questions include objective questions and / or semantically similar questions, the method further includes: After the test question is corrected according to the correction mode corresponding to the question type, if the correction result is that the user answer to the test question is wrong, the test question is corrected for the second time; wherein, the method of correcting the test question for the second time includes: using a feature comparison model of a large language binary classification model to correct the test question for the second time.

16. A test question grading device, characterized in that: include: An information acquisition module, used to acquire test information of test questions; A question type classification module, used to classify the question type of the question based on the question information, and determine the question type of the question, wherein the question type of the question includes at least one of a semantically similar question, an open question, and an objective question; The marking module is used to mark the test questions according to the marking mode corresponding to the question type of the test questions.

17. A computer device, characterized in that: The method comprises a processor and a memory, wherein the memory stores a program or instruction running on the processor, and when the program or instruction is executed by the processor, the steps of the test question marking method as described in any one of claims 1 to 15 are implemented.

18. A readable storage medium having a program or instruction stored thereon, characterized in that: When the program or instruction is executed by a processor, the steps of the test question marking method according to any one of claims 1 to 15 are implemented.

Images