Blank filling question correcting method, device and equipment, medium and program product

Abstract

The invention provides a gap filling question correcting method and device, equipment, a medium and a program product, and the method comprises the steps: obtaining a to-be-corrected gap filling question which comprises a question of the to-be-corrected gap filling question and a to-be-corrected answer; distributing question type labels for the questions by using a pre-trained large language model; the question type label comprises at least one of a replaceable sequence label, a unique answer label, a candidate answer set label, a keyword set label and a taboo word set label; determining a correction strategy of the to-be-corrected answer according to the question type label corresponding to the question; and correcting the answer to be corrected by using the determined correction strategy. According to the method and the device, the proper correction strategy can be automatically selected according to the question type characteristics of the blank-filling questions, so that complex and diversified blank-filling question correction requirements are effectively met, and the accuracy and the flexibility of blank-filling question correction are improved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a fill-in-the-blank question grading method, device, equipment, medium and program product. Background Art

[0002] In the field of education, there is a growing demand for automatic grading of fill-in-the-blank questions in order to improve grading efficiency and reduce teachers' workload. However, the current grading methods for fill-in-the-blank questions mainly rely on fixed templates and predefined standard answers for judgment. This method is only suitable for simple fill-in-the-blank questions with unique answers. When faced with complex fill-in-the-blank questions with diverse answer expressions, it is difficult to provide an effective processing strategy, resulting in an inability to accurately determine whether the student's answer is correct, thus affecting the final grading effect. Summary of the invention

[0003] In order to overcome the problems existing in the related art, the present application provides a fill-in-the-blank question grading method, device, equipment, medium and program product.

[0004] According to a first aspect of an embodiment of the present application, a fill-in-the-blank question marking method is provided, the method comprising:

[0005] Obtaining a fill-in-the-blank question to be corrected, wherein the fill-in-the-blank question to be corrected includes the title of the fill-in-the-blank question to be corrected and the answer to be corrected;

[0006] Assigning a question type label to the question using a pre-trained large language model; the question type label includes at least one of a permutable label, a unique answer label, a candidate answer set label, a keyword set label, and a taboo word set label;

[0007] Determining a marking strategy for the answer to be marked according to the question type label corresponding to the question;

[0008] Correcting the answers to be corrected using the determined correction strategy;

[0009] Among them, the interchangeable label represents that there are multiple fill-in-the-blank items in the question and the order of the reference answers of each fill-in-the-blank item can be interchanged, the unique answer label represents that the reference answer to the question is unique, the candidate answer set label represents that the reference answer to the question has multiple expressions, the keyword set label represents that the reference answer to the question contains all the keywords in the preset keyword set, and the taboo word set label represents that the reference answer to the question does not contain any taboo words in the preset taboo word set.

[0010] According to a second aspect of an embodiment of the present application, a fill-in-the-blank question correction device is provided, the device comprising:

[0011] A fill-in-the-blank question acquisition module to be corrected is used to obtain the fill-in-the-blank question to be corrected, wherein the fill-in-the-blank question to be corrected includes the title of the fill-in-the-blank question to be corrected and the answer to be corrected;

[0012] A question type label assignment module, used to assign question type labels to the questions using a pre-trained large language model; the question type labels include at least one of a permutable label, a unique answer label, a candidate answer set label, a keyword set label, and a taboo word set label;

[0013] A marking strategy determination module, used to determine the marking strategy of the answer to be marked according to the question type label corresponding to the question;

[0014] A correction module, used for correcting the answers to be corrected by using the determined correction strategy;

[0015] Among them, the interchangeable label represents that there are multiple fill-in-the-blank items in the question and the order of the reference answers of each fill-in-the-blank item can be interchanged, the unique answer label represents that the reference answer to the question is unique, the candidate answer set label represents that the reference answer to the question has multiple expressions, the keyword set label represents that the reference answer to the question contains all the keywords in the preset keyword set, and the taboo word set label represents that the reference answer to the question does not contain any taboo words in the preset taboo word set.

[0016] According to a third aspect of an embodiment of the present application, a computer device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the method described in the first aspect when executing the computer program.

[0017] According to a fourth aspect of an embodiment of the present application, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the method described in the first aspect are implemented.

[0018] According to a fifth aspect of an embodiment of the present application, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the method described in the first aspect is implemented.

[0019] The technical solution provided by the embodiments of the present application may have the following beneficial effects:

[0020] In an embodiment of the present application, by utilizing a pre-trained large language model to assign question type labels to the fill-in-the-blank questions to be corrected, it is possible to automatically select appropriate correction strategies based on the question type characteristics of the fill-in-the-blank questions, thereby effectively adapting to the complex and diverse fill-in-the-blank question correction needs and improving the accuracy and flexibility of fill-in-the-blank question correction.

[0021] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The accompanying drawings, which are incorporated in the specification and constitute a part of this application, illustrate embodiments consistent with the application and, together with the description, serve to explain the principles of the application.

[0023] Figure 1 It is a flow chart of a fill-in-the-blank question correction method shown in the present application according to an exemplary embodiment.

[0024] Figure 2 It is a schematic diagram of a bipartite graph matching situation shown in the present application according to an exemplary embodiment.

[0025] Figure 3 It is a schematic diagram of another bipartite graph matching situation shown in the present application according to an exemplary embodiment.

[0026] Figure 4 It is a hardware structure diagram of a computer device according to an exemplary embodiment of the present application.

[0027] Figure 5 It is a structural block diagram of a fill-in-the-blank question correction device shown in the present application according to an exemplary embodiment. DETAILED DESCRIPTION

[0028] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0029] The terms used in this application are for the purpose of describing specific embodiments only and are not intended to limit this application. The singular forms of "a", "said" and "the" used in this application and the appended claims are also intended to include plural forms unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0030] It should be understood that although the terms first, second, third, etc. may be used in the present application to describe various information, these information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of the present application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0031] The current fill-in-the-blank question grading methods lack the ability to adapt to the diversity and complexity of questions, making it difficult to provide effective processing strategies when faced with fill-in-the-blank questions with complex question types and diverse answer expressions, resulting in an inability to accurately determine whether students' answers are correct, thus affecting the final grading effect. Specifically, when the answers to fill-in-the-blank questions can have multiple reasonable expressions, the grading methods with fixed templates and single standard answers are difficult to accurately capture all correct answer forms. For example, for fill-in-the-blank questions where the order of answers can be adjusted or involve multiple concept combinations, students may express their answers in different but correct ways, and the current fill-in-the-blank question grading methods may mistakenly judge the answers as wrong because they cannot recognize these variations.

[0032] In view of the above problems, the present application provides an improved fill-in-the-blank question grading method. The method uses a pre-trained large language model to assign question type labels to fill-in-the-blank questions to be graded, so that it can automatically select appropriate grading strategies according to the question type characteristics of the fill-in-the-blank questions, thereby effectively adapting to the complex and diverse fill-in-the-blank question grading needs and improving the accuracy and flexibility of fill-in-the-blank question grading.

[0033] The embodiments of the present application are described in detail below in conjunction with the accompanying drawings.

[0034] Figure 1 FIG. 1 is a flow chart of a fill-in-the-blank question correction method according to an exemplary embodiment of the present application. Figure 1 As shown, the method comprises the following steps:

[0035] S101, obtaining fill-in-the-blank questions to be corrected, where the fill-in-the-blank questions to be corrected include the title of the fill-in-the-blank questions to be corrected and the answers to be corrected;

[0036] S102, using a pre-trained large language model to assign a question type label to the question; the question type label includes at least one of a permutable label, a unique answer label, a candidate answer set label, a keyword set label, and a taboo word set label;

[0037] S103, determining a marking strategy for the answer to be marked according to the question type label corresponding to the question;

[0038] S104: Correct the answers to be corrected using the determined correction strategy.

[0039] In step S101, the fill-in-the-blank questions to be corrected are fill-in-the-blank questions that need to be corrected, and the specific content includes the title of the fill-in-the-blank question and the answers to be corrected filled in by the students. The fill-in-the-blank questions to be corrected can be in text format, image format or any other format that can be recognized and processed.

[0040] In step S102, a large number of fill-in-the-blank questions and their reference answers can be collected in advance as training data, and the large language model can be trained based on the deep learning algorithm so that it can accurately identify the key features of different questions and assign question type labels accordingly. During the training process, some data can also be manually annotated to prompt fine-tuning of the large language model and improve the accuracy of question type label assignment.

[0041] Specifically, the large language model can assign one or more question type labels to the question according to its characteristics, including at least one of a permutable label, a unique answer label, a candidate answer set label, a keyword set label, and a taboo word set label. Among them, the permutable label indicates that the question has multiple fill-in-the-blank items and the order of the reference answers of each fill-in-the-blank item can be interchanged, the unique answer label indicates that the reference answer of the question is unique, the candidate answer set label indicates that the reference answer of the question has multiple expressions, the keyword set label indicates that the reference answer of the question contains all the keywords in the preset keyword set, and the taboo word set label indicates that the reference answer of the question does not contain any taboo word in the preset taboo word set.

[0042] In step S103, corresponding grading strategies can be designed in advance for questions with each question type label, and a mapping relationship between question type labels and grading strategies can be constructed. This allows the user to quickly select appropriate grading strategies based on the mapping relationship after determining the question type label of the question, thereby ensuring a high degree of match between the grading strategies and question features, and improving the efficiency and accuracy of grading fill-in-the-blank questions.

[0043] In step S104, after the marking strategy is determined, the student's answer to be marked can be compared with the reference answer of the fill-in-the-blank question based on the marking rules in the marking strategy, so as to evaluate the accuracy of the student's answer.

[0044] The fill-in-the-blank question grading method provided in the embodiment of the present application uses a pre-trained large language model to assign question type labels to the fill-in-the-blank questions to be graded, so that it can automatically select appropriate grading strategies according to the question type characteristics of the fill-in-the-blank questions, thereby effectively adapting to the complex and diverse fill-in-the-blank question grading needs and improving the accuracy and flexibility of fill-in-the-blank question grading.

[0045] Next, the specific correction process of the correction strategies corresponding to the above-mentioned question type labels will be explained in detail.

[0046] In some embodiments, when the question type label corresponding to the question includes a permutable label, the determined grading strategy will at least include the grading strategy corresponding to the permutable label. The grading strategy is used to grading the answer to be graded, specifically including: constructing a bipartite graph, bipartite Figure 1 The nodes on one side of the bipartite graph are the answers to be corrected for each fill-in-the-blank item, and the nodes on the other side of the bipartite graph are the reference answers for each fill-in-the-blank item; in the bipartite graph, the maximum number of matches between the answers to be corrected and the reference answers is determined; based on the maximum number of matches, the correction results of the answers to be corrected are determined.

[0047] In this embodiment, the reason for selecting this marking strategy is that, for questions with interchangeable labels, there are multiple fill-in-the-blank items in the question and the order of the reference answers of each fill-in-the-blank item can be interchanged, which means that when marking, the correctness cannot be determined solely by the order of the answers, but the matching degree of the answer content itself should be focused on. Therefore, by constructing a bipartite graph and determining the maximum matching number of the answer to be corrected and the reference answer therein, it can be used as an effective marking strategy in this case. This method can compare the answer to be corrected filled in by the student with the reference answer of the question, and represent the matching relationship between the answers by connecting the edges, so as to find the best match between the answer to be corrected and the reference answer without considering the order. In order to determine the maximum matching number, the Hungarian algorithm can also be used, which can efficiently find the maximum matching in the bipartite graph.

[0048] In the specific implementation process, we can first construct a bipartite graph G = (U, V, E), where U is the set A consisting of the answers to be corrected for each fill-in-the-blank item. user , (A user ={a1,a2,…,a n}), V is the set A consisting of the reference answers of each fill-in-the-blank item answer , (A answer ={b1,b2,...,b n}). Then, for each answer a to be corrected i ∈A user and each reference answer b j ∈A answer , if they meet the matching condition, then add an edge from a to the edge set E i to b j The edge ij . Execute the Hungarian maximum matching algorithm to find the maximum matching on the bipartite graph, which means matching the answers to be corrected and the reference answers as much as possible, ensuring that each answer to be corrected and the reference answer appear at most once.

[0049] After determining the maximum number of matches, you can also set the following two methods to determine the grading results according to actual needs. One is to determine the grading results based on the relationship between the maximum number of matches and the number of blank items in the question, and the other is to determine the grading results directly based on the value of the maximum number of matches.

[0050] Specifically, in some embodiments, the grading result is determined based on the relationship between the maximum matching number and the number of fill-in-the-blank items in the question. When the maximum matching number is consistent with the number of fill-in-the-blank items in the question, the answer to be graded can be graded as correct; when the maximum matching number is inconsistent with the number of fill-in-the-blank items in the question, the answer to be graded can be graded as incorrect.

[0051] For example, if a question has 5 fill-in-the-blank items and the maximum number of matches is also 5, then the answer to be corrected can be determined to be correct; if the maximum number of matches is 3, which is inconsistent with the number of fill-in-the-blank items of 5, it means that the answers to 2 fill-in-the-blank items are incorrect, then the answer to be corrected can be directly determined to be wrong.

[0052] Alternatively, in other embodiments, the correction result is determined based on the value of the maximum matching number. When the maximum matching number is 0, the answer to be corrected can be corrected to the fill-in-the-blank item with the maximum matching number when the maximum matching number is not 0.

[0053] For example, if a question has 5 fill-in-the-blank items and the maximum number of matches is 0, that is, no answer to be corrected matches the reference answer, then the answer to be corrected will be corrected to wrong. If the maximum number of matches is 3, then it means that the answers to 3 fill-in-the-blank items are correct. In this case, although there are still 2 fill-in-the-blank items with wrong answers in the answer to be corrected, you can not directly correct the fill-in-the-blank question as wrong, but change the answer to be corrected to 3 fill-in-the-blank items with correct answers. This method can flexibly determine the correctness of the answer based on the number of matches, so as to flexibly deal with the correction of partially correct answers.

[0054] Let’s take a specific example to illustrate the above correction process:

[0055] Suppose a question has 5 fill-in-the-blank items, and the reference answers are {A, B, C, D, E}, and the answers to be corrected by the students are {B, A, D, C, E}. By constructing a bipartite graph and applying the Hungarian algorithm, we can find that the maximum number of matches is 5, that is, the student's answer completely matches the reference answer, but the order is different. The matching of the bipartite graph is as follows: Figure 2 Therefore, according to the first and second methods of determining the correction results, the answer to be corrected can be corrected to be completely correct.

[0056] If the student's answer to be corrected is {B,A,X,Y,E}, the bipartite graph matching is as follows Figure 3As shown in the figure, X and Y are not options in the reference answer, and the maximum number of matches is 3. Then according to the first method of determining the correction result, the answer to be corrected will be directly corrected to wrong, because only the answers to two of the blanks are correct. According to the second method of determining the correction result, since the maximum number of matches is not 0, the answer to be corrected can be corrected to 3 blanks.

[0057] In some embodiments, when the question type label corresponding to the question includes both a keyword set label and a taboo word set label, the determined correction strategy will at least include the correction strategy corresponding to the keyword set label and the correction strategy corresponding to the taboo word set label. Correcting the answer to be corrected using the correction strategy specifically includes: if the answer to be corrected includes all keywords in the preset keyword set and does not include any taboo word in the preset taboo word set, correcting the answer to be corrected to correct; if the answer to be corrected does not include any keyword in the preset keyword set or the answer to be corrected includes any taboo word in the preset keyword set, correcting the answer to be corrected to incorrect.

[0058] In this embodiment, the keyword set label indicates that the reference answer of the question contains all the keywords in the preset keyword set, so in the process of grading, the answer can only be considered correct if the answer to be graded also contains these keywords. Similarly, the taboo word set label indicates that the reference answer of the question does not contain any taboo words in the preset taboo word set, so in the process of grading, the answer can only be considered correct if the answer to be graded does not contain these taboo words. Therefore, when the question type label corresponding to the question includes both the keyword set label and the taboo word set label, the answer to be graded can only be considered correct if it satisfies the conditions of containing all the preset keywords and not containing any preset taboo words.

[0059] In the specific implementation process, the following logical formula can be used for judgment:

[0060]

[0061] Among them, (∧) represents the logical operation of "true for all...". (k i ∈T) represents the keywords in the preset keyword set (k i ) exists in the answer to be graded (T). Represents the taboo words in the preset taboo word set (b j ) does not exist in the answer to be corrected (T). K represents the preset keyword set of the question, and B represents the preset taboo word set of the question. If (Valid(T,K,B)) is true, the answer to be corrected (T) is corrected, otherwise it is corrected.

[0062] Let’s take a specific example to illustrate the above correction process:

[0063] Assume that the question is "Please determine which of the following options is correct", and the reference answer is "Option A". At this time, if the answer to be corrected filled in by the student is "Option A", it will be corrected to correct. However, in some cases, the answer to be corrected filled in by the student may also be "Not Option A". At this time, although the answer to be corrected contains the keyword "Option A", it also contains the taboo word "Not". If only keywords are used for correction, it is easy to correct this answer to correct, but using the correction strategy of this embodiment, it can accurately identify that this answer is wrong and correct it to wrong. It can be seen that the correction strategy in this embodiment not only takes into account the keywords that must be included in the answer, but also takes into account the taboo words that should not be included in the answer, effectively improving the accuracy and reliability of the correction results.

[0064] In some embodiments, when the question type label corresponding to the question only includes a keyword set label, the determined correction strategy will at least include the correction strategy corresponding to the keyword set label. Correcting the answer to be corrected using the correction strategy specifically includes: if the answer to be corrected includes all the keywords in the preset keyword set, correcting the answer to be corrected to be correct; if the answer to be corrected does not include any keyword in the preset keyword set, correcting the answer to be corrected to be wrong.

[0065] This embodiment is similar to the above embodiment, and reference may be made to the above description, which will not be repeated here.

[0066] Let’s take a specific example to illustrate the above correction process:

[0067] Suppose the question requires students to identify specific items in a set of elements, and the reference answer is "A, B, C", where "A", "B", and "C" can be understood as keywords in the reference answer. In this case, as long as the key words "A", "B", and "C" are included in the answer to be corrected by the student, it can be corrected as correct regardless of the form of expression of the answer. For example, the answer to be corrected can be "the answer includes A, B, and C", or "the answer covers A, B, and C", or even "A, B, and C are all mentioned in the answer". Such a correction strategy not only ensures the accuracy of the answer, but also allows students to have a certain degree of flexibility in expression.

[0068] In some embodiments, when the question type label corresponding to the question includes a keyword set label, it may also involve the situation that mathematical terms exist in the answer to be corrected and the reference answer. In this case, the determined correction strategy may at least include the correction strategy that matches the mathematical term content in the correction strategy corresponding to the keyword set label. Correcting the answer to be corrected using the correction strategy specifically includes: determining the synonym set corresponding to the mathematical term in the answer to be corrected in the preset synonym library, if the synonym set contains all the keywords in the preset keyword set, then correcting the answer to be corrected to be correct; if the synonym set does not contain any keyword in the preset keyword set, then correcting the answer to be corrected to be wrong.

[0069] In this embodiment, the keyword set tag indicates that the reference answer of the question contains all the keywords in the preset keyword set, and these keywords may appear in different mathematical noun forms. In the field of mathematics, mathematical nouns may have multiple synonyms, and the keywords of the answers to be corrected filled in by students may be synonyms of the keywords of the reference answer. Therefore, during the correction process, the synonym set of the mathematical nouns in the answers to be corrected can be obtained in advance. As long as the synonym set corresponding to the mathematical nouns in the answers to be corrected contains all the keywords of the reference answer, the corrected answer can be determined to be correct.

[0070] In the specific implementation process, a synonym database (D) can be pre-built based on mathematical knowledge, and the set of mathematical terms (N) in the answers to be corrected can be user ) is mapped into its synonym form, that is, for each mathematical term (n∈N user ), find its corresponding item in the synonym database, if found, use the synonym form to replace the original mathematical noun in the answer to be corrected, otherwise keep the original mathematical noun in the answer to be corrected. The set of mathematical nouns in the answer to be corrected after mapping is represented as (N ′ user), that is, N ′ user=D(n)|n∈Nuser. Determine the set of these mathematical terms (N ′ user) contains all the keywords of the reference answer to the question. If so, the answer to be corrected is corrected, otherwise it is corrected.

[0071] Let’s take a specific example to illustrate the above correction process:

[0072] Suppose the question is about understanding inequalities, and the key word in the reference answer is "not greater than", while the answer to be corrected filled in by the student uses "less than or equal to" to express the same meaning. In mathematics, "not greater than" and "less than or equal to" are synonyms, both of which represent the inequality relationship x≤y. According to the preset synonym library, these two expressions can be mapped to the same mathematical concept. Therefore, even if the student does not use the exact vocabulary in the reference answer, the answer to be corrected can still be corrected because the synonym set contains all the key words of the reference answer. This strategy ensures the flexibility and accuracy of the correction process, and can identify different expressions in the student's answer that are equivalent to the reference answer.

[0073] In some embodiments, when the question type label corresponding to the question includes a unique answer label, the determined correction strategy will at least include the correction strategy corresponding to the unique answer label. Correcting the answer to be corrected using the correction strategy specifically includes: if the answer to be corrected is consistent with the reference answer, correcting the answer to be corrected to correct; if the answer to be corrected is inconsistent with the reference answer, correcting the answer to be corrected to incorrect.

[0074] In this embodiment, the unique answer tag emphasizes the uniqueness of the answer, so during the correction process, the answer to be corrected will be strictly compared with the reference answer. Only when the two are completely consistent, the answer to be corrected will be considered correct.

[0075] In the specific implementation process, an exact matching algorithm can be used to analyze each answer to be corrected and compare it with the reference answer one by one. For example, for a simple math calculation problem, if the reference answer is "7", then only when the student's answer is also "7" will the answer be corrected.

[0076] The grading strategies corresponding to the candidate answer labels can be further divided into the following situations according to the specific contents involved in the answers to be graded and the reference answers.

[0077] In some embodiments, when the answer to be corrected and the reference answer can be converted into a fraction form or a false fraction form, and the question type label corresponding to the question includes a candidate answer set label, the determined correction strategy will at least include the correction strategy corresponding to the candidate answer set label that matches the fraction and false fraction content. Correcting the answer to be corrected using the correction strategy specifically includes: converting both the answer to be corrected and the reference answer into a fraction or a false fraction, if the converted answer to be corrected is consistent with the false fraction corresponding to the converted reference answer, then correcting the answer to be corrected to correct, if the converted answer to be corrected is inconsistent with the false fraction corresponding to the converted reference answer, then correcting the answer to be corrected to incorrect.

[0078] In this embodiment, the marking strategy is mainly applicable to questions where the answers involve multiple forms of fraction expressions. By uniformly converting all possible answer forms into a standard fraction or improper fraction form, it is possible to accurately compare and judge the correctness of the answers, avoiding being affected by different expression forms.

[0079] In the specific implementation process, a numerical conversion algorithm can be adopted to convert the answer to be marked and the reference answer into a unified fraction or improper fraction form. For example, if the answer to be marked or the reference answer is in the form of a mixed fraction "23 1 ", it can be converted into the improper fraction form "7 / 3". If the answer to be marked or the reference answer is in the decimal form "0.75", it can be converted into the fraction form "3 / 4". Alternatively, regular expressions and mapping tables can also be used to convert the fraction expressed in words into a unified fraction or improper fraction form. For example, converting "nine tenths" into "9 / 10". By comparing whether the converted answer to be marked is consistent with the reference answer, the correctness of the answer to be marked is determined. In this way, various different forms of fraction answers can be effectively processed, and the accuracy of marking can be effectively improved.

[0080] In some embodiments, when there are numbers in the answer to be marked and the reference answer, and the question type label corresponding to the question includes a candidate answer set label, the determined marking strategy at least includes the marking strategy that matches the number content in the marking strategy corresponding to the candidate answer set label. Using this marking strategy to mark the answer to be marked specifically includes: converting the numbers in the answer to be marked and the reference answer into the reference number format. If the converted answer to be marked is consistent with the converted reference answer, the answer to be marked is marked as correct; if the converted answer to be marked is inconsistent with the converted reference answer, the answer to be marked is marked as wrong.

[0081] In this embodiment, the marking strategy is mainly applicable to questions where the answers involve numbers. The same number may呈现 different formats due to different habits of the writers. For example, the number 1 can be written in multiple different formats such as "one, 1, (1), (1), ①", etc. By uniformly converting all numbers into a standard reference number format, it can be ensured that the marking process is not affected by different number expression forms, thereby accurately evaluating the correctness of the answers.

[0082] In the specific implementation process, a number standardization algorithm can be adopted to convert the numbers in the answer to be marked and the reference answer into a unified reference number format. For example, if the number in the answer to be marked is "(1)" and the number in the reference answer is "①", then through the number standardization algorithm, these two numbers can be converted into the same format, such as the Arabic numeral "1". Such a conversion ensures the consistency of the numbers, making the comparison process more direct and accurate.

[0083] In some embodiments, when there are mathematical formulas in the answer to be corrected and the reference answer, and the question type label corresponding to the question includes a candidate answer set label, the determined correction strategy includes at least the correction strategy that matches the mathematical formula content in the correction strategy corresponding to the candidate answer set label. Correcting the answer to be corrected using the correction strategy specifically includes: converting the mathematical formula in the answer to be corrected and the mathematical formula in the reference answer into a reference formula format, and if the converted answer to be corrected is consistent with the converted reference answer, correcting the answer to be corrected to correct, and if the converted answer to be corrected is inconsistent with the converted reference answer, correcting the answer to be corrected to incorrect.

[0084] In this embodiment, the grading strategy is mainly applicable to questions whose answers involve mathematical formulas. Since mathematical formulas may express the same or equivalent mathematical relationships in different forms, by converting all formulas into a standard reference formula format, it can be ensured that the grading process is not affected by different formula expressions, thereby accurately evaluating the correctness of the answers. It can be understood that the mathematical formula conversion involved in this embodiment is not limited to basic arithmetic operations of addition, subtraction, multiplication and division, but may also involve advanced mathematical operations such as trigonometric functions, power functions, and limits.

[0085] During the specific implementation process, a mathematical formula normalization algorithm or a preset mathematical formula library can be used to convert the mathematical formulas in the answers to be corrected and the reference answers into a unified reference formula format. For example, if the mathematical formula in the answer to be corrected is "sin(2x)", and the mathematical formula in the reference answer is "2sin(x)cos(x)", then through the mathematical formula normalization algorithm, it can be recognized that these two formulas are actually equivalent, and both are converted to "2sin(x)cos(x)". For another example, if the mathematical formula in the answer to be corrected is "e (xlna) +y=8", and the mathematical formula in the reference answer is "a x +y-8=0”, can be converted into “a x +y-8=0", and then recognize that these two formulas are actually equivalent.

[0086] Of course, it should be noted that in some special cases, the large language model may assign multiple question type labels to the same fill-in-the-blank question. In this case, the above-mentioned multiple grading strategies can be used to comprehensively grade the answers to be graded based on the specific question type label assignments, so as to comprehensively evaluate the fill-in-the-blank questions and improve the accuracy and reliability of the grading. This application does not specifically limit the specific combination of grading strategies to flexibly apply to a wide range of fill-in-the-blank question grading needs.

[0087] In order to further improve the accuracy of fill-in-the-blank question grading, in addition to using question type labels to determine grading strategies, pre-processing and optimization can also be performed on the obtained fill-in-the-blank questions before grading.

[0088] In some embodiments, obtaining a fill-in-the-blank question to be corrected may include the following steps: obtaining an image of the fill-in-the-blank question to be corrected, performing OCR recognition on the image, and obtaining an OCR recognition result; screening out preset characters from the OCR recognition result, where the preset characters are a number of characters with a high probability of recognition error in pre-stated OCR recognition; calculating the similarity between the preset characters and reference characters, replacing the preset characters whose similarity is higher than a preset threshold with the reference characters, and using the replaced OCR recognition result as the obtained fill-in-the-blank question to be corrected.

[0089] In this embodiment, the image of the fill-in-the-blank question to be corrected can be obtained by taking a photo or scanning the content of the fill-in-the-blank question. The preset characters can collect the OCR recognition results of a large number of fill-in-the-blank questions in advance, and the characters with a higher probability of recognition errors in these recognition results are counted as preset characters. For example, in the OCR recognition process, the word "two" is often mistakenly recognized as the symbol "=", and the symbol "+" is mistakenly recognized as the word "ten", etc. By calculating the similarity between the preset character and the reference character, and replacing the preset character with the reference character when the similarity is higher than the preset threshold, the threshold of character replacement can be effectively raised, so that not only errors that may occur in the OCR recognition process can be corrected, but also the characters that students actually wrote incorrectly can be effectively avoided from being replaced by mistake, thereby ensuring the fairness and accuracy of the correction process.

[0090] In the specific implementation process, the similarity calculation can be performed using various methods such as cosine similarity and Jaccard similarity according to the representation of character features. For example, cosine similarity is usually used to calculate the similarity of vectorized character features, and its calculation formula is:

[0091]

[0092] Wherein, x represents the reference character, y represents the currently recognized character, sim(x,y) represents the similarity value of x and y. If sim(x,y)>threshold is calculated, it can be considered that the character similarity between x and y is higher than the preset threshold, and the currently recognized character y can be replaced by the reference character x.

[0093] This pre-processing optimization can significantly improve the accuracy of OCR recognition results, providing more reliable input data for subsequent grading. This method is especially suitable for math fill-in-the-blank questions, because the recognition errors of mathematical symbols and numbers may lead to errors in the grading results of the entire question. By pre-correcting these characters with high error probability, the grading errors caused by OCR recognition errors can be greatly reduced, ensuring the accuracy and fairness of the grading results.

[0094] In addition, we also take into account that in the actual grading process, inaccurate grading results may occur due to incorrect assignment of language model question type labels, ambiguity in the questions themselves, or inapplicability of grading strategies.

[0095] In order to correct grading errors in a timely manner and improve the accuracy of grading results, in some embodiments, after the answers to be grading are corrected according to the grading strategy determined in the above embodiments, the following steps may also be included: obtaining the grading results, and screening out target fill-in-the-blank questions from the fill-in-the-blank questions to be corrected according to the grading results, the target fill-in-the-blank questions being fill-in-the-blank questions that are marked as errors more frequently than a preset frequency; re-grading the target fill-in-the-blank questions using a pre-trained large language model to obtain re-grading results; and updating the question type label corresponding to the target fill-in-the-blank question based on the re-grading results.

[0096] In this embodiment, by analyzing the correction results, those fill-in-the-blank questions that are frequently incorrectly corrected can be identified, which may indicate that the previous correction strategies for these fill-in-the-blank questions are wrong or inapplicable. In the aforementioned embodiment, in order to ensure the response speed, the large language model is only used to assign question type labels and does not directly participate in the correction process. In this embodiment, in order to provide more accurate correction results with the help of the large language model's understanding and analysis capabilities of natural language, the large language model can be directly used to participate in the correction, so as to verify the correction situation by combining the title of the fill-in-the-blank question to be corrected and the answer to be corrected through the given prompt words. The prompt words here refer to a series of instructions or questions used to guide the large language model to understand and process specific tasks. In the context of correcting fill-in-the-blank questions, the prompt words can be specific instructions on how to evaluate the answers to be corrected, for example, "Judge whether the following answers to be corrected are correct: the title is..., the answer to be corrected is..., and the reference answer is..." Through such prompt words, the large language model can focus on the evaluation task and use its advanced language processing capabilities to conduct in-depth analysis of the answers to the fill-in-the-blank questions.

[0097] For example, we can use the judgment function F1(P1,Q,A u ,A r ,LLM)∈[0,1], to determine whether the answer to be corrected is correct, where (P) represents the prompt word, (Q) represents the question, (A u ) indicates the answer to be corrected, (A r ) represents the reference answer, and LLM represents the large language model. The output value of this function can be expressed as the probability that the answer to be corrected is correct. Based on this probability value, the correctness of the answer to be corrected can be evaluated more accurately.

[0098] Based on the re-marking results, the question type labels corresponding to the target fill-in-the-blank questions are updated, which can be specifically divided into the following situations:

[0099] If the re-marking result shows that the answer to the target fill-in-the-blank question is correct, and the target fill-in-the-blank question has multiple fill-in-the-blank items, a reversible label can be assigned to the target fill-in-the-blank question;

[0100] Alternatively, if the re-marking results show that the answer to the target fill-in-the-blank question is correct, and the answer to the target fill-in-the-blank question is a number or a mathematical formula, the answer to the target fill-in-the-blank question can be used as a reference answer to the target fill-in-the-blank question;

[0101] Alternatively, if the re-marking result shows that the answer to the target fill-in-the-blank question is correct, a preset keyword generation function may be used to generate keywords for the answer to the target fill-in-the-blank question, and the generated keywords may be added to the preset keyword set.

[0102] For example, a keyword generation technology based on a large language model can be used to generate keywords through the keyword generation function F2 (P2, Q, A u ,LLM), generate a keyword set, where (P) represents the prompt word, (Q) represents the title, (A u ) indicates answers to be corrected, and LLM indicates large model.

[0103] This mechanism of automatically correcting errors based on grading results can identify and correct errors in a timely manner, adjust and optimize grading strategies according to actual grading situations, reduce manual intervention, and ensure the reliability and effectiveness of grading results.

[0104] Corresponding to the embodiment of the aforementioned correction method, the present application also provides an embodiment of a fill-in-the-blank question correction device and a terminal used therein.

[0105] The embodiments of the apparatus for correcting fill-in-the-blank questions in this application can be applied to computer devices, such as servers or terminal devices. The apparatus embodiments can be implemented by software, or by hardware or a combination of software and hardware. Taking software implementation as an example, as a logically sensed apparatus, the apparatus is formed by the processor in which it is located reading the corresponding computer program instructions in the non-volatile memory into the memory and running them. From the hardware level, if Figure 4 As shown in FIG. 1 , a hardware structure diagram of the computer device in which the fill-in-the-blank question correction device of the embodiment of the present application is located is shown in FIG. Figure 4 In addition to the processor 401, memory 402, network interface 403, and non-volatile memory 404 shown, the server or electronic device where the fill-in-the-blank question grading apparatus is located in the embodiment may also include other hardware, usually according to the actual function of the computer device, which will not be described in detail.

[0106] Figure 5 1 is a structural block diagram of a fill-in-the-blank question correction device according to an exemplary embodiment of the present application. Figure 5As shown, the abnormality detection device 500 includes:

[0107] The fill-in-the-blank question acquisition module 501 is used to acquire the fill-in-the-blank question to be corrected, which includes the title of the fill-in-the-blank question to be corrected and the answer to be corrected;

[0108] The question type label assignment module 502 is used to assign question type labels to questions using a pre-trained large language model; the question type label includes at least one of a permutable label, a unique answer label, a candidate answer set label, a keyword set label, and a taboo word set label;

[0109] The marking strategy determination module 503 is used to determine the marking strategy of the answer to be marked according to the question type label corresponding to the question;

[0110] A correction module 504, used for correcting the answers to be corrected using the determined correction strategy;

[0111] Among them, the interchangeable label indicates that there are multiple fill-in-the-blank items in the question and the order of the reference answers of each fill-in-the-blank item can be interchanged. The unique answer label indicates that the reference answer of the question is unique. The candidate answer set label indicates that the reference answer of the question has multiple expressions. The keyword set label indicates that the reference answer of the question contains all the keywords in the preset keyword set. The taboo word set label indicates that the reference answer of the question does not contain any taboo words in the preset taboo word set.

[0112] Correspondingly, the present application also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of correcting fill-in-the-blank questions recorded in any of the above embodiments are implemented.

[0113] Correspondingly, the present application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of correcting fill-in-the-blank questions recorded in any of the above embodiments.

[0114] Correspondingly, the present application also provides a computer program product, including a computer program, which, when executed by a processor, implements the steps of correcting fill-in-the-blank questions recorded in any of the above embodiments.

[0115] The implementation process of the functions and effects of each module in the above-mentioned device is specifically described in the implementation process of the corresponding steps in the above-mentioned method, which will not be repeated here.

[0116] For the device embodiment, since it basically corresponds to the method embodiment, the relevant parts can refer to the partial description of the method embodiment. The device embodiment described above is only schematic, wherein the modules described as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical modules, that is, they may be located in one place, or they may be distributed on multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present application scheme. A person of ordinary skill in the art can understand and implement it without paying creative labor.

[0117] The above describes specific embodiments of the present application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0118] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the inventions claimed herein. The present application is intended to cover any variations, uses or adaptations of the present application, which follow the general principles of the present application and include common knowledge or customary techniques in the art that are not claimed in the present application. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present application are indicated by the following claims.

[0119] It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

[0120] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application shall be included in the scope of protection of the present application.

Claims

1. A method for correcting fill-in-the-blank questions, characterized in that: include: Obtaining a fill-in-the-blank question to be corrected, wherein the fill-in-the-blank question to be corrected includes the title of the fill-in-the-blank question to be corrected and the answer to be corrected; Assigning a question type label to the question using a pre-trained large language model; the question type label includes at least one of a permutable label, a unique answer label, a candidate answer set label, a keyword set label, and a taboo word set label; Determining a marking strategy for the answer to be marked according to the question type label corresponding to the question; Correcting the answers to be corrected using the determined correction strategy; Among them, the interchangeable label represents that there are multiple fill-in-the-blank items in the question and the order of the reference answers of each fill-in-the-blank item can be interchanged, the unique answer label represents that the reference answer to the question is unique, the candidate answer set label represents that the reference answer to the question has multiple expressions, the keyword set label represents that the reference answer to the question contains all the keywords in the preset keyword set, and the taboo word set label represents that the reference answer to the question does not contain any taboo words in the preset taboo word set.

2. The method according to claim 1, characterized in that When the question type label corresponding to the question includes a permutable label, correcting the answer to be corrected by using the determined correction strategy specifically includes: Constructing a bipartite graph, wherein the nodes on one side of the bipartite graph are the answers to be corrected for each fill-in-the-blank item, and the nodes on the other side of the bipartite graph are the reference answers for each fill-in-the-blank item; In the bipartite graph, determining the maximum number of matches between the answer to be corrected and the reference answer; Based on the maximum matching number, a correction result of the answer to be corrected is determined.

3. The method according to claim 2, characterized in that Based on the maximum matching number, determining the correction result of the answer to be corrected specifically includes: If the maximum matching number is consistent with the number of fill-in-the-blank items of the question, the answer to be corrected is corrected; If the maximum matching number is inconsistent with the number of fill-in-the-blank items of the question, the answer to be corrected will be corrected as wrong.

4. The method according to claim 2, characterized in that: Based on the maximum matching number, determining the correction result of the answer to be corrected specifically includes: If the maximum matching number is 0, the answer to be corrected is corrected as wrong; If the maximum matching number is not 0, the answer to be corrected is corrected to the blank items with the maximum matching number.

5. The method according to claim 1, characterized in that When the question type label corresponding to the question includes a keyword set label and a taboo word set label, correcting the answer to be corrected by using the determined correction strategy specifically includes: If the answer to be corrected contains all the keywords in the preset keyword set and does not contain any taboo words in the preset taboo word set, correct the answer to be corrected; If the answer to be corrected does not include any keyword in the preset keyword set or the answer to be corrected includes any taboo word in the preset keyword set, the answer to be corrected is corrected as an error.

6. The method according to claim 1, characterized in that When the question type tag corresponding to the question includes a keyword set tag, correcting the answer to be corrected by using the determined correction strategy specifically includes: If the answer to be corrected contains all the keywords in the preset keyword set, correcting the answer to be corrected; If the answer to be corrected does not contain any keyword in the preset keyword set, the answer to be corrected is corrected as an error.

7. The method according to claim 1, characterized in that When the answer to be corrected and the reference answer can be converted into a fraction form or a false fraction form, and the question type label corresponding to the question includes a candidate answer set label, correcting the answer to be corrected using the determined correction strategy specifically includes: Convert the answer to be corrected and the reference answer into fractions or false fractions. If the false fractions corresponding to the converted answer to be corrected are consistent with those corresponding to the converted reference answer, correct the answer to be corrected to be correct. If the false fractions corresponding to the converted answer to be corrected are inconsistent with those corresponding to the converted reference answer, correct the answer to be corrected to be incorrect.

8. The method according to claim 1, characterized in that When the answer to be corrected and the reference answer have numbers, and the question type label corresponding to the question includes a candidate answer set label, correcting the answer to be corrected using the determined correction strategy specifically includes: The numbers in the answer to be corrected and the numbers in the reference answer are converted into reference number format. If the converted answer to be corrected is consistent with the converted reference answer, the answer to be corrected is corrected as correct. If the converted answer to be corrected is inconsistent with the converted reference answer, the answer to be corrected is corrected as incorrect.

9. The method according to claim 1, characterized in that: When there are mathematical formulas in the answer to be corrected and the reference answer, and the question type label corresponding to the question includes a candidate answer set label, correcting the answer to be corrected by using the determined correction strategy specifically includes: The mathematical formulas in the answer to be corrected and the mathematical formulas in the reference answer are converted into reference formula formats. If the converted answer to be corrected is consistent with the converted reference answer, the answer to be corrected is corrected as correct; if the converted answer to be corrected is inconsistent with the converted reference answer, the answer to be corrected is corrected as incorrect.

10. The method according to claim 1, characterized in that When there are mathematical terms in the answer to be corrected and the reference answer, and the question type label corresponding to the question includes a keyword set label, correcting the answer to be corrected by using the determined correction strategy specifically includes: Determine a synonym set corresponding to the mathematical noun in the answer to be corrected in a preset synonym library, and if the synonym set includes all the keywords in the preset keyword set, correct the answer to be corrected; If the synonym set does not include any keyword in the preset keyword set, the answer to be corrected is corrected as an error.

11. The method according to claim 1, characterized in that: When the question type label corresponding to the question includes a unique answer label, correcting the answer to be corrected by using the determined correction strategy specifically includes: If the answer to be corrected is consistent with the reference answer, the answer to be corrected is corrected; If the answer to be corrected is inconsistent with the reference answer, the answer to be corrected will be corrected as wrong.

12. The method according to claim 1, characterized in that Get the fill-in-the-blank questions to be corrected, including: Obtaining an image of the fill-in-the-blank question to be corrected, performing OCR recognition on the image, and obtaining an OCR recognition result; Filtering out preset characters from the OCR recognition results, wherein the preset characters are a number of characters with high recognition error probability in OCR recognition that are previously counted; The similarity between the preset character and the reference character is calculated, and the preset character with a similarity higher than a preset threshold is replaced with the reference character, and the replaced OCR recognition result is used as the obtained fill-in-the-blank question to be corrected.

13. The method according to claim 1, characterized in that The method further comprises: Acquire the correction result, and select the target fill-in-the-blank question from the fill-in-the-blank questions to be corrected according to the correction result, wherein the target fill-in-the-blank question is a fill-in-the-blank question that has been corrected as wrong more frequently than a preset frequency; Re-marking the target fill-in-the-blank question using a pre-trained large language model to obtain a re-marking result; Based on the re-marking result, the question type label corresponding to the target fill-in-the-blank question is updated.

14. The method according to claim 13, characterized in that Based on the re-marking result, updating the question type label corresponding to the target fill-in-the-blank question specifically includes: If the re-marking result is that the answer to the target fill-in-the-blank question is correct, and the target fill-in-the-blank question has multiple fill-in-the-blank items, assign a replaceable label to the target fill-in-the-blank question; Alternatively, if the re-marking result is that the answer to the target fill-in-the-blank question is correct, and the answer to the target fill-in-the-blank question is a number or a mathematical formula, the answer to the target fill-in-the-blank question is used as a reference answer to the target fill-in-the-blank question; Alternatively, if the re-marking result is that the answer to the target fill-in-the-blank question is correct, a preset keyword generation function is used to generate keywords for the answer to the target fill-in-the-blank question, and the generated keywords are added to a preset keyword set.

15. A fill-in-the-blank question correction device, characterized in that: include: A fill-in-the-blank question acquisition module to be corrected is used to obtain the fill-in-the-blank question to be corrected, wherein the fill-in-the-blank question to be corrected includes the title of the fill-in-the-blank question to be corrected and the answer to be corrected; A question type label assignment module, used to assign question type labels to the questions using a pre-trained large language model; the question type labels include at least one of a permutable label, a unique answer label, a candidate answer set label, a keyword set label, and a taboo word set label; A marking strategy determination module, used to determine the marking strategy of the answer to be marked according to the question type label corresponding to the question; A correction module, used for correcting the answers to be corrected by using the determined correction strategy; Among them, the interchangeable label represents that there are multiple fill-in-the-blank items in the question and the order of the reference answers of each fill-in-the-blank item can be interchanged, the unique answer label represents that the reference answer to the question is unique, the candidate answer set label represents that the reference answer to the question has multiple expressions, the keyword set label represents that the reference answer to the question contains all the keywords in the preset keyword set, and the taboo word set label represents that the reference answer to the question does not contain any taboo words in the preset taboo word set.

16. A computer device, characterized in that: The method comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method according to any one of claims 1 to 13 when executing the computer program.

17. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 13 is implemented.

18. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the method according to any one of claims 1 to 13 is implemented.

Images