Point rating support device, and control method and control program therefor
The scoring support device automates grading for descriptive and essay questions by using image processing and similarity calculation, reducing workload and standardizing evaluations.
Patent Information
- Application Number
- JP2024150511
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2024-09-02
- Publication Date
- 2024-11-08
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
Existing grading methods for descriptive and essay questions require manual scoring by multiple graders, leading to increased workload and variability in evaluation.
A scoring support device that automates the grading process by using image processing, similarity calculation, and judgment information updating to determine correct/incorrect answers based on predetermined thresholds, reducing the need for manual scoring.
The device efficiently reduces the workload and standardizes grading, ensuring consistent evaluation across multiple graders.
Smart Images

Figure 2024159967000001_ABST
Abstract
Description
[Technical field]
[0001] The present invention relates to a marking support device that supports the marking process of answers to descriptive or essay-style questions, and a control method and control program for the same. [Background technology]
[0002] Conventionally, university entrance examinations, various qualification examinations, and mock examinations for exam preparation have adopted multiple-choice methods, which are broadly classified into multiple-choice methods in which answers are selected from multiple options using a mark sheet, descriptive methods in which answers are given using terms, phrases, or simple sentences, and essay methods such as short essays. In these examinations, multiple-choice methods using a mark sheet are graded using an automatic grading method. On the other hand, the descriptive and essay methods require grading by one or more graders, which increases the grading work time and increases the workload, and there are problems with variations in the evaluation of answers.
[0003] For this reason, various methods have been proposed to solve such problems. For example, Patent Document 1 describes a method in which model answer data and answer data of answerers are presented to a grader terminal of a grader for each question in the descriptive or essay style, and a grading process is executed, so that grading of the same answer sheet is divided among multiple graders at the same time, while grading of each question is assigned to a specific grader. [Prior art documents] [Patent documents]
[0004] [Patent Document 1] Publication No. 2004-069746 Summary of the Invention [Problem to be solved by the invention]
[0005] In the marking method described above, basically, one or more markers mark the exams manually based on model answers, etc., so the workload during marking could not be sufficiently reduced and there was room for improvement.
[0006] Therefore, an object of the present invention is to provide a marking support device, a control method and a control program for the same, which can reduce the workload when marking descriptive and essay-style questions and enable more efficient marking. [Means for solving the problem]
[0007] The scoring support device of the present invention is equipped with a control unit that executes a correct / incorrect judgment process that judges whether answer data for a question is correct or incorrect based on judgment information that indicates the criteria for judging whether the question is correct or incorrect, and a judgment information update process that updates the judgment information based on the judgment result of whether the answer data is correct or incorrect judged by the grader, and is characterized in that the control unit judges whether the answer data is correct or incorrect based on the judgment information in which it is judged that the correct / incorrect judgment rate calculated based on the judgment result of whether the answer is correct or incorrect by the grader in the correct / incorrect judgment process has reached a predetermined threshold value. Effect of the Invention
[0008] According to the present invention, the workload of marking descriptive and essay style questions can be reduced, and marking can be performed more efficiently. [Brief description of the drawings]
[0009] [Figure 1] 1 is a block diagram showing an embodiment of a scoring support device according to the present invention; [Diagram 2] 10 is a flowchart illustrating an example of a control method of the scoring support device according to an embodiment. [Diagram 3] 13 is a flowchart showing a method of registering answer data in an example of a control method of the scoring support device according to an embodiment. [Figure 4]13 is a flowchart showing setting of scoring criteria in an example of a control method of the scoring support device according to an embodiment. [Diagram 5] FIG. 13 is a schematic diagram showing a modified example of the scoring support device according to an embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0010] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, a scoring support device, a control method and a control program thereof according to the present invention will be described in detail with reference to embodiments. FIG. 1 is a block diagram showing an embodiment of a marking support device according to the present invention.
[0011] 1, the scoring support device 100 according to the present invention generally includes an image reading unit 110, an image processing unit 120, a similarity calculation unit 130, a judgment information update unit 140, an achievement level judgment unit 150, a correct / incorrect judgment processing unit 160, a display unit 170, an input operation unit 180, and a storage unit 190. Here, the scoring support device 100 is realized by a system including an information processing terminal such as a personal computer. At least the image processing unit 120, the similarity calculation unit 130, the judgment information update unit 140, the achievement level judgment unit 150, and the correct / incorrect judgment processing unit 160 are realized by an arithmetic processing unit such as a CPU (Central Processing Unit) or an MPU (Microprocessor) or a computer.
[0012] The image reading unit 110 reads the answer sheet as image data, and stores the image data in a predetermined storage area of the storage unit 190 in association with the attributes of the answerer (such as the examinee number). Here, the image reading unit 110 may be provided integrally with other components of the marking support device 100, or may have a separate configuration, such as an external scanner.
[0013] The image processing unit 120 performs a predetermined image processing on the image data of the answer sheet read by the image reading unit 110, and obtains answer data that has been character-recognized and converted into text for each question. Specifically, the image processing unit 120 breaks down the image data of the answer sheet read by the image reading unit 110 into each question, and extracts image data for descriptive and essay-style questions to be graded. The image processing unit 120 then performs character recognition processing on each of the extracted image data, and converts the answer contents into text. The answer data of the answer contents to be graded that has been converted into text data is sorted in order of the questions with the most number of answerers, and is stored in a predetermined storage area of the storage unit 190.
[0014] The similarity calculation unit 130 calculates the similarity to the correct answer (correct answer) of the question and the similarity to the incorrect answer. Here, for example, when the important words and sentence structure of the model answer that is the correct answer are substantially the same, similarity A, which is the similarity to the correct answer, is set to 100%, and similarity B, which is the similarity to the incorrect answer, is set to 0%, and when the important words or synonymous sentence structures, etc. are not used at all, similarity A is set to 0% and similarity B is set to 100%, and the similarity to the correct answer and the similarity to the incorrect answer are calculated according to the frequency of use of the important words, synonymous sentence structures, etc.
[0015] Specifically, the similarity calculation unit 130 first performs morphological analysis on the answer data for the extracted question, and vectorizes it according to the importance of each topic (noun) in the answer data. If the importance of all topics (nouns) in the model answer is vector A, and the importance of all topics (nouns) in the answer data is vector B, the product of the magnitude of vector A and the magnitude of vector B is used as the denominator, and the sum of the products of the importance vectors of each topic in vector A and the topic vectors of the corresponding answer data is used as the numerator to calculate the correct answer similarity of the answer data to the model answer. Similarly, if the importance of an incorrect topic different from all topics (nouns) in the model answer is vector C, the product of the magnitude of vector C and the magnitude of vector B is used as the denominator, and the sum of the products of the importance vectors of each topic in vector C and the topic vectors of the corresponding answer data is used as the numerator to calculate the incorrect answer similarity of the answer data to the model answer. The calculation processing means stores in the storage unit 190 the correct answer similarity and the incorrect answer similarity of the answer data with respect to the model answer.
[0016] The judgment information update unit 140 updates the parameters that are the grading criteria based on the judgment result of whether the answer to the question is correct / incorrect. Specifically, the judgment information update unit 140 updates the "correct answer list / incorrect answer list" that indicates correct answers and incorrect answers and is the criterion for grading each question based on the judgment result of "correct answer / incorrect answer" judged by the grader by comparing the answer content converted into text data by the image processing unit 120 with the model answer of the question, or based on the similarity calculated by the similarity calculation unit 130. In addition, the judgment information update unit 140 updates the correct / incorrect judgment rate calculated based on the judgment of "correct answer / incorrect answer" by the grader. The updated "correct answer list / incorrect answer list" and the correct / incorrect judgment rate are stored in a predetermined storage area of the storage unit 190.
[0017] The achievement level determination unit 150 determines the degree of achievement of the parameter threshold updated by the determination information update unit 140. Specifically, when the updated correct / incorrect determination rate reaches the level of a preset threshold (determination threshold Ath, Bth), the achievement level determination unit 150 determines that a sufficiently reliable correct / incorrect determination can be realized based on the above-mentioned "correct / incorrect list". This determination result is stored in a predetermined storage area of the storage unit 190.
[0018] The correct / incorrect judgment processing unit 160 judges whether the answer to the question is correct / incorrect based on the parameters that the achievement level judgment unit 150 judges to have reached a predetermined achievement level. Specifically, the correct / incorrect judgment processing unit 160 automatically judges whether the answer to the question is correct / incorrect based on the "correct answer list / incorrect answer list" whose accuracy judgment rate has been judged to have reached a predetermined threshold by the achievement level judgment unit 150, and determines the score based on the judgment. The determined score is saved in a predetermined memory area of the memory unit 190.
[0019] The display unit 170 has a display or a display panel, and displays various data handled in each of the above-mentioned units. The input operation unit 180 has an input device such as a keyboard or a mouse, and selects data or a menu displayed on the display unit 170 to execute a predetermined process. The storage unit 190 stores various data handled or generated in each of the above-mentioned units in a predetermined storage area. The storage unit 190 also stores a control program for realizing the processes and functions of each of the above-mentioned units. Here, the storage unit 190 may be partially or entirely in the form of a removable storage medium, and may be configured to be detachable from the scoring support device 100. The storage unit 190 is a memory that stores the answer contents, the number of people who answered, the answer content vector, the normalized marking result, and the like.
[0020] In this embodiment, although not shown, the scoring support device 100 may include an interface unit (I / F unit) for transmitting and receiving various data between the external device and the external device. Here, the I / F unit may be directly connected to the external device or may be connected via a network. In this case, a wired connection method connected via a communication cable or a wireless connection method based on a communication standard such as Wi-Fi (registered trademark) can be applied.
[0021] (Method of controlling scoring support device) Next, a control method and a control program for the scoring support device according to the present embodiment will be described with reference to the drawings. 2, 3, and 4 are flowcharts showing an example of a control method of the scoring support device according to this embodiment.
[0022] In the control method of the marking support device according to this embodiment, a process of generating a list of correct answers / incorrect answers for each descriptive or essay style question to be marked, and an actual marking process of automatically determining whether the answers to the questions are correct / incorrect based on the generated list of correct answers / incorrect answers and determining the marks based on the determination are executed. These processes are realized by executing a specific program in a calculation processing means applied to the marking support device 100, including the image processing unit 120, the similarity calculation unit 130, the judgment information update unit 140, the achievement judgment unit 150, and the correct / incorrect judgment processing unit 160, or in a computer.
[0023] As shown in the flowcharts of Figures 2, 3, and 4, the control method of the marking support device 100 according to this embodiment is as follows: first, in a business or educational institution that manages the work of marking handwritten answer sheets with writing implements and various information such as the attributes of answerers, the answer sheets of all answerers are read (scanned) at once by the image reading unit 110 of the marking support device 100 after the end of a test (step S102). Here, the image reading unit 110 may read the entire answer sheet, or may specify a specific area and read it.
[0024] Next, in the image processing unit 120 of the marking support device 100, the image data of the answer sheet read by the image reading unit 110 is divided into each question, and image data of the descriptive or essay style questions to be marked is extracted (step S104). The image processing unit 120 executes character recognition processing on each extracted image data, and performs text conversion to convert the answer contents into text (step S106). The answers are on the answer sheet, but text conversion may be performed by recognizing the answer contents by voice and converting them into text, or analog data answers answered by other means may be converted into text. Here, the text-converted answer data is collectively referred to as answer data, including unanswered data. After step S106, the calculation processing means records the unregistered data in the storage unit 190 in association with the question.
[0025] Next, as shown in FIG. 2 and FIG. 3, the process proceeds to step S107. Step S107 executed by the calculation processing means includes steps S202 to S222. First, the calculation processing means specifies the total number of answerers corresponding to the answer data, and registers the total number of answerers in the storage unit 190 (step S202). Then, extracts one question for which the registered number of answerers does not reach the total number of answerers (step S204), and extracts multiple answer data corresponding to the extracted one question from the storage unit 190 (step S206). If the total of the registered number of answerers of the multiple answer data corresponding to the extracted one question does not reach the total number of answerers (step S208 NO), extracts one unregistered answer data from the multiple answer data corresponding to the extracted one question (step S210).
[0026] If registered answer data that is determined to be substantially identical to the extracted unregistered answer data is stored in memory unit 190 (step S212 YES), the unregistered answer data is stored in memory unit 190 as data of a group of registered answer data, the number of registered answerers for the corresponding question is counted up by +1 and stored in memory unit 190 (step S214), and the process returns to step S208.
[0027] On the other hand, if answer data having substantially the same content as the extracted unregistered answer data is not stored in memory unit 190 as having already been registered (step S212 NO), the unregistered answer data is stored in memory unit 190 as data of a new group of registered answer data separate from the answer data already registered (step S216), the number of registered answerers for the corresponding question is incremented by 1 and stored in memory unit 190 (step S218), and the process returns to step S208.
[0028] In step S208, if the total number of registered answerers in steps S214 and S218 reaches the total number of answerers (step S208 YES), and if the number of answerers corresponding to all questions does not reach the total number of answerers (step S220 NO), the process returns to step S204; if the number of answerers reaches the total number of answerers (step S220 YES), the group of answer data for the answer content to be scored that has been converted into digital data is rearranged in order of the question with the largest number of answerers and stored in a specified memory area of memory unit 190 (step S222).
[0029] 2 and 4, the process proceeds to setting the marking criteria (step S108). Step S108, which is executed by the calculation means, includes steps S302 to S332. If the calculation means has not yet completed setting the marking criteria for all questions (step S302 NO), it extracts questions for which marking criteria have not been set, reads one or more groups of corresponding answer data from storage unit 190 (step S304), and extracts the group of answer data whose normalized marking results are unregistered and which has the largest number of registered answerers (step S306).
[0030] When automatic scoring of the extracted answer data group is enabled (YES in step S308), the similarity calculation unit 130 calculates the correct answer similarity and the incorrect answer similarity of each answer data (step S310).
[0031] If the calculated correct answer similarity is equal to or greater than the set correct answer similarity threshold and the calculated incorrect answer similarity is equal to or greater than the set incorrect answer similarity threshold, manual grading (normalized grading result determination) is performed by a grader (step S312 YES), and if the result of manual grading is deemed valid (step S316 YES), the process proceeds to step S318. If the result of manual grading is not deemed valid (step S316 NO), the process proceeds to step S314, where the normalized grading result of the answer data for the model answer is registered based on the similarity by automatic grading. In this case, the higher value of the correct answer similarity and the incorrect answer similarity is registered as the normalized grading result. Here, the correct answer similarity approaches "1" as the similarity to the model answer increases, and the incorrect answer similarity approaches "0" as the similarity to the model answer increases.
[0032] On the other hand, if the calculated correct answer similarity is less than the set correct answer similarity threshold, manual grading by the grader is not performed (step S312 NO), automatic grading is performed, and the normalized grading result of the answer data for the model answer based on the similarity by the automatic grading is registered as an incorrect answer similarity (e.g., "0") (step S314). Also, if the calculated correct answer similarity is equal to or greater than the set correct answer similarity threshold and the calculated incorrect answer similarity is less than the set incorrect answer similarity threshold, manual grading is not performed (step S312 NO), automatic grading is performed, and the normalized grading result of the answer data for the model answer based on the similarity by the automatic grading is registered as a correct answer similarity (e.g., "1") (step S314).
[0033] In step S308, if automatic grading of the extracted answer data group is not enabled (step S308 NO), or in step S316, if it is determined that manual grading by the grader is enabled (step S316 YES), the answer data group is presented to the grader (step S318).
[0034] The grader appropriately grades the presented answer data according to the model answer and inputs the graded results (step S320), and the calculation processing means registers the normalized graded results of the answer data with respect to the model answer based on the similarity according to the input results (step S322).
[0035] Next, the system calculates the graded ratio (step S324), which is the ratio between the number of people in the extracted answer data group who have normalized scoring results registered with priority given to automatic scoring (step S314) and the number of people in the normalized scoring results registered with priority given to manual scoring (step S322). If the calculated value is greater than or equal to the automatic scoring start threshold (step S326 YES), automatic scoring is enabled (step S328). If the calculated value is less than the automatic scoring start threshold (step S326 NO), the system proceeds to step S330.
[0036] In step S330, if the calculated value is equal to or greater than the manual marking start threshold (step S330 If the calculated value is less than the manual marking start threshold value (step S326 NO), the process returns to step S302.
[0037] Then, if the setting of the marking criteria for all questions has been completed (YES in step S302), the process proceeds to adjusting the marking criteria in step S110. In step S110, the correct answer similarity and the incorrect answer similarity between the answer vector of the model answer and the answer vector of the answer data group are calculated, and it is confirmed whether the normalized marking result obtained in step S108 matches the higher correct answer similarity or incorrect answer similarity, and if they do not match, the marker or the marking manager who manages the marking is notified of the fact that they do not match.
[0038] The grader or the scoring manager compares the answer contents that are determined to have a high similarity to the correct answer with the answer contents that are determined to have a high similarity to the incorrect answer and judges whether the scoring results need to be changed. If the grader or the scoring manager judges that a change is necessary, they make corrections to the normalized scoring results and start over from step S110, and if they judge that a change is not necessary, they move on to grading in step S112. In step S112, the grader assigns points to each group of answer data according to the points allocated by the normalized scoring results, and grading of all answerers is completed when grading of all groups of all questions is completed.
[0039] Note that step S107 can be omitted by having a grader manually grade all answers without using the arithmetic processing means, in which case all answers are considered to have been registered and the process proceeds to step S108.
[0040] Also, step S107 can be omitted by having the same grader grade similar answers. In this case, a list of correct answers / incorrect answers that have been graded by each grader is created, and answers similar to the correct answer list / incorrect answer list are graded by the grader.
[0041] (Modification) Next, a modified example of the scoring support device according to the present embodiment will be described. FIG. 5 is a schematic diagram showing a modified example of the scoring support device according to the present embodiment.
[0042] The scoring support device 100 according to the above-described embodiment can be applied to a center device installed in a business, an educational institution, etc. The present invention is not limited to such an embodiment, and may have a modified example in which, for example, some of the functions of the scoring support device 100 are executed by other external devices or terminals.
[0043] Specifically, as shown in Fig. 5, the scoring support device according to the modified example has a center device 210 installed in a business operator or the like, grader terminals 220 assigned to a plurality of graders, and a network 300 to which at least the center device 210 and the grader terminals 220 are connected. Here, the network 300 may be a network available for public use such as the Internet, or may be a network available only to a specific organization such as a business operator or an educational institution.
[0044] The center device 210 has a configuration equivalent to that of the marking support device 100 shown in the above-mentioned embodiment, for example, and the grader terminal 220 has a configuration including at least a display unit, an input operation unit, a storage unit, an I / F unit, and a CPU for controlling each of these units. Here, the grader terminal 220 receives answer data and correct answer data of the question to be marked provided from the center device 210 via the network 300 and displays them on the display unit, so that the person in charge of marking can judge whether the answer is correct or incorrect. The judgment result is transmitted to the center device 210 via the network 300.
[0045] As a result, among the control methods shown in the above-mentioned embodiments, processes including steps that require judgment by the grader can be executed on the grader terminal 220, so that work for each question can be done simultaneously by multiple graders, reducing the burden caused by increased work time during grading, and even when grading is done by multiple graders, variation in the evaluation of answers can be suppressed, making grading more efficient.
[0046] 5, the grader terminal 220 is not limited to the form in which it receives (downloads) answer data and correct answer data for questions to be graded from the center device 210 via the network 300 and judges whether the answers are correct or incorrect. For example, the grader terminal 220 may be a cloud-type terminal that allows the correct answer data and answer data for questions to be graded to be used only when the center device 210 is accessed via the network 300, and data is not downloaded and saved in the grader terminal 220. According to this form, it is possible to prevent leakage of personal data and important data related to the grading work, including the grading results and attributes of the answerers.
[0047] In the above embodiment, it is preferable to initially disable automatic marking and perform manual marking, and then switch to automatic marking in step S308 after data has been accumulated. In the above embodiment, the points assigned to incorrect answers may be divided into 0 points for completely wrong answers and partial points higher than 0 points, and the point assigned to the incorrect answer with the highest similarity may be used.In addition, if there are multiple similarities that exceed a set threshold value, the average point assigned to the multiple similarities that exceed the threshold value may be used as the point assigned.
[0048] Although several embodiments of the present invention have been described above, the present invention is not limited to the above-described embodiments, but includes combinations of the respective configurations of the above-described embodiments to the extent feasible, and also includes the inventions described in the claims and their equivalents. The invention as originally claimed in the present application is set forth below.
[0049] (Additional Note) [1] classifying answer data including answers to questions to be scored into groups of similar answer contents for each question; A marking support device which sets marking standards for each group of similar answer contents.
[0050] [2] The scoring support device described in [1] is characterized in that the setting of the scoring criteria includes switching from automatic scoring to manual scoring by a grader based on the similarity of correct answers and incorrect answers of each answer content to the model answer content.
[0051] [3] The scoring support device described in [1] or [2] is characterized in that at least one of enabling automatic scoring and disabling manual scoring is set according to a scoring ratio corresponding to the normalized scoring result by automatic scoring or the normalized scoring result by manual scoring based on the set scoring criteria.
[0052] [4] classifying answer data including answers to questions to be scored into groups of similar answer contents for each question; A method for controlling a marking support device, comprising the steps of: setting a marking standard for each group of similar answer contents.
[0053] [5] On the computer, Classifying answer data including answers to questions to be scored into groups of similar answer contents for each question; A control program for a marking support device, which causes marking criteria to be set for each group of similar answer contents. [Explanation of symbols]
[0054] 100 Scoring Support Device 110 Image reading unit 120 Image Processing Unit 130 Similarity calculation unit 140 Judgment information update section 150 Achievement Judgment Department 160 Correctness judgment processing unit 170 Display section 180 Input operation section 190 Memory section
Claims
1. a correct / incorrect determination process for determining whether the answer data to the question is correct or incorrect based on determination information indicating a criterion for determining whether the answer to the question is correct or incorrect; a judgment information update process for updating the judgment information based on a judgment result of whether the answer data is correct or incorrect judged by a grader; A control unit that executes The control unit determines whether the answer data is correct or incorrect based on the determination information that indicates that a correct / incorrect determination rate calculated based on a result of the grader's determination of whether the answer is correct or incorrect has reached a predetermined threshold value in the correct / incorrect determination process. Scoring support device.
2. In the judgment information update process, the control unit updates the judgment information based on a comparison result between the answer data and a model answer to the question and a judgment result of whether the answer data is correct or incorrect judged by a grader. The scoring support device according to claim 1 .
3. In the judgment information update process, the control unit updates the judgment information based on a similarity between the answer data and a model answer to the question and a judgment result of whether the answer data is correct or incorrect judged by a grader. The scoring support device according to claim 1.
4. The control unit, in the correct / incorrect judgment process, judges whether the answer data for each of the multiple questions is correct or incorrect based on the multiple judgment information indicating a criterion for judging whether the answer is correct or incorrect for each of the multiple questions, calculates a correct / incorrect judgment rate for each question based on the result of the judgment by the grader, and judges whether the answer data for each question is correct or incorrect based on the judgment information in which it is judged that the calculated correct / incorrect judgment rate has reached a predetermined threshold value; In the judgment information update process, the control unit updates the judgment information for each of the plurality of questions based on a judgment result of whether the answer data for each question is correct or incorrect judged by a grader. The scoring support device according to claim 1 .
5. the control unit executes a similarity calculation process to calculate a correct answer similarity of the answer data to the model answer based on a first vector indicating the importance of each of a plurality of topics extracted from the answer data and a second vector indicating the importance of each of a plurality of topics extracted from the model answer, and calculates an incorrect answer similarity of the answer data to the model answer based on the first vector and a third vector indicating the importance of each of a plurality of topics different from the topics extracted from the model answer; the control unit, in the judgment information update process, updates the judgment information based on a similarity between the answer data and a model answer of the question, which is determined based on the correct answer similarity and the incorrect answer similarity calculated by the similarity calculation process, and a judgment result of whether the answer data is correct or incorrect, judged by a grader. The scoring support device according to claim 1.
6. the control unit performs manual grading by a grader when the calculated correct answer similarity is equal to or greater than a set correct answer similarity threshold and the calculated incorrect answer similarity is equal to or greater than a set incorrect answer similarity threshold for each answer data. The scoring support device according to claim 5.
7. The device, a correct / incorrect determination process for determining whether the answer data to the question is correct or incorrect based on determination information indicating a criterion for determining whether the answer to the question is correct or incorrect; a judgment information update process for updating the judgment information based on a judgment result of whether the answer data is correct or incorrect judged by a grader; Run The correct / incorrect determination process includes a process of determining whether the answer data is correct or incorrect based on the determination information in which it is determined that a correct / incorrect determination rate calculated based on a result of the determination by the grader as to whether the answer data is correct or incorrect has reached a predetermined threshold value. Scoring support methods.
8. On the computer, a correct / incorrect determination process for determining whether the answer data to the question is correct or incorrect based on determination information indicating a criterion for determining whether the answer to the question is correct or incorrect; a judgment information update process for updating the judgment information based on a judgment result of whether the answer data is correct or incorrect judged by a grader; Run the command, The correct / incorrect determination process includes a process of determining whether the answer data is correct or incorrect based on the determination information in which it is determined that a correct / incorrect determination rate calculated based on a result of the determination by the grader as to whether the answer is correct or incorrect has reached a predetermined threshold value. Control program.
Citation Information
Patent Citations
Marking support method, marking support system, marking support device, marking management device and computer program
JP2006277086A
Integration of automatic and manual defect classification
JP2013224943A
Point rating rule application object specification program, erroneous judgment rule setting program, point rating rule application object specification method, erroneous judgment rule setting method, application object specification device, and erroneous judgment rule setting unit
JP2016212284A
Program, device and method for estimating score of text by calculating multiple similarity degrees
JP2017188039A
Answer grading method, answer grading system, and program
JP2004069746A
