Educational support system, educational support method, and program

Abstract

To provide an automatic scoring program, server, automatic scoring method, and automatic scoring system, which enable easy checking of automatic scoring results.SOLUTION: An automatic scoring program disclosed herein causes a processor to perform steps of receiving input of information including answers to questions, scoring the answers according to one or more evaluation items preset for the questions, and displaying scoring results in different ways according to evaluation items matching the answers.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an automatic marking program, a server, an automatic marking method, and an automatic marking system. [Background technology]

[0002] In recent years, online classes have been introduced. In these online classes, exams and other tests are also conducted online. Systems that can automatically grade such online exams have also been proposed. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2019-61189 A Summary of the Invention [Problem to be solved by the invention]

[0004] In order to improve the accuracy of automatic marking, it is desirable for the marking results to be finally confirmed by a human. Here, even in automatic marking, partial points may be given based on a predetermined marking standard. When confirming, it is considered important that the user can easily understand what marking standard was used for marking.

[0005] An object of the present invention is to provide an automatic marking program, a server, an automatic marking method, and an automatic marking system that enable easy confirmation of the results of automatic marking. [Means for solving the problem]

[0006] The automatic marking program of the first aspect of the present invention causes a processor to accept input of information including an answer to a question, mark the answer according to one or more evaluation items pre-set for the question, and distinguish on a display the marking results according to the evaluation items that match the answer.

[0007] A server according to a second aspect of the present invention is a server capable of communicating with a terminal and comprises a processor configured to accept input of information including an answer to a question, grade the answer according to one or more evaluation items pre-set for the question, differentiate on a display the results of the grading according to the evaluation items that match the answer, and transmit the results of the grading to the terminal.

[0008] An automatic marking method of a third aspect of the present invention is an automatic marking method executed by an automatic marking system, and comprises accepting input of information including an answer to a question, marking the answer in accordance with one or more evaluation items pre-set for the question, and displaying a distinction between the marking results according to the evaluation items that match the answer.

[0009] An automatic marking system of a fourth aspect of the present invention has a server having a first processor configured to accept input of information including answers to questions, mark the answers according to one or more evaluation items preset for the questions, distinguish on display the marking results according to the evaluation items that match the answers, and transmit the marking results, and a terminal having a second processor configured to receive the marking results transmitted from the server and display the received marking results on a display device. Effect of the Invention

[0010] According to the present invention, it is possible to provide an automatic marking program, a server, an automatic marking method, and an automatic marking system that allow the results of automatic marking to be easily confirmed. [Brief description of the drawings]

[0011] [Figure 1] FIG. 1 is a diagram showing an example of a configuration of a system according to an embodiment of the present invention. [Diagram 2] FIG. 2 is a diagram showing an example of a screen display by an application running on a web browser of the terminal 20a. [Diagram 3]FIG. 3 is a sequence diagram showing an example of the operation of the system when creating an answer paper for an examination or the like. [Figure 4A] FIG. 4A is a diagram showing an example of a dialog displayed on the paper. [Figure 4B] FIG. 4B is a diagram showing an example of a dialog displayed on the paper. [Figure 4C] FIG. 4C is a diagram showing an example of a dialog displayed on the paper. [Diagram 5] FIG. 5 is a sequence diagram showing an example of the operation of the system when answering a question. [Figure 6] FIG. 6 is a diagram showing an example of the display screen of the terminal 20b on which the answer paper is displayed. [Figure 7] FIG. 7 is a sequence diagram showing an example of the operation during scoring. [Figure 8] FIG. 8 is a flowchart showing an example of the process of the server during automatic marking. [Figure 9] FIG. 9 is a diagram showing an example of the display screen of the terminal 20a on which a paper showing the automatic marking results is displayed. [Figure 10] FIG. 10 is a diagram showing an example of the display screen of the terminal 20a on which a paper showing the automatic English marking result is displayed. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] An embodiment of the present invention will be described with reference to the drawings. FIG. 1 is a diagram showing an example of the configuration of a system 1 according to an embodiment of the present invention. The system 1 includes a server 10 and terminals 20a and 20b. The server 10 and the terminals 20a and 20b are communicatively connected via a network 30. The network 30 is, for example, the Internet. The number of terminals is not limited to two. At least one terminal 20a and at least one terminal 20b are required.

[0013] The server 10 includes a processor 11, a ROM 12, a RAM 13, a storage 14, and a communication device 15. These are all connected to each other via a system bus 19.

[0014] The processor 11 may be an integrated circuit including a CPU (Central Processing Unit) and the like. The ROM 12 records information used for the operation of the processor 11 and the like. The RAM 13 is a main storage device for the operation of the processor 11 and the like. The storage 14 stores various programs, such as a server control program used by the processor 11, an arithmetic program for executing various arithmetic operations, parameters, and the like. The server control program includes an automatic scoring program. The processor 11 controls the operation of the server 10 according to the programs stored in the storage 14. The processor 11 may be a processor other than a CPU, such as an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or a GPU (Graphic Processing unit). The communication device 15 includes a circuit for communicating with an external communication network such as the network 30.

[0015] The terminals 20a and 20b may be electronic devices such as personal computers (PCs), tablet terminals, and smartphones. The terminals 20a and 20b may also be scientific calculators equipped with communication functions. The terminal 20a is a terminal operated by a test grader such as a teacher. The terminal 20b is a terminal operated by a test taker such as a student. Hereinafter, the configuration of the terminal 20a will be described assuming that the terminals 20a and 20b have the same configuration. In addition, below, the grader may be distinguished as a first user and the test taker as a second user, as necessary.

[0016] The terminal 20a has a CPU 21, a ROM 22, a RAM 23, a storage 24, an input device 25, a display device 26, and a communication device 27. Each of these is connected to each other via a system bus 29. Note that the terminal 20a and the terminal 20b do not necessarily have to have the same configuration.

[0017] The CPU 21 is a processor that controls various operations of the terminal 20a. The ROM 22 records a startup program for the terminal 20, etc. The RAM 23 is a main storage device for the CPU 21, etc. The storage 24 stores various programs, such as a terminal control program, used by the CPU 21, parameters, etc. The CPU 21 controls the operation of the terminal 20a by executing various programs in response to an input signal from the input device 25 or a signal received by the communication device 27. The various programs may be downloaded to the storage 24 from a web server (not shown) via the network 30 and the communication device 27. The communication device 27 includes a circuit for communicating with an external communication network such as the network 30.

[0018] The input device 25 includes a keyboard, a mouse, a touch panel, etc. In response to a user's operation via the input device 25, a signal indicating the content of the user's operation is input to the CPU 21 via the system bus 29.

[0019] The display device 26 is a liquid crystal display, an organic EL display, etc. The display device 26 may be provided integrally with the terminal 20a, or may be provided separately from the terminal 20a. Various images are displayed on the display device 26.

[0020] In one example, the user specifies the address of the server 10 in a web browser running on the terminal 20a. At this time, a display screen for a web application stored in the server 10 is displayed on the web browser on the terminal 20a. A request to the server 10 is issued in response to an operation on this display screen via the input device 25. This operation includes, for example, an operation related to grading an exam taken by the user of the terminal 20b. The server 10 executes a process in response to the request and returns the process result to the terminal 20a as a response. The terminal 20a performs a display or the like in response to the user's operation according to the response from the server 10. In this way, in the system 1, a function as a web application for exams or the like is realized by a program running on the web browser of the terminal 20a and a calculation program of the server 10. Similarly, in the system 1, a function as a web application for exams or the like is realized by a program running on the web browser of the terminal 20b and a calculation program of the server 10.

[0021] Such web applications can be used, for example, in mathematics classes in the field of school education where ICT (Information and Communication Technology) is becoming more prevalent.

[0022] FIG. 2 is a diagram showing an example of a screen display by an application running on a web browser of the terminal 20a.

[0023] The screen 26a displayed on the display device 26 of the terminal 20a includes an upper area 100a and a lower area 100b. The upper area 100a is displayed on the upper side of the screen 26a. The upper area 100a is a long and narrow area compared to the lower area 100b. A new paper creation icon 100c is displayed in the upper area 100a. Also, an answer sheet creation button 100d is displayed in the upper area 100a. The lower area 100b is an area located below the upper area 100a in the screen 26a. In the following description, the lower area 100b is also referred to as paper 100. Various "sticky notes" 101 may be displayed on the paper 100. The sticky notes 101 are a display area for displaying various information related to a web application. For example, the sticky notes 101 include a formula sticky note for creating a formula, a graph sticky note for creating a graph, a table sticky note for creating a table, a figure sticky note for creating a figure, and a comment sticky note for creating a comment. The sticky note 101 may be a floating object. A floating object is an object (display body) that is displayed on the screen and can change at least the display position in response to a user operation.

[0024] In this embodiment, the system 1 can create various sticky notes starting from a blank paper 100. The same paper 100 as that of the terminal 20a can be displayed in the terminal 20b. However, the answer sheet creation button 100d does not have to be displayed on the paper 100 of the terminal 20b.

[0025] A series of processing flows in the system 1 will be described below. FIG. 3 is a sequence diagram showing an example of the operation of the system 1 when creating an answer paper for an exam or the like. It is assumed here that questions and marking criteria for the questions have been created by a first user such as a teacher prior to the processing of FIG. 3. The marking criteria for the questions are stored in the server 10 as a marking criteria information file. The marking criteria information file is a file that contains marking criteria information as data in a text format such as JSON format or XML format. An example of creating a mathematics answer paper will be described below.

[0026] The grading criteria information includes, for example, information such as "question name," "answer type," "model answer," "grading criteria," and "comments." "Question name," "answer type," "model answer," "grading criteria," and "comments" may be stored for each question. For example, if there are three questions, three each of "question name," "answer type," "model answer," "grading criteria," and "comments" may be stored.

[0027] "Question name" is information that indicates the name of a question. A "question name" is uniquely assigned to each question in the same test. The "question name" may be assigned by the first user by inputting text, or may be selected by the first user from among several options.

[0028] "Answer type" is information on the type of sticky note that can be used to answer the corresponding question. For example, if the "answer type" for a math question is "mathematical formula," the setting is such that sticky notes other than mathematical formula notes cannot be used to answer the corresponding question.

[0029] The "model answer" is information on a model answer as one of the pieces of information on an evaluation item that a first user such as a teacher expects. When an answer by a second user such as a student matches the model answer described in the "model answer", the answer is determined to be correct.

[0030] The "grading criteria" is information on evaluation items that are the criteria for automatic grading. Specifically, the "grading criteria" is information on evaluation items such as how many points to give as a reference point when the answer of the second user such as a student matches the model answer, how many points to subtract from the reference point when the answer does not match the model answer but is mathematically equivalent, and how many points to add when the answer does not match the model answer and is not mathematically equivalent but should be added, and can be freely described by the first user such as a teacher. "Mathematically equivalent" means that the model answer is equivalent to the model answer although only the expression format is different from that of the model answer, and is in a state where it matches any evaluation item other than the model answer. For example, when the model answer is in a decimal format of "0.25" and the answer by the second user is in a fraction format of "1 / 4", it is determined to be "mathematically equivalent". The "scoring criteria" may be described, for example, by combining an evaluation formula and variables for evaluating whether an answer defined in advance in a web application is mathematically equivalent to a model answer. In addition, exclusion conditions may be described as the "scoring criteria". An answer that meets the exclusion conditions may be determined to be neither identical to nor mathematically equivalent to the model answer. The format of the information described in the "scoring criteria" is not limited to a specific format as long as it is a format that can be used for automatic grading in the server 10.

[0031] "Comment" is text information of a comment added by a teacher, etc. Input of "Comment" may be omitted.

[0032] The process of FIG. 3 is started when the terminal 20a makes a request to the server 10 to start a web application. When making the request to start the web application, a login process such as input of an ID and a password may be performed. Upon receiving the request to start the web application, the server 10 sends a web application program including data of an initial screen to the terminal 20a. In response to this, the terminal 20 displays the initial screen on the web browser. On the initial screen, the paper 100 has not been created, and only a new paper creation icon 100c and an answer sheet creation button 100d are displayed in the upper area 100a. The process of FIG. 3 is executed by the cooperation of the CPU 21 of the terminal 20a and the processor 11 of the server 10.

[0033] In step S1, a first user, such as a teacher, who is in a position to create an answer paper, operates the input device 25 of the terminal 20a to select the new paper creation icon 100c. In step S2, the terminal 20a transmits a request to create a new paper 100 to the server 10. In step S3, the server 10 generates the new paper 100 and transmits it to the terminal 20a.

[0034] In step S4, the first user performs an operation to add a sticky note as necessary. The operation of adding a sticky note may be, for example, an operation of selecting a sticky note addition icon (not shown) displayed on the new paper 100, or an operation of selecting an item for adding a sticky note from a menu displayed by right-clicking the mouse. After adding the sticky note, the first user writes a supplementary comment for the question on the sticky note as necessary. In step S5, the terminal 20a transmits the sticky note information input by the first user to the server 10. The server 10 associates the sticky note information with the paper 100. The sticky note information includes, for example, information such as the position information of the sticky note on the paper 100 and the text written on the sticky note. The processes in steps S4-S5 may be repeated until the first user selects the answer sheet creation button 100d in step S6.

[0035] In step S6, after the first user has finished writing supplementary comments and the like on the sticky note, he / she operates the input device 25 to select the answer sheet creation button 100d. In step S7, the terminal 20a transmits an answer sheet creation request to the server 10. In step S8, the server 10 generates a dialogue for creating an answer paper and transmits the generated dialogue to the terminal 20a. The terminal 20a displays the received dialogue on the paper 100.

[0036] 4A, 4B, and 4C are examples of dialogues displayed on the paper 100. First, a dialogue for selecting a grading criterion information file shown in FIG. 4A is transmitted from the server 10 to the terminal 20a. The dialogue 201 for the grading criterion information file includes a file name display field 201a, a select button 201b, a cancel button 201c, and a next button 201d. The file name display field 201a is a display field in which the file name of the currently selected grading criterion information file is displayed. The select button 201b is a button for accepting an operation to display a selection screen for the grading criterion information file. When the select button 201b is selected, a list of grading criterion information files stored in the server 10 is displayed. When the first user selects a desired grading criterion information file, the file name of the selected grading criterion information file is displayed in the file name display field 201a. The cancel button 201c is a button for accepting an operation to cancel the selection of the grading criterion information file. When the cancel button 201c is selected, the selection of the scoring criteria information file is cancelled and the display of the dialog 201 is terminated. The next button 201d is a button for accepting an operation to confirm the selection of the scoring criteria information file.

[0037] In step S9, the first user sets the grading criteria. Specifically, the first user selects a grading criteria information file in the dialog 201. After selecting the grading criteria information file, the first user selects the next button 201d. In step S10, the terminal 20a transmits information of the selected grading criteria information file, for example, the file name of the grading criteria information file, to the server 10. In step S11, the server 10 adds grading criteria information to the paper 100 based on the information of the grading criteria information file. Specifically, the server 10 associates the grading criteria information file selected by the first user with the paper 100. After that, the server 10 transmits a dialog for the next setting to the terminal 20a. The terminal 20a displays the received dialog on the paper 100.

[0038] A dialogue for setting the answer deadline shown in FIG. 4B is transmitted from the server 10 to the terminal 20a. The dialogue 202 for setting the answer deadline includes an answer deadline display field 202a, a cancel button 202b, and a next button 202c. The answer deadline display field 202a is a display field in which the currently set answer deadline is displayed. The first user can set the answer deadline including the unit of hours, minutes, and seconds by selecting the answer deadline display field 202a. The answer deadline may be set, for example, by inputting text or selecting from a calendar or the like. The cancel button 202b is a button for accepting an operation to cancel the setting of the answer deadline. When the cancel button 202b is selected, the setting of the answer deadline is canceled and the display of the dialogue 202 is also terminated. In this case, the selection of the grading criteria information file may also be canceled. The next button 202c is a button for accepting an operation to confirm the setting of the answer deadline.

[0039] In step S12, the first user sets an answer deadline in the dialog 202. After setting the answer deadline, the first user selects the next button 202c. In step S13, the terminal 20a transmits information about the set answer deadline to the server 10. In step S14, the server 10 adds the information about the answer deadline to the paper 100. Specifically, the server 10 associates the information about the answer deadline with the paper 100. After that, the server 10 generates a unique URL for each paper 100. Then, the server 10 transmits a dialog including the generated URL to the terminal 20a. The terminal 20a displays the received dialog on the paper 100.

[0040] A dialog for notifying the terminal 20a that the creation of the answer paper shown in FIG. 4C has been completed is transmitted from the server 10 to the terminal 20a. The dialog 203 for notifying the terminal 20a that the creation of the answer paper has been completed includes a URL display field 203a, a copy button 203b, and a close button 203c. The URL display field 203a is a URL for accessing the answer paper. The copy button 203b is a button for copying a character string representing the URL displayed in the URL display field 203a. The character string copied with the copy button 203b may be pasted into, for example, an e-mail. The URL is notified to a second user such as a student by this e-mail. The close button 203c is a button for accepting an operation to close the dialog 203.

[0041] In step S15, the first user checks the URL, copies the URL if necessary, and distributes the email to the corresponding terminal 20b of the second user. The email may be distributed by the server 10. After these processes, the first user selects the close button 203c. In step S16, the terminal 20a transmits a notification to the server 10 that the process of creating the answer paper is complete. This allows the server 10 to recognize the end of the process. Incidentally, the display of the paper 100 on the terminal 20a may also be terminated with the end of the process.

[0042] Fig. 5 is a sequence diagram showing an example of the operation of the system 1 when answering a question. The process of Fig. 5 is started when the web browser is started by the terminal 20b. The process of Fig. 5 is mainly executed by the CPU 21 of the terminal 20b and the processor 11 of the server 10 in cooperation with each other.

[0043] In step S101, a second user such as a student operates the terminal 20b to input a URL designated in advance by a first user such as a teacher by e-mail or the like into a web browser. In step S102, the terminal 20b accesses the server 10 according to the input URL. Note that a login process such as input of an ID and a password may be performed when accessing the server 10.

[0044] The server 10 judges whether the status of the paper of the corresponding URL is a status error. For example, the server 10 judges whether the answer deadline set in the paper of the corresponding URL has passed. The server 10 also judges whether the submission status of the second user for the paper of the corresponding URL is "submitted". As will be described later, when the second user answers the question and "submits" the answer, the submission status of the second user is set from "not submitted" to "submitted" in the server 10. The server 10 judges the submission status of the second user who has accessed. If the answer deadline has passed or the submission status of the second user is "submitted", the server 10 judges that there is a status error. If it is judged that there is a status error, in step S103, the server 10 notifies the terminal 20b of the error and then ends the process. This error includes, for example, a message that the answer cannot be given to the paper of the corresponding URL. On the other hand, if the answer deadline has not passed and the submission status of the second user is "not submitted", the server 10 determines that there is no status error. If it is determined that there is no status error, in step S104, the server 10 transmits the paper with the corresponding URL to the terminal 20b. The terminal 20b displays the received paper on the web browser.

[0045] Fig. 6 is a diagram showing an example of the display screen of terminal 20b in a state in which answer paper 300 is displayed. As in Fig. 2, the screen displayed on display device 26 of terminal 20b includes upper area 300a and lower area 300b. Upper area 300a corresponds to upper area 100a, and lower area 300b corresponds to lower area 100b. In addition, when answer paper 300 is displayed, answer submission button 300c is displayed in upper area 300a.

[0046] A sticky note for each question can be added to the answer paper 300 by the second user. The second user can create an answer sticky note when answering a question. Three sticky notes 301, 302, and 303 are displayed in FIG. 6. That is, FIG. 6 shows that the second user has created three sticky notes. The sticky notes 301, 302, and 303 can be added by the second user by selecting from the above-mentioned formula sticky notes, graph sticky notes, table sticky notes, and shape sticky notes.

[0047] The user can make various inputs according to the added sticky note, such as creating a mathematical formula, creating a graph, creating a table, or creating a figure. The user can also cause the server 10 to execute the calculation of the mathematical formula. The calculation result of the mathematical formula is displayed, for example, at the bottom right of the sticky note as the answer to the question. Also, each of the sticky notes 301, 302, and 303 is provided with a check box 301a, 301b, and 301c. For example, when the sticky note 303 is selected, a check is displayed in the corresponding check box 303a. In addition to displaying the check, the selected sticky note may be distinguished from the unselected sticky note by other displays, etc. Also, an answer label may be attached to each sticky note. When the second user selects a drop-down list provided for the answer, a list of available question names for the corresponding sticky note is displayed. When the second user selects a desired question name from the list, an answer label is attached to the sticky note. 6 shows an example in which answer label 301b is assigned to sticky note 301, and answer label 302b is assigned to sticky note 302. Here, in the list of question names, question names with answer types that do not match the sticky note created by the second user are not displayed. For example, for a mathematical formula sticky note, only question names with an answer type set to mathematical formula are displayed in the list. Therefore, sticky note 303 cannot be assigned a question name with an answer type set to mathematical formula.

[0048] In step S105, the second user adds a sticky note to the paper 300 to answer the question. In step S106, the terminal 20b transmits information of the added sticky note to the server 10. The server 10 associates the sticky note information with the paper 300 for each second user. The sticky note information includes, for example, identification information of each sticky note on the paper 300, position information, and information of the answer written on the sticky note. In step S107, the second user attaches an answer label to the sticky note. In step S108, the terminal 20b transmits information of the answer label to the server 10. The server 10 associates the answer label information with the sticky note information for each second user. The answer label information includes, for example, information of the question name represented by the answer label. The processes of steps S105-S108 may be repeated until the second user selects the answer submission button 300c in step S109.

[0049] The second user creates a sticky note, writes an answer, and further attaches an answer label to the sticky note. Then, at step S109, which is the timing when the second user wishes to submit the answer, the second user selects the answer submission button 300c. At step S110, the terminal 20b transmits a notification to the server 10 that the answer will be submitted.

[0050] The server 10 determines whether there is an unanswered question by tallying up the information of the answer label associated with the paper 300 indicating that an answer will be submitted. If it is determined that there is an unanswered question, in step S111, the server 10 transmits a warning dialogue to the terminal 20b. The warning dialogue includes, for example, a message for confirming whether the submission process may be continued even though there is an unsubmitted answer. The warning dialogue may also have a button for accepting an operation to continue the submission of the answer and a button for accepting an operation to cancel the submission of the answer. In this case, the server 10 may determine whether to continue or cancel the process of submitting the answer according to the button selected by the operation of the input device 25 of the second user. If it is determined that the process of submitting the answer is to be continued, the process proceeds to step S112. If it is determined that the process of submitting the answer is to be canceled, the process returns to step S105. If it is determined that there is no unanswered question, the process also proceeds to step S112.

[0051] In step S112, the server 10 changes the submission status of the paper 300 of the second user, who is the user of the terminal 20b that requested the submission of the answer, to "Submitted". Then, the server 10 generates a unique URL for each second user associated with the paper 300 that has been submitted. Then, the server 10 transmits a dialogue including the generated URL to the terminal 20b. The terminal 20b displays the received dialogue on the paper 300. Note that the dialogue displayed on the terminal 20b in the process of step S112 may be the same as the dialogue shown in FIG. 4C, and may be a dialogue that differs only in that the displayed message is a message indicating that the submission of the answer has been completed. Note that the second user can check the paper 300 on which he / she has provided his / her answer using the URL transmitted in step S112. However, the second user is restricted from editing the paper 300 on which he / she has provided his / her answer.

[0052] In step S113, the server 10 copies the URL and delivers an e-mail to the terminal 20a of the first user. The first user can check the paper 300 answered by the second user using the URL sent in step S113, for example, for grading.

[0053] Fig. 7 is a sequence diagram showing an example of the operation during scoring. The process of Fig. 7 is started when the web browser is started by the terminal 20a. The process of Fig. 7 is mainly executed by the CPU 21 of the terminal 20a and the processor 11 of the server 10 in cooperation with each other.

[0054] In step S201, a first user such as a teacher operates the terminal 20a to input a URL notified in advance by the server 10 via e-mail or the like into a web browser. In step S202, the terminal 20a accesses the server 10 according to the input URL. Note that a login process such as input of an ID and password may be performed when accessing the server 10.

[0055] In step S202, the server 10 calls the paper 300 of the input URL. Then, the server 10 extracts the answer by the second user from the sticky note information associated with the paper 300. After that, the server 10 performs automatic grading by comparing the answer by the second user with the grading criteria information associated with the paper 300. After the automatic grading is completed, the server 10 transmits a paper including the automatic grading result to the terminal 20a. The terminal 20a displays the received paper on a web browser.

[0056] 8 is a flowchart showing an example of the process of the server 10 during automatic grading. In step S301, the server 10 selects one answer according to the order of the question labels. If there is a missing question label, the server 10 assigns 0 points to that question label.

[0057] In step S302, the server 10 acquires the reference points for the question names corresponding to the question labels from the marking criteria information, and assigns the reference points to the score of the corresponding question label of the second user. As described above, the reference points are points that are assigned when the answer by the second user is correct.

[0058] In step S303, the server 10 judges whether the answer by the selected second user matches the model answer for the corresponding question name. If it is judged in step S303 that the answer by the second user matches the model answer, the process proceeds to step S304. If it is judged in step S303 that the answer by the second user does not match the model answer, the process proceeds to step S305.

[0059] In step S304, the server 10 sets the background color of the sticky note of the corresponding question label to "green". After that, the process proceeds to step S310. The background color is the background color of the sticky note included in the paper displayed on the terminal 20a as a result of the automatic marking. The background color is changed according to the result of the automatic marking based on the marking criteria information. In other words, when the answer by the second user matches the model answer, the background color is set to "green". Note that the score of the question label corresponding to the second user is given the same standard score as it is.

[0060] In step S305, the server 10 determines whether the answer by the second user is mathematically equivalent to the model answer based on the marking criteria information. Here, in step S305, when the answer by the second user meets the exclusion condition described in the marking criteria information, it may be determined that the answer is not mathematically equivalent to the model answer. In step S305, when it is determined that the answer by the second user is mathematically equivalent to the model answer, the process proceeds to step S306. In step S305, when it is determined that the answer by the second user is not mathematically equivalent to the model answer, the process proceeds to step S308.

[0061] In step S306, the server 10 subtracts points from the reference points for the corresponding question label in accordance with the evaluation items of the scoring criteria.

[0062] In step S307, the server 10 sets the background color of the sticky note of the corresponding question label to "yellow." Then, the process proceeds to step S310. That is, when the answer by the second user does not match the model answer but is mathematically equivalent, the background color is set to "yellow."

[0063] In step S308, the server 10 sets the score of the corresponding question label to 0. Note that in step S308, the server 10 may increase the score from 0 depending on the evaluation item of the marking criteria.

[0064] In step S309, the server 10 sets the background color of the sticky note of the corresponding question label to "red." After that, the process proceeds to step S310. That is, when the answer by the second user is incorrect, the background color is set to "red."

[0065] In step S310, the server 10 judges whether or not to end the automatic marking. For example, when the marking of all question labels is completed, it is judged that the automatic marking is to be ended. When it is judged in step S310 that the automatic marking is not to be ended, the process returns to step S301. When it is judged in step S310 that the automatic marking is to be ended, the process proceeds to step S311.

[0066] In step S311, the server 10 calculates the total score given to each question label.

[0067] In step S312, the server 10 creates a paper showing the automatic marking results, and then ends the process of FIG.

[0068] 9 is a diagram showing an example of the display screen of terminal 20a in a state in which paper 400 showing the automatic marking result is displayed. The screen displayed on display device 26 of terminal 20a includes upper area 400a and lower area 400b. Upper area 400a corresponds to upper area 100a, and lower area 400b corresponds to lower area 100b. In addition, when paper 400 showing the automatic marking result is displayed, a marking completion button 400c is displayed in upper area 400a.

[0069] The automatic marking result paper 400 displays answer notes that correspond to the answer notes added by the user to the answer paper 300. In Fig. 9, three sticky notes 401, 402, and 403 are displayed. The contents displayed on the sticky notes 401, 402, and 403 are the same as the sticky notes added to the answer paper 300. However, unlike the answer sticky notes, the contents of the sticky notes 401, 402, and 403 cannot be edited.

[0070] Here, background colors are set for the sticky notes 401, 402, and 403. The background colors are set according to the results of automatic grading. As described above, when the answer matches the model answer, a "green" background color is set for the sticky note, when the answer does not match the model answer but is mathematically equivalent to the model answer, a "yellow" background color is set for the sticky note, and when the answer does not match the model answer and is not mathematically equivalent, a "red" background color is set for the sticky note. FIG. 9 shows an example in which the background color of sticky note 401 is "yellow," the background color of sticky note 402 is "green," and the background color of sticky note 403 is "red."

[0071] Furthermore, a marking criteria tag 404 is displayed on the automatic marking result paper 400. The marking criteria tag 404 is a list of evaluation items of the marking criteria for each question associated with the corresponding answer paper 300.

[0072] The grading criteria sticky note 404 includes a model answer 404a, evaluation items 404b, and a total score 404c. The model answer 404a is the "model answer" recorded in the grading criteria information for the corresponding question. The evaluation items 404b are texts representing each evaluation item of the "grading criteria" recorded in the grading criteria information for the corresponding question. The total score 404c is the total score for each question calculated as a result of automatic grading. The numerical value displayed in the total score 404c can be modified by the first user.

[0073] In the marking criteria sticky note 404, the model answer 404a is colored according to the result of automatic marking. For example, when the answer matches the model answer, the model answer 404a is colored "green". If the answer does not match the model answer but is mathematically equivalent to the model answer, the model answer 404a is colored "yellow". For example, if the answer matches an evaluation item other than the model answer, it is determined to be mathematically equivalent. If the answer does not match the model answer and is not mathematically equivalent, the model answer 404a is colored "red". In other words, the model answer 404a is colored the same as the background color of the sticky notes 401, 402, and 403. Note that the color scheme shown here is merely an example and can be changed as appropriate.

[0074] Here, we return to the description of FIG. 7. In step S204 after the paper 400 shown in FIG. 9 is displayed on the terminal 20a, the first user performs an operation to add a sticky note as necessary. After adding the sticky note, the first user writes a feedback comment or the like on the sticky note as necessary. In step S205, the terminal 20a transmits the sticky note information input by the first user to the server 10. The server 10 associates the sticky note information with the paper 400. The sticky note information includes, for example, information such as the position information of the sticky note on the paper 400 and the text written on the sticky note. In addition, in step S206, the first user modifies the total score 404c when the result of the automatic grading is not what the first user desires. In step S207, the terminal 20a transmits the information of the total score modified by the first user to the server 10. The server 10 modifies the information of the total score of the corresponding user associated with the paper 300. The processes of steps S204-S205 may be repeated until the first user selects the scoring complete button 400c in step S206.

[0075] The first user selects the marking completion button 400c in step S208, which is the timing when the marking result is as desired. In step S209, the terminal 20a transmits a marking completion notification to the server 10. This allows the server 10 to recognize the end of the process. Note that with the end of the process, the display of the paper 400 on the terminal 20a may also be terminated. Also, the paper 400 for which marking has been completed may be set so that it cannot be edited.

[0076] As described above, according to this embodiment, an answer submitted by a second user such as a student is automatically graded according to grading criteria information including predetermined evaluation items. Then, as a result of the automatic grading, depending on which evaluation item the answer matches, an answer that matches the model answer, an answer that does not match the model answer but is substantially equivalent to the model answer, and an answer that does not match the model answer and is not substantially equivalent are displayed in a distinguished manner. This allows a first user such as a teacher to intuitively recognize what grading criteria the automatic grading of each question was performed according to. In particular, questions colored in "yellow" are graded based on evaluation items created in advance based on the first user's grading philosophy, rather than on the absolute criterion of whether or not they match the model answer. Therefore, a first user such as a teacher may mainly focus on checking the questions colored in "yellow". In this way, the burden of the first user's visual confirmation of the automatic grading results and confirmation errors are reduced. As a result, the quality of automatic grading is improved.

[0077] [Variations] A modified example of the embodiment will be described below. In the embodiment, an example of automatic grading of answers to questions in mathematics is described. In contrast, the method of the embodiment can be applied to automatic grading of answers to various questions other than mathematics. FIG. 10 is a diagram showing an example of the display screen of the terminal 20a in a state in which a paper 500 of an automatic grading result of English is displayed. The screen displayed on the display device 26 of the terminal 20a also includes an upper area 500a and a lower area 500b. The upper area 500a corresponds to the upper area 100a, and the lower area 500b corresponds to the lower area 100b. In addition, when the paper 500 of the automatic grading result is displayed, a grading completion button 500c is displayed in the upper area 500a. These are the same as the paper 400.

[0078] The automatically graded result paper 500 displays answer notes that correspond to the answer notes added by the user to the answer paper 300. In Fig. 10, one sticky note 501 is displayed. This sticky note corresponds to the sticky notes 401, 402, and 403 on the paper 400. For example, Fig. 10 shows a question in which the questioner must write semantically correct words in the correct form in brackets.

[0079] The background color of the sticky note 501 is set according to the result of automatic grading. For example, when the answer matches the model answer, the background color "green" is set on the sticky note, when the answer matches the model answer semantically but there is a problem with the expression format, the background color "yellow" is set on the sticky note, and when the answer does not match the model answer semantically, the background color "red" is set on the sticky note. The expression format problem is an answer that is expected to have the same meaning as the model answer but contains an error, such as "words that should be written in lowercase contain capital letters," "verb forms are different," and "words contain spelling mistakes," as shown in FIG. 10.

[0080] In a modified example, the part of the evaluation item that matches the answer is colored. For example, in the example of FIG. 10, the word "assess" or "measure" is set as the model answer, but an example is shown in which the answer includes a spelling mistake such as "asessed" and is written in the past tense. In this case, the part of the evaluation item "-3 points if the verb form is different" and the part of the evaluation item "-1 point if there is a spelling mistake" are colored "yellow". By coloring in this way, the first user can more intuitively recognize what kind of scoring criteria was used for scoring. Such coloring for each evaluation item may be performed in the example of FIG. 9 described above. Also, when multiple evaluation items are colored as in FIG. 10, different colors may be used for these multiple evaluation items.

[0081] In the embodiment, the server 10 performs automatic grading according to the grading criteria created by the first user, and colors the answers according to the results of the automatic grading. Alternatively, the server 10 may perform automatic grading using, for example, an artificial intelligence that has learned the grading criteria created by the first user. In this case, the server 10 also distinguishes the answers on the display according to the results of the automatic grading.

[0082] In the embodiment, the background colors of the sticky notes are differentiated according to whether the answer matches the model answer, whether it does not match the model answer but is substantially equivalent, or whether it does not match the model answer but is not substantially equivalent. However, it is sufficient if the answer matches the model answer, whether it does not match the model answer but is substantially equivalent, or whether it does not match the model answer but is not substantially equivalent is displayed so that it can be visually distinguished by the first user. For example, the sticky note frame may be displayed in a different shape, color, or thickness.

[0083] In the embodiment, the background colors of the sticky notes are differentiated according to whether the answer matches the model answer, whether the answer does not match the model answer but is substantially equivalent, or whether the answer does not match the model answer but is not substantially equivalent. Alternatively, the background color of the sticky note may be differentiated according to whether the answer does not match the model answer but is substantially equivalent.

[0084] The present invention is not limited to the above-mentioned embodiment, and can be modified in various ways in the implementation stage without departing from the gist of the invention. In addition, each embodiment may be implemented in combination as appropriate as possible, and in that case, the combined effect can be obtained. Furthermore, the above-mentioned embodiment includes inventions at various stages, and various inventions can be extracted by appropriate combinations of the disclosed multiple constituent elements. For example, even if some constituent elements are deleted from all the constituent elements shown in the embodiment, if the problem described in the "Problem to be Solved by the Invention" column can be solved and the effect described in the "Effect of the Invention" column can be obtained, the configuration from which the constituent elements are deleted can be extracted as the invention.

[0085] The invention as described in the claims of the present application as originally filed is set forth below. [1] accepting input of information, including answers to questions; and Scoring the answer according to one or more evaluation items preset for the question; differentiating the results of the grading in accordance with the evaluation items that have been met for the answer; and An automatic scoring program for causing a processor to execute the above. [2] The evaluation items include model answers to the answers, The distinguishing in the indication includes distinguishing between when the answer matches the model answer, when the answer does not match but is equivalent to the model answer, and when the answer does not match or is not equivalent to the model answer. The automatic scoring program described in [1]. [3] The evaluation items include model answers to the answers, Distinguishing in the representation includes distinguishing only when the answer is equivalent but not identical to the model answer; The automatic scoring program described in [1]. [4] The marking includes a marking on the location where the answer is displayed. An automatic scoring program according to any one of [1]-[3]. [5] Such marking may include marking the location of the assessment item that is matched to the answer. An automatic scoring program according to any one of [1]-[4]. [6] The distinguishing step may include color-coding the results of the assessment according to the matched evaluation items. An automatic scoring program according to any one of [1]-[5]. [7] A server capable of communicating with a terminal, Accept input of information, including answers to questions; Score the answer according to one or more evaluation items preset for the question; Displaying and differentiating the results of the grading according to the evaluation items that matched the answers; Transmitting the result of the scoring to the terminal; A server comprising a processor configured to: [8] An automatic scoring method executed by an automatic scoring system, comprising: accepting input of information including answers to questions; Scoring the answer according to one or more evaluation items preset for the question; differentiating the results of the grading in accordance with the evaluation items that have been met for the answer; and An automatic scoring method comprising: [9] Accept input of information, including answers to questions; Score the answer according to one or more evaluation items preset for the question; Displaying and differentiating the results of the grading according to the evaluation items that matched the answers; Transmitting the results of said scoring; A server including a first processor configured as described above; receiving the results of the scoring transmitted from the server; displaying the received results of the scoring on a display device; a terminal including a second processor configured to An automatic scoring system having [Explanation of symbols]

[0086] 1 system, 10 server, 11 processor, 12 ROM, 13 RAM, 14 storage, 15 communication device, 19 system bus, 20a, 20b terminal, 21 CPU, 22 ROM, 23 RAM, 24 storage, 25 input device, 26 display device, 27 communication device, 29 system bus, 30 network.

Claims

1. accepting input of information including answers to questions; Grading the answer according to one or more evaluation items preset for the question; differentiating the results of the grading in accordance with the evaluation items that have been met for the answer; and An automatic scoring program for causing a processor to execute the above.

2. The evaluation items include model answers to the answers, The distinguishing in the indication includes distinguishing between when the answer matches the model answer, when the answer does not match but is equivalent to the model answer, and when the answer does not match or is not equivalent to the model answer. The automatic marking program according to claim 1.

3. The evaluation items include model answers to the answers, Distinguishing in the representation includes distinguishing only when the answer is equivalent but not identical to the model answer; The automatic marking program according to claim 1.

4. The marking includes marking a location where the answer is displayed.

4. The automatic marking program according to claim 1.

5. The marking includes marking the location of the assessment item that is matched with the answer.

5. The automatic marking program according to claim 1.

6. The distinguishing on the display includes color-coding the results of the scoring according to the matched evaluation items.

6. The automatic marking program according to claim 1.

7. A server capable of communicating with a terminal, Accept input of information, including answers to questions; Score the answer according to one or more evaluation items preset for the question; Displaying and differentiating the results of the grading according to the evaluation items that matched the answers; Transmitting the result of the scoring to the terminal; A server comprising a processor configured to:

8. An automatic scoring method executed by an automatic scoring system, comprising: accepting input of information including answers to questions; Grading the answer according to one or more evaluation items preset for the question; differentiating the results of the grading in accordance with the evaluation items that have been met for the answer; and An automatic scoring method comprising:

9. Accept input of information, including answers to questions; Score the answer according to one or more evaluation items preset for the question; Displaying and differentiating the results of the grading according to the evaluation items that matched the answers; Transmitting the results of said scoring; A server including a first processor configured as described above; receiving the results of the scoring transmitted from the server; displaying the received results of the scoring on a display device; a terminal including a second processor configured to An automatic scoring system having

Images