Scoring support method

Abstract

To provide a marking support method that supports a marker to grasp learning states by answerers in marking work even in the middle thereof.SOLUTION: In a marking support method, image data on a plurality of answer sheets having answers entered into answer fields of a first test to be marked and marking information associated with a second test having been marked as an object of comparison are stored; partial images of the respective answer sheets by answer regions set in the answer fields of the first test are arranged side by side and displayed; marking information is received individually by the partial images; by using the marking information with respect to the first test, the image data of marked answer sheets corresponding to the respective answer sheets are generated, respectively; the image data of the respective marked answer sheets associated with the first test are sorted and displayed based upon the marking information of the first test and the marking information of the second test.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present application relates to a marking support method. [Background technology]

[0002] Conventionally, methods for assisting graders in marking test questions that require test takers to enter characters such as kanji and numbers in answer boxes have been known. For example, Patent Documents 1 and 2 disclose marking support technologies for assisting graders in marking answers entered on paper answer sheets. In these methods, the answer sheets of each test taker are photographed, and images of the test takers' answers are displayed side by side on a screen for each question. Graders typically perform marking work by repeatedly marking (e.g., checking whether an answer is correct or incorrect) each of the test takers' answers that are displayed side by side on the screen in this manner for each question. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2020-027484 [Patent Document 2] Patent Publication No. 2021-182100 Summary of the Invention [Problem to be solved by the invention]

[0004] In conventional marking support technology, as a result of marking work by a marker, the marking information entered for each question is tallied and the marking result for each answerer is calculated (for example, the scores are tallied for each answerer). Then, image data of the marked answer sheet is output and stored in a file. A paper medium printed out from this is returned to the answerer as the marked answer sheet.

[0005] On the other hand, it was customary for graders to visually check the output and stored electronic file format or the printed graded answer sheets in order to properly grasp the learning situation, such as each respondent's level of understanding. Before finalizing the marks, graders often had to look at the entire graded answer sheet of each respondent and make final adjustments to the marks and point allocation, which was time-consuming.

[0006] In view of the above, the present disclosure aims to provide a scoring support technology that enables a grader to grasp the learning status of each answerer even while marking. In particular, a further aim is to provide a scoring support technology that enables a grader to preview the marked answer sheets while marking.

[0007] Another objective is to provide a scoring support technology that allows the scorer to effectively understand, during or after the scoring process, whether the guidance he or she provided from planning to implementing the test was appropriate, and how the guidance given to the test takers would have been more effective. [Means for solving the problem]

[0008] In order to solve such problems, the present disclosure provides a scoring support method, in which image data of multiple answer sheets, each with an answer written in an answer column for a question in a first test to be scored, for multiple answerers is stored in a storage unit, and the scoring support method includes the steps of: displaying partial images of each answer sheet in an order for each answer area set in the answer column for the first test, and individually accepting scoring information for each partial image; generating image data of a graded answer sheet corresponding to each answer sheet for each answerer using the scoring information; accepting specification of predetermined sorting conditions; determining a sorting order for the images of the graded answer sheets based on the scoring information and the sorting conditions; and displaying the images of the graded answer sheets in the order mentioned above.

[0009] The present disclosure also provides another scoring support method, in which correct answer information for each question in a test is stored in a storage unit, and the scoring support method includes the steps of: accepting answer information for each question from each answerer's terminal; using the correct answer information for each question to determine scoring information for the accepted answers to each question; generating image data of a graded answer sheet for each answerer using the answer information for each question and the scoring information; determining a sorting order for the images of the graded answer sheets for the test based on the scoring information; and displaying the images of the graded answer sheets in the order described above.

[0010] In these marking support methods, the step of displaying in the above order may include at least one of displaying images of multiple marked answer sheets individually in the above order, and displaying images of multiple marked answer sheets in a list arranged in order.

[0011] In these marking support methods, images of marked answer sheets that satisfy predetermined highlighting conditions may be highlighted in the images of the marked answer sheets displayed in a list.

[0012] According to the present disclosure, the grader can preview the graded answer sheets while the grader is marking the answers, and can effectively grasp the learning status of each answerer. [Brief explanation of the drawings]

[0013] [Figure 1] 1 is a schematic diagram of an exemplary hardware configuration of a scoring support device. [Figure 2] FIG. 2 is a functional block diagram of an example of a scoring support device. [Figure 3] FIG. 10 is an exemplary operational flow diagram executed by the external input / output device and the scoring support device. [Figure 4] FIG. 10 is an exemplary process flow diagram for a scoring process. [Figure 5] FIG. 10 is a diagram illustrating an example of a scoring information input screen in the scoring process. [Figure 6] FIG. 10 is a diagram illustrating an example comment input screen in the scoring process. [Figure 7] FIG. 10 is a diagram illustrating an example score tally screen in the scoring process. [Figure 8] FIG. 10 is a process flow diagram illustrating an example of a preview display process. [Figure 9] FIG. 10 is an exemplary process flow diagram of a score determination process. [Figure 10] FIG. 10 is a diagram illustrating an example sort setting screen in the preview display process. [Figure 11] FIG. 10 is a diagram showing an example page display screen of a score sheet in a preview display process. [Figure 12] FIG. 10 is a process flow diagram illustrating another example of the preview display process. [Figure 13] FIG. 10 is a diagram showing a thumbnail display screen of another example of a score sheet in the preview display process. [Figure 14] FIG. 10 is an exemplary operational flow diagram executed by the user terminal and the scoring support device. [Figure 15] FIG. 1 is a diagram of an exemplary computer-based test screen created in a web (World Wide Web) form. [Figure 16] FIG. 10 is a diagram of an exemplary scoring information entry screen in a computer-based test scoring process. [Figure 17] FIG. 10 is a diagram of an example score sheet page display screen in the preview display process of a computer-based test. DETAILED DESCRIPTION OF THE INVENTION

[0014] Hereinafter, embodiments of the scoring support method according to the present invention will be described with reference to the accompanying drawings. In the accompanying drawings, identical or similar elements are designated by identical or similar reference symbols, and duplicate descriptions of identical or similar elements may be omitted in the description of each embodiment. Furthermore, features shown in each embodiment may also be applied to other embodiments as long as they are not mutually inconsistent.

[0015] <<First Embodiment>> 1 is a schematic diagram of an exemplary hardware configuration of a scoring support device 1. The scoring support device 1 is used to execute the scoring support method according to this embodiment and can be realized by any computer. The scoring support device 1 may include a PC (Personal Computer), a tablet terminal, and a smartphone.

[0016] The scoring support device 1 includes a processor 2, a memory 3, a storage 4, an input / output interface 5, and a communication interface 6. These elements are electrically connected to each other via a bus 9. The scoring support device 1 may be realized as hardware using an LSI (Large Scale Integration), an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or the like.

[0017] The processor 2 is a computing device that executes various information processing and controls the transmission and reception of data between each element. The scoring support method according to this embodiment is executed by the processor 2. For example, the processor 2 is a CPU (Central Processing Unit) and / or a GPU (Graphical Processing Unit), and executes programs stored in the storage 4 and deployed in the memory 3 to execute information processing.

[0018] The memory 3 includes a main memory configured with a volatile storage device such as a DRAM (Dynamic Random Access Memory) and an auxiliary memory configured with a non-volatile storage device such as a flash memory or an HDD (Hard Disc Drive). The memory 3 is used as a work area for the processor 2, and stores a BIOS (Basic Input / Output System) that is executed when the OS (Operating System) is started, various setting information, etc.

[0019] The storage 4 stores a program, an OS, and the like for information processing by the processor 2. The storage 4 may also have a database built therein that stores various data used to execute the program.

[0020] The input / output interface 5 receives information input from information input devices such as a mouse and a keyboard, and outputs information to a screen such as a display. In the scoring support method according to this embodiment, data may also be input and / or output to / from an external input / output device (not shown in FIG. 1) such as a scanner and / or a printer via the input / output interface 5. The input / output interface 5 may also include a touch panel.

[0021] The communication interface 6 is connected to a network such as the Internet and / or an intranet via a wired and / or wireless connection, and communicates with external input / output devices. In this embodiment, external input / output devices such as the scanner and / or printer described above may be connected to the scoring support device 1 via the communication interface 6 in addition to or instead of the input / output interface 5. The communication interface 6 may include a short-range communication interface such as Bluetooth (registered trademark) and BLE (Bluetooth Low Energy).

[0022] The bus 9 interconnects each element including the processor 2, memory 3, storage 4, input / output interface 5, and communication interface 6, and transmits, for example, address signals, data signals, and control signals.

[0023] The above elements are merely examples, and the scoring support device 1 is not limited to these configurations. For example, in addition to the above elements, the scoring support device 1 may also include a camera such as a web camera, an in-camera, and / or an out-camera, and images and videos captured by the user may be stored in the storage 4.

[0024] Fig. 2 is a functional block diagram illustrating an example of functions implemented by the scoring support device 1 of Fig. 1. Specifically, the functions of the terminal support device 1 include an application unit 10 and a memory unit 30. The application unit 10 includes an area setting unit 11, a display unit 12, a scoring unit 13, a scoring information adding unit 14, a file output unit 15, and a preview unit 16. The memory unit 30 stores test information 31, student information 32, correct answer sheet image information 33, answer sheet image information 34, scoring information 35, and score sheet image information 36.

[0025] The area setting unit 11 of the application unit 10 sets a predetermined marking area from the captured image. For example, in the image of the answer sheet, an answer area in the answer column is set for each question. The answer area is preferably specified by the marker within a desired range of multiple coordinates, and may be specified for each question or for multiple questions. When marking, the image of the answer area corresponding to the range is cut out for each answer sheet.

[0026] The display unit 12 extracts partial images of the answer areas set by the area setting unit 11 from each of the images of the multiple answer sheets, and then displays these in a tiled manner. For example, for each answer area set by the area setting unit 11, an image of the correct answer sheet for that part and each image of the multiple answer sheets corresponding to it are extracted, and these are displayed in a tiled manner. In this specification, "displaying in a tiled manner" may mean, for example, arranging the image of the correct answer sheet and each image of the multiple answer sheets corresponding to it for the desired answer area on the screen of the scoring support device 1 and displaying them side by side in a list. In addition, the display unit 12 rearranges and displays the images of the graded answer sheets in an order according to predetermined conditions.

[0027] The marking unit 13 individually accepts input of marking information 35, such as correct / incorrect information, by a marker for a partial image of the answer sheet displayed in a tiled pattern for each answer area. The marking unit 13 also performs marking for each answer sheet based on the accepted marking information 35, and, for example, tallying up the data for each answer sheet to calculate data such as the score, ranking, average score, standard deviation, deviation value, percentage of correct answers, and percentage of correct answers for each question. Note that the input of marking information 35, such as correct / incorrect information, may be accepted automatically by a computer.

[0028] The marking information adding unit 14 associates and adds marking information 35 received by the marking unit 13 to the answer sheet image information 34. It also associates and adds data calculated in connection with the tallying as marking information 35. The marking information 35 may include image data of correct / incorrect information received for each answer area, and data of individual comments for each answer area. Alternatively, it may include data of the score for each answer sheet, data of commentary comments by the marking unit, and data of general commentary comments for all answerers. These comments may be text data or image data. The added marking information may be displayed on the display unit 12 in an editable format together with an image of the marked answer sheet.

[0029] The file output unit 15 generates image data of each answerer's graded answer sheet based on the answer sheet image information 34 and the marking information 35, and outputs it to a file for preview by the grader and / or for return to the answerer. Specifically, the file output unit 15 outputs image data of the graded answer sheet (i.e., the marking sheet) to which the marking information 35 received by the marking unit 13 has been added to the answer sheet image information 34. The image data of the marking sheet is stored in the memory unit 30 as marking sheet image information 36. The output file is then printed by the grader on a printer for return to the answerer.

[0030] In response to a request from a grader, the preview unit 16 sorts the images of the graded answer sheets output by the file output unit 15 in a sorting order determined according to predetermined conditions. In other words, the images of the graded answer sheets are rearranged in the sorting order and displayed individually on the display unit 12 for each graded answer sheet. In this specification, the display of the images of the graded answer sheets individually in a predetermined order for each graded answer sheet is referred to as "page display." In one example, the sorting order may be determined based on a comparison of the score information for a test to be graded with the score information for another test administered in the past for comparison, for multiple answerers. In addition, the default display order of the images of the graded answer sheets when no sorting is performed may usually be in attendance number order.

[0031] The preview unit 16 may also display a list of images of multiple graded answer sheets in thumbnail format, arranging them in sort order. In this specification, displaying a list of images of multiple graded answer sheets in thumbnail format, arranging them in a predetermined order, is referred to as "thumbnail display." In this case, among the images of multiple graded answer sheets displayed in the list, images of graded answer sheets that meet predetermined conditions may be highlighted (highlight display).

[0032] Test information 31 stored in storage unit 30 includes course information data indicating units and data on tests administered in the courses. Specifically, test information 31 may include course information (e.g., "third year of junior high school," "English"), theme (e.g., "relative pronouns"), type of test question (e.g., "grade-level test"), class (e.g., "class 1"), and test date and time. Test information 31 may also include point allocation information for administered test questions, data on templates of answer sheets for the test questions, and data on various setting information set for scoring support device 1, which may be modified.

[0033] The score information for test questions included in the test information 31 may be associated with point allocations for each aspect set by the grader (for example, "knowledge points," "skill points," "consideration points," "judgment points," "expression points," etc.). In this case, not only the total score but also the scores for each aspect will be tallied as a breakdown of the total score on the marking sheet as the test results. The test information 31 may also include information on tests administered in the past for each of the same units.

[0034] The student information 32 includes data on students taking the course (i.e., students who take the test administered in the course). Specifically, it may include basic information about the students (such as their name, class, and attendance number). After the test is administered, the student information 32 is matched with an image of the student extracted from the image of the answer sheet (for example, an image of the name written by the student).

[0035] The answer sheet image information 33 includes image data of an answer sheet on which a grader has written model answers in the answer columns of a template. The answer sheet image information 33 may also include image data of a commented answer sheet on which the grader has added marking information (for example, commentary and tally data) to the image data of the answer sheet.

[0036] The answer sheet image information 34 includes image data of the answer sheet on which the answerer has written the answers in the answer columns of the template. Note that the image data of the answer sheet is generated individually for each answerer.

[0037] The scoring information 35 includes score data input by the grader to the scoring support device 1, comment data input by the grader to the scoring support device 1, and aggregated data compiled as a result of the scoring. Specifically, the score data may include correct / incorrect information, aggregated scores, and scores by perspective. The comment data may also include text data, character strings (such as URLs (Uniform Resource Locators)), and image data (such as stamp images like "Like!" marks and two-dimensional barcodes). In addition, the scoring information 35 may include student's scoring information 35' for tests previously administered in the same course (unit) as historical information.

[0038] The marking sheet image information 36 includes image data of a marked answer sheet (i.e., a marking sheet) to which marking information by a marker has actually been added to the image data of each answer sheet. The image of the marked answer sheet is previewed by the preview unit 16.

[0039] Referring to FIG. 3, an exemplary overall operational flow executed by the marking support device 1 and the external input / output device 50 will be described. This operational flow begins when a marking system grader uses the external input / output device 50 to convert the answer sheet and multiple answer sheets into image data (step S51). For example, the external input / output device 50 is preferably a scanner capable of scanning paper to generate digital images. This generates image data of the answer sheet with the model answer written in the answer column and image data of multiple answer sheets for multiple answerers with their respective answers written in the answer column in image file format. It should be understood that the number of image files for the answer sheets generated is equal to the number of answerers. The image files may be in formats commonly used by those skilled in the art, such as JPEG, PNG, or BMP. They may also be in PDF or TIFF format.

[0040] The marking support device 1 stores the image data generated in step S51 in the storage unit 30 as answer sheet image information 33 and answer sheet image information 34 (step S12).

[0041] Next, the application unit 10 executes the marking process for the answer sheets using the answer sheet image information 33 and the answer sheet image information 34 (step S13, described below). As a result of step S13, marked answer sheet image data corresponding to each answer sheet is generated, output to a file, and temporarily stored in the storage unit 30 as marking sheet image information 36. However, the marked answer sheet image data here is provisional image data before the marks are finalized, and may be subsequently modified or edited by the marker. Alternatively, marking sheet image data with comments from the marker may be generated, output to a file, and stored in the storage unit 30 as the answer sheet image information 33. The image data file may be in any format, such as the aforementioned jpeg, png, bmp, pdf, or tiff format. Alternatively, it may be in a format compatible with a spreadsheet application (e.g., Excel) to facilitate editing by the marker.

[0042] When the marking process is executed in step S13, the application unit 10 displays images of the marked answer sheets page by page in response to a request from the marker (step S14 (preview display I) / described later). Here, the images of the marked answer sheets may be sorted in a desired order. Note that, in response to a request from the marker, it is possible to return to the marking process of the answer sheets in step S14 one by one (dashed arrow in FIG. 3). In other words, the marker can go back and forth between marking and checking by preview.

[0043] After step S14, the application unit 10 finalizes the marking, outputs image data of the marked answer sheet as a marked answer sheet image file, and stores it in the memory unit 30 as marking sheet image information 36 (step S15). The marking sheet image information 36 that was provisionally generated in the above-mentioned step S13 is updated and finalized at this timing. The answer sheet image information 33 is also finalized at this timing, and the answer sheet image file with comments is also output. The marked answer sheet image file and the answer sheet image file with comments are sent to the external input / output device 50.

[0044] After the marks are determined in step S15, the application unit 10 displays images of the marked answer sheets in a list as thumbnails in the desired sort order (step S16 (preview display II) / described later). Note that step S16 may be performed in preview display I as part of the above-mentioned step S14.

[0045] The external input / output device 50 prints the received image file of the marked answer sheet and the image file of the correct answer sheet (step S56). In other words, the external input / output device 50 is preferably, for example, a printer that can print image files on paper. Then, the answerer is returned a printed copy of the image file of his / her marked answer sheet and the image file of the correct answer sheet.

[0046] In this way, the marking support method according to this embodiment can efficiently mark collected paper answer sheets, and can improve the learner's learning efficiency through effective feedback from the marker.

[0047] Furthermore, in test marking activities conducted for the purpose of assessment for learning (integration of instruction and assessment), teachers (i.e., graders) typically grasp the learning comprehension status of learners (i.e., answerers) and, based on the information obtained, review and improve their own instruction. That is, by using the marking support method according to this embodiment, graded answer sheet images can be sorted in the order desired by the grader, and by displaying thumbnails in addition to page display, graders can effectively grasp the learning comprehension status of answerers. As a result, graders can effectively review and improve their instruction.

[0048] The scoring support method according to this embodiment will be described with reference to Figures 4 to 7. Figure 4 shows an example detailed processing flow relating to the scoring process of step S13 shown in Figure 3. Figures 5 to 7 show an example series of screen examples relating to the scoring process that are displayed on the screen of the scoring support device 1. Specifically, Figure 5 is an example of a scoring information input screen that accepts scoring operations by a grader, Figure 6 is an example of a comment input screen that accepts input of comments by a grader, and Figure 7 is an example of a score tally screen in which scoring results are tallied and listed.

[0049] In the marking process of step S13 shown in FIG. 4, first, the area setting unit 11 sets an item area for a predetermined item and an answer area for an answer column in the answer sheet image included in the answer sheet image information 33 and displayed on the display unit 12 (step S31). For example, the predetermined item is a name, and the area setting unit 11 sets a name area in the answer sheet image. The area setting unit 11 also sets an answer area in the answer column associated with one or more questions. The grader can determine the range of the area by, for example, dragging and dropping with a mouse on the screen of the marking support device 1. As a result, multiple coordinates defining the area are acquired, and each area of ​​an arbitrary shape is set.

[0050] Next, for the item areas among the areas set in step S31, the display unit 12 extracts images of the item areas in the corresponding answer sheet images contained in the answer sheet image information 34 and displays them on the screen in order to match the students and answerers (step S32). For example, an image of the name area is extracted from each answer sheet image. As a result, an image of the name written on the answer sheet by each answerer is displayed on the screen.

[0051] The grader can match the answerer on the answer sheet image with the student by comparing the image of each answerer's name with the name information included in the student information 32. This associates the student in the student information 32 with the answer sheet image.

[0052] Next, the display unit 12 extracts a partial image of the answer sheet and a partial image of each of the corresponding answer sheets for each answer area (i.e., for each question), and displays these images in a tiled pattern on the screen (step S33). Note that the grader may specify which answer areas are to be displayed in this tiled pattern.

[0053] Steps S31 to S33 are preparation stages for the marking process in step S13, and the actual marking process will be carried out from this point onwards.

[0054] The marking unit 13 accepts input of marking information for each partial image of the answer sheet displayed in tile form in step S33 (step S34). Marking information is, for example, correct / incorrect information for a question. That is, the marking person judges the validity of the answer for each answerer's answer sheet image displayed in tile form for a certain answer area, and inputs score data including correct / incorrect information one by one. The input score data is accepted as marking information 35 and stored in the memory unit 30.

[0055] FIG. 5 shows an example of a scoring information input screen that accepts the input of scoring information 35 individually for each partial image of the answer sheet displayed in tiled form. The grader is currently grading the "Group 1" test of "English_3rd Year_3rd Term: Grade Test_01." Here, the correct answer sheet image in the answer area corresponding to the answer column for "Question 1 / 32" (the first question) and each answer sheet image are displayed in tiled form in the tile display area TA. The correct answer sheet image is displayed in the correct answer area CA in the upper left of the tile area TA. Furthermore, in the answer areas AA of the tile area TA other than the correct answer area CA, the answer sheet images of each answerer are displayed side by side.

[0056] Specifically, the correct answer "A" is identified from the answer sheet image displayed in the correct answer area CA. Then, by visually comparing this with the answer sheet image displayed in the answer area AA, the grader can identify whether each answer sheet image displayed in the answer area AA is correct or incorrect.

[0057] More specifically, when the grader selects an answer sheet image in the answer area AA that displays "I" or "U" (i.e., an incorrect answer) and left-clicks with the mouse, an image of an "X" indicating an incorrect answer is superimposed. At the same time, the fact that the answer "I" or "U" is an incorrect answer is set in the scoring information 35 in association with the information on the test, question, and answerer. On the other hand, for answer sheet images for which no incorrect answers have been set, the correct answer is set in the scoring information 35. In this way, for example, by accepting checks only for incorrect answers, the scoring work by the grader can be made more efficient.

[0058] Returning to FIG. 4, after receiving the input of score data in step S34, the scoring unit 13 may further receive the input of additional information, such as commentary comments, for each tiled answer sheet image (step S35). The scoring information adding unit 14 adds the received information as scoring information 35. The information added here may include, for example, individual comment data from the grader to a specific answerer. In other words, the grader can set additional comment data for any answer area. The received scoring information 35 is stored in the memory unit 30.

[0059] In the example of the grading information input screen in Figure 5, among the answer sheet images in the answer area AA, those that already have comments entered have a circle mark cmm displayed in the upper left of the answer column. To enter a new individual comment, the grader can, for example, select a specific answer sheet image and right-click the mouse. This will display the action act dialog box for that answer sheet image. The grader can further display the comment input screen shown in Figure 6 by selecting "Enter comment" included in the action act.

[0060] Figure 6 is an example of a comment input screen displayed in response to the "Enter Comment" selection operation in Figure 5. The comment input screen is displayed overlaid on the scoring information input screen shown in Figure 5. The comment input screen includes a comment input field cm1 and an already entered comment selection field cm2. The grader can enter a comment in text format in the comment input field cm1. Furthermore, if the same comment has already been entered in another answer sheet image, that comment is aggregated as an already entered comment and displayed in the already entered comment selection field cm2. In this way, by dynamically aggregating comments that have already been entered as already entered comments, duplication of already entered comments is eliminated. When the grader "adds" the desired comment from the already entered comment selection field cm2, the already entered comment is reflected in the comment input field cm1.

[0061] Comment data entered in the comment input field and confirmed will be added to the image data of the answer sheet of the specific answerer currently being graded. This will cause the individual comments to be reflected on the graded answer sheet. If a comment is entered for the correct answer area CA, the comment will be added to the image data of the answer sheet. This will cause the overall comments to be reflected on the answer sheet.

[0062] The processes from step S33 to step S35 in Fig. 4 are repeatedly executed for all answer areas (i.e., all questions). This allows the grader to grade questions collectively, rather than grade answer sheets individually, thereby enabling efficient grading work.

[0063] In addition to the comment input screen shown in FIG. 6, there may be other comment input screens that allow the grader to input general comments for all contestants for each question. One example is a screen that allows the grader to input a URL or two-dimensional barcode that indicates the link information for an explanatory video related to a specific question as reference information. The comment data of the general comments entered here is accepted as the scoring information 35 and stored in the memory unit 30. The general comments are also added to the image data of the answer sheets of all contestants and reflected as overall advice.

[0064] 4 are repeated for all answer areas, the marking unit 13 then performs a process of tallying the marking results for all answer sheet images (step S36). For example, the scores of all answerers are tallied, and marking information such as the rank, correct answer rate, correct answer rate for each question, average score, standard deviation, and deviation value for all answerers may be calculated. These pieces of information also constitute part of the marking information 35.

[0065] Specifically, when the grader presses the "Go to scoring result list" button rbt on the scoring information input screen in Fig. 5, the display of the scoring support device 1 transitions to the scoring result list screen shown in Fig. 7. In this example of the scoring result list screen, a list of answerers, scores for each answerer by perspective ("knowledge points," "skill points," "consideration points," "judgment points," and "expression points"), total points ("front total" and "back total"), and the correct answer results for each question ("○" or "×") are displayed in a list.

[0066] Returning to Fig. 4 again, after the scoring results are tallied in step S36, the file output unit 15 uses the answer sheet image information 34 and the scoring information 35 to generate image data of a scored answer sheet corresponding to each answerer's answer sheet (step S37). In other words, image data of a scored answer sheet is generated for each of the answer sheet images, to which score data such as correct / incorrect information and commentary comments are added. Note that this image data of the scored answer sheets is provisional before the scoring is finalized, and is stored in the storage unit 30 as answer sheet image information 34 in an editable format.

[0067] For example, the data of such a provisional image of the graded answer sheet (not shown) may be image data in which images of "○" and "×" are added in red to the answer columns based on the correct / incorrect information received in step S34 and displayed superimposed. The "○" and "×" should be handwritten in red and converted into image data before being added to the answer sheet image. This makes it possible to create a graded answer sheet when the image of the graded answer sheet is printed, which looks as if the grader had actually graded the answers using a red pen.

[0068] The provisional image data of the graded answer sheet may also include additional information such as the commentary received in step S35. The output image file may be in any format, such as the aforementioned jpeg, png, bmp, pdf, or tiff format. Alternatively, it may be in a format compatible with a spreadsheet application (e.g., Excel) so that it can be easily edited by the grader.

[0069] In the same way as outputting the graded answer sheet image file, the marking unit 13 may generate answer sheet image data (not shown) with the grader's overall comments added to the answer sheet image through steps S35 and S36. In this case, the file output unit 15 outputs the answer sheet image file with the comments.

[0070] Reference will now be made to Figures 8 to 11. Among the marking support methods according to this embodiment, the following describes the process of using the generated image data of the marked answer sheets (step S13 in Figure 3 / step S37 in Figure 4) to individually display the images of the marked answer sheets of each answerer on a page (preview display I) and finalize the marks. This process may be executed as the process of steps S14 and S15 in Figure 3, or as an accompanying process.

[0071] Fig. 8 is an exemplary detailed processing flow relating to the preview display I processing shown in step S14 of Fig. 3. Fig. 9 is an exemplary detailed processing flow relating to the score determination processing shown in step S15 of Fig. 3. Fig. 10 is an example of a sorting setting screen that accepts sorting conditions in the preview display processing, and Fig. 11 is an example of a score sheet display screen that has been sorted according to the sorting conditions specified in Fig. 10 and displayed as individual pages. The sorting setting screen of Fig. 10 may be transitionable from, for example, the score tally screen of Fig. 7 or the score sheet display screen of Fig. 11, and the score sheet display screen of Fig. 11 may be transitionable from the sorting setting screen of Fig. 10. Either screen may be displayed superimposed on the screen before the transition.

[0072] The preview display I process shown in the process flow of Figure 8 is started by accepting a preview request from the grader after accepting input of correct / incorrect information for each question on all answer sheets. Specifically, it is started by accepting pressing of the "sort" button pv displayed on the score tally screen of Figure 7. Alternatively, it is started by accepting pressing of the sort icon srt displayed on the score sheet display screen of Figure 11.

[0073] First, the preview unit 16 receives input of predetermined sorting conditions from the grader (step S41). For example, as shown in the sorting setting screen of FIG. 10, the sorting condition set includes a sorting type, unit information such as the subject and test type, and a sorting order. Specifically, in FIG. 10, the sorting type can be selected from "standard" or "score difference," and "score difference" is selected here. Furthermore, with regard to the unit information, "English" is input as the subject and "grade test" is input as the test type. Furthermore, the sorting order can be selected from "ascending order" or "descending order," and "descending order" is selected here. In other words, in the example of FIG. 10, the sorting condition set specifies that the multiple graded answer sheet images are sorted in "descending order" based on the "score difference" between the "grade test" for "English" and the test to be graded by each answerer, using the "grade test" for "English" as the comparison target.

[0074] In addition, in cooperation with step S41, a process of accepting input of a predetermined filter condition (a process related to step S62 in FIG. 12) may be executed (described later).

[0075] Returning to FIG. 8, when "score difference" is specified as the sorting condition, the preview unit 16 identifies options for previously administered tests related to the current test being graded (i.e., tests to be compared) and further selects one of these as a comparison target (step S42). The test to be graded this time is "English_3rd Year_3rd Semester: Grade Test_01" (FIG. 5). In the example of FIG. 10, the options sel are presented: "English_3rd Year_2nd Semester: Grade Test 01" (midterm test) and "English_3rd Year_2nd Semester: Grade Test 02" (final test), which are associated with the same unit, "Grade Test" for "English." In the example of FIG. 10, furthermore, of the options sel, "English_3rd Year_2nd Semester: Grade Test 02" (final test), which was administered most recently in chronological order, is selected (the "Selected" display in FIG. 10).

[0076] Previously administered tests on the same unit to be compared can be selected manually by the grader, or the preview unit 16 can automatically select the most recently administered test on the same unit in chronological order and suggest it to the grader. Also, previously administered tests to be selected should be associated with the same unit. Because test takers have strengths and weaknesses in each unit, there is no reason to expand the selection options to include different units.

[0077] 8, in response to the selection of a previously administered test to be compared in step S42, the preview unit 16 acquires marking information 35' related to the test to be compared from the storage unit 30 (step S43). Note that the marking information 35' related to the previously administered test is included in advance as part of the marking information 35 as history information.

[0078] Next, the preview unit 16 determines the sort order for rearranging the graded answer sheet images based on the rearrangement conditions received in step S41, the score information 35 of the test to be graded, and the score information 35' of the test to be compared (step S44). Note that the score information 35 of the test to be graded has already been compiled in step S36 of FIG.

[0079] The sorting order should be determined based on the score difference between the test score information 35 of the test to be graded and the test score information 35' of the comparison test. In the example of FIG. 10, the sorting conditions specify "score difference" as the sorting type and "descending order" as the sorting order. In other words, it is determined that the image data of the graded answer sheets of the test to be graded will be sorted in order from "the respondent with the largest score difference compared to the most recent 'English_3rd year_2nd semester: grade test 02'" (final test).

[0080] In response to step S44, the preview unit 16 sorts the graded answer sheet images in the determined order, rearranging the graded answer sheet images and displaying them individually on the display unit 12 (step S45). On the grade sheet display screen of Fig. 11, the answer sheet images have been rearranged by comparison with the most recent "English_3rd year_2nd semester: Grade Test 02" (final test), and the graded answer sheet image of "Asakura Sota," whose score difference from the previous test was the largest, is displayed first. By pressing the right-facing triangular button (icon) btn shown in Fig. 11, the graded answer sheet images for each answerer are displayed in descending order of the score difference.

[0081] If the grader referring to the score sheet display screen in Fig. 11 wants to rearrange the results in a different display order, he or she can simply press the rearrange icon srt, which returns to step S41 in Fig. 8 and allows the grader to input new rearrangement conditions via the rearrangement setting screen in Fig. 10 (dashed arrow in Fig. 8).

[0082] By previewing the image of each answer sheet that has been graded by each answerer in an order determined by the difference in score from the previous test, graders can effectively grasp trends related to the test they are grading. The difference in score is calculated by subtracting the previous score from the current score, and is a positive or negative signed number. In particular, if the grader specifies "descending order" of the score difference (i.e., as shown in Figure 11, in descending order), the grader can first identify answerers with large positive score differences and significant improvement in their performance. Conversely, if the grader specifies "ascending order" of the score difference, the grader can first identify answerers with large negative score differences and significant decline in their performance.

[0083] In the above example, "score difference" is selected as the predetermined sorting condition, but the sorting condition may be simply specified according to the answerer's (student's) number (for example, student ID number or attendance number). Also, the condition may be specified so that the images of multiple graded answer sheets are sorted in order of total score (i.e., in "descending order" or "ascending order" of the score data included in the scoring information 35).

[0084] In addition to or instead of these, conditions may be specified to sort the images of multiple graded answer sheets in "descending" or "ascending" order based on the score data for each of the aforementioned viewpoints (for example, a breakdown of the total score, such as "knowledge scores," "skill scores," "discussion scores," "judgment scores," and "expression scores") included in the scoring information 35. In other words, conditions may be specified to sort the images of multiple graded answer sheets using only the score information for the test being graded, without using the score information for related tests that have already been performed for comparison as in step S42, and the sort order may be determined.

[0085] These sorting conditions are merely examples and are not limiting. Those skilled in the art will understand that two or more of the above may be combined, and any other sorting conditions may be employed, including the sorting conditions in step S61 of FIG. 12 (described later).

[0086] Furthermore, by dynamically switching between grasping test trends through such a preview (preview display I) and grading, graders can not only improve the efficiency of their grading operations but also further increase the accuracy of grading. For example, graders can grasp test trends from the perspective of, for example, score differences through the preview, and then return to grading each time to adjust the points assigned to specific questions and the scores. For example, for questions that were generally poorly performed throughout the test, graders can take measures such as providing partial credit and / or relaxing the grading criteria, and then re-perform the grading and tallying operations.

[0087] Following the preview display process of steps S41 to S45 shown in FIG. 8, the marking unit 13 and the file output unit 15 execute the marking confirmation process shown in FIG. 9 to finalize the marked answer sheet image data.

[0088] First, the marking unit 13 accepts adjustments by the marking maker to the marked answer sheet image displayed in the preview display process described above (step S51). Adjustments by the marking maker include reflecting new points that the marking maker notices through the preview as commentary on each marked answer sheet image. In other words, the marking maker can directly reflect commentary on each marked answer sheet image without returning to the comment input screen shown in FIG. 6.

[0089] Specifically, the scoring unit 13 can add the grader's comment data for each answerer and / or general comment data for all answerers directly to the image data of the graded answer sheet. In the scoring sheet display screen of Figure 11, a text image of "Pay attention to the difference between relative pronouns and relative adverbs" has been added to the first question. This is done by the grader entering text in the comment input field cm3 displayed on the right side of the screen and adjusting the placement of the text image. Also, in the scoring sheet display screen of Figure 11, a "Like!" stamp image sti has been added. This is done by the grader entering text through the stamp tab stt of the comment input field cm3 and adjusting the placement of the stamp image.

[0090] In response to the adjustment of the graded answer sheet image in step S51, the graded answer sheet image data is finalized. Then, the file output unit 15 outputs the graded answer sheet image data and stores it in the storage unit 30 as graded answer sheet image information 36 (step S52).

[0091] Through the marking confirmation process, the grader can dynamically make adjustments, such as adding commentary comments, while previewing the marked answer sheet. This allows the graded answer sheet image data to be generated more efficiently. In other words, the grader can not only mark the answer sheets effectively, but also mark and comment simultaneously, thereby improving the efficiency of the grader's work.

[0092] For example, if the preview results identify a respondent with a large positive score difference and a significant improvement in their grades, the grader can evaluate the respondent's improvement in grades and encourage them to study further. Conversely, if the preview results identify a respondent with a large negative score difference and a significant decline in grades, the grader can pay more attention to the respondent's learning attitude and learning comprehension status.

[0093] As described above, according to the marking support method of this embodiment, while a grader is marking a test, the grader can preview the marked answer sheets in order of importance, thereby effectively grasping the learning status of each answerer. In particular, for example, by rearranging and displaying marked answer sheet images through comparison with past tests of the same unit, the grader can effectively grasp trends related to the test being marked. Furthermore, by dynamically moving back and forth between the preview and marking operations, the grader's marking operation can be made more efficient and the accuracy of marking can be further improved.

[0094] Furthermore, test takers can receive effective feedback from the grader by having the printed image file of their own graded answer sheet returned to them, further improving their learning efficiency.

[0095] Reference will now be made to Figures 12 and 13. The following describes the process of the marking support method according to this embodiment, which uses the generated image data of the marked answer sheets (step S13 in Figure 3 / step S37 in Figure 4) to further display thumbnail images of the marked answer sheets of each answerer in a list (preview display II). Preview display II may be executed as the process of step S16 in Figure 3 or as ancillary to this. Note that preview display II may also be executed in step S14 in Figure 3 as part of the preview display I described above.

[0096] Preview Display II provides a function to sort the images of the graded answer sheets of answerers that need to be understood by the grader according to predetermined sorting conditions and display them in thumbnail format from the large number of graded answer sheet images created through step S52 of FIG. 9, etc. Preview Display II allows the grader to more effectively understand the answerers' learning comprehension status based on the test results that they themselves have graded. As a result, the grader can more effectively review and improve their guidance. Furthermore, by using Preview Display II in combination with Preview Display I, the grader can more effectively understand the answerers' learning comprehension status. Of course, only one of Preview Display I and Preview Display II may be implemented or used by the grader.

[0097] Fig. 12 is a detailed exemplary processing flow for the preview display II processing shown in step S16 of Fig. 3. Fig. 13 is an example of a score sheet display screen in which thumbnails are displayed after sorting according to the preview display II processing of Fig. 12. The preview display II processing is started, for example, by accepting pressing of a thumbnail display button thn provided on the score sheet display screen of Fig. 11, but is not limited to this.

[0098] First, the preview unit 16 receives input of predetermined sorting conditions from the grader (step S61). The sorting conditions may be the same as those described with respect to step S41 in FIG.

[0099] In one example, the sorting conditions may be input by the grader to specify the order in which the images of multiple graded answer sheets are sorted, as listed below. - The respondent's number (e.g., student ID number or attendance number). · Graded test scores for each respondent. -For graded tests, the scores for each respondent for each specific aspect. (For example, the subtotal of each respondent's score for questions related to one of the perspectives, such as "knowledge points," "skill points," "consideration points," "judgment points," and "expression points") The difference between the test score for each respondent and the test score for each respondent on the same unit selected chronologically (for example, the most recently administered test). The difference between the score for a particular aspect of the test being graded for each respondent and the score for that particular aspect for each respondent in the test of the same unit administered most recently in chronological order. The difference between the test score for each respondent and the moving average of the test scores for each respondent for multiple tests on the same subject administered over a given period according to time series. (The moving average here may be, for example, the average score for each respondent in multiple tests on the same subject administered within the last six months.) The difference between the score for a specific aspect of a graded test for each respondent and the moving average of the scores for that specific aspect for each respondent for multiple tests on the same subject administered over a specified period in chronological order.

[0100] The sorting conditions may be any one or more of the above, and in particular, may be a combination of two or more of them. It will be understood by those skilled in the art that the sorting conditions are not limited to these and may be any conditions designated by the grader.

[0101] Following step S61, the preview unit 16 receives input of predetermined filter conditions from the grader (step S62). According to the filter conditions, image data of graded answer sheets that match the conditions will be extracted.

[0102] In one example, the filter conditions may be specified by the grader to determine the image data of the multiple graded answer sheets to be extracted, such as the following: - Pre-designated respondent numbers (For example, filter conditions can be specified so that only specified respondents or their groups are displayed.) The results of multiple respondents' correct answers to specific questions selected according to the percentage of correct answers per question in a graded test. (For example, image data of the graded answer sheets of those who answered correctly (or incorrectly) to specific questions with a correct answer rate above (or below) a predetermined correct answer rate is extracted. Note that the correct answer results here have already been determined as score data in step S34 of FIG. 4, and the correct answer rate for each question has already been tallied as part of the score information 35 in step S36 of FIG. 4.)

[0103] It should be noted that the filter conditions may be any one or more of the above, and in particular, may be a combination of two or more of them. It should be understood by those skilled in the art that the filter conditions are not limited to these, and any filter conditions may be specified by the grader.

[0104] Next, the preview unit 16 determines the sort order for rearranging the graded answer sheet images based on either or both of the sorting conditions received in step S61 and the filter conditions received in step S62, and the score information 35 of the test to be graded (step S63). For example, it is preferable that the sort order be determined according to the sorting conditions specified in step S61, particularly for the images of the graded answer sheets extracted according to the filter conditions specified in step S62. Note that the score information 35 of the test to be graded has already been compiled in step S36 of FIG. 4.

[0105] Once the sorting order is determined in step S63, the preview unit 16 then selects image data of graded answer sheets that satisfy predetermined highlight conditions (step S64).

[0106] In one example, the highlighting conditions may be specified by the grader so that when images of the graded answer sheets are displayed in a list of thumbnails, items that deserve particular attention are highlighted, such as those listed below. An image of the graded answer sheet of a test taker whose ranking in the test to be graded has changed by a predetermined value or more compared to the ranking of that test taker in the most recent test of the same subject administered in chronological order. (Note that the ranking in the test to be graded has already been determined in step S36 of FIG. 4 as part of the grade information 35. Rankings in past tests are also stored in the memory unit 30 as part of the grade information 35'.) An image of the graded answer sheet of a respondent who answered correctly (or incorrectly) a specific question whose correct answer rate for the selected specific question in the graded test was higher (or lower) than the expected correct answer rate set in advance. (For example, if the percentage of correct answers to a particular question exceeds the expected percentage, the images of the graded answer sheets of the answerers who answered that question correctly are highlighted in blue. Conversely, if the percentage of correct answers to a particular question is lower than the expected percentage, the images of the graded answer sheets of the answerers who answered that question incorrectly are highlighted in red. Note that the percentage of correct answers for each question has already been tallied as part of the scoring information 35 in step S36 of FIG. 4.)

[0107] It should be noted that the highlighting conditions may be any one or more of the above, and in particular, may be a combination of two or more of them. It is understood by those skilled in the art that the highlighting conditions are not limited to these and may be any conditions designated by the grader.

[0108] Finally, the preview unit 16 displays the images of the multiple graded answer sheets as thumbnails in a list in the sort order determined in step S63 (step S65), highlighting the images of the graded answer sheets that satisfy a predetermined highlighting condition.

[0109] In the example screen display in Figure 13, images of the graded answer sheets of "Yamada Taro" (75 points), "Tanaka Jiro" (65 points), and "Nakayama Kazuo" (55 points) are arranged in a list and displayed as thumbnails. Here, the sort order of the graded answer sheet images is determined according to the score order (descending order). In addition, the image of the graded answer sheet of "Nakayama Kazuo" is highlighted with a bold frame hlt because it meets the highlighting conditions.

[0110] According to this embodiment, images of graded answer sheets can be sorted and displayed in a sorting order determined according to the desired conditions specified by the grader. The images on the screen can be displayed as individual pages for each graded answer sheet (preview display I), or multiple images of graded answer sheets can be displayed as thumbnails in a list (preview display II). In particular, in preview display II, images of graded answer sheets that meet predetermined conditions can be highlighted from the multiple images of graded answer sheets displayed in the list.

[0111] This allows graders to effectively grasp trends related to the test they are grading. Furthermore, by being able to dynamically switch between preview and grading, graders can further streamline their grading operations and further improve the accuracy of grading. Furthermore, during and after grading, graders can more effectively grasp the learner's understanding of the test answers based on the results of the tests they have graded, allowing graders to more effectively review and improve their instruction.

[0112] <<Modification of the First Embodiment>> 1) In the above description of the scoring support method, it is assumed that the scoring support device 1 is used as a stand-alone computer. Alternatively, the scoring support device 1 may be configured so that its functions are distributed between a server and a client, and they interact with each other via a network. Such an information processing system may also be implemented on a cloud basis. This eliminates the need to limit the scoring support device 1 to a specific terminal, thereby further improving the efficiency of the scoring work by the grader. Furthermore, instead of limiting the grader to one person, the grading work can be shared and performed by multiple graders.

[0113] 2) In addition, in the explanation of the scoring support method above, we assumed that a printed image file of the scored answer sheet would be returned to the answerer. Alternatively, the answer sheet could be returned to the answerer by email as electronic data or by storing it in a virtual folder on the cloud. This would eliminate the need to create a printed copy and allow for more efficient feedback to be provided to the answerer.

[0114] 3) Furthermore, in the above-described grading support method, the grader manually inputs correct / incorrect information during the grading process (e.g., step S34 in FIG. 4). Alternatively, an image recognition program may be implemented to automatically recognize the image of the answer area and determine whether the answer is correct or not. For example, a publicly known image recognition program based on a supervised learning model using artificial intelligence technology can be applied. This can further streamline the grading process performed by the grader.

[0115] 4) In addition, in the above description of the scoring support method, the test that has already been conducted and selected for comparison in the preview display process may be the test of the same unit that was conducted most recently in chronological order and automatically selected by the preview unit 16 (for example, step S42 in FIG. 8). Other methods for automatically selecting a test of the same unit that has already been conducted in the past may include the following.

[0116] In one example, the preview unit 16 may focus on the point allocation by perspective (in the above example, "knowledge points," "skill points," "consideration points," "judgment points," and "expression points") and automatically select from among tests on the same unit those with similar point allocations by perspective. In this case, the preview display may be rearranged and displayed in preview according to the score difference by perspective, not just the difference in the total score. In another example, the preview unit 16 may focus on the performance value (e.g., average score, standard deviation value, etc.) of the test to be graded, calculated using the aggregated scores of all test takers, and automatically select from among tests on the same unit that have been administered in the past those with similar performance values.

[0117] If the scores and performance values ​​for each aspect of the test are similar, it is expected that similar scoring results will be obtained for the test being graded. In other words, sorting the score differences in ascending or descending order and focusing on test takers with large positive or negative score differences is expected to effectively lead to an understanding of the learning status of each test taker and an understanding of overall trends.

[0118] 5) Furthermore, with regard to the preview display process, the above-described screen transitions are merely examples. In the above example, when the "sort" button pv is pressed on the score tally screen of FIG. 7, the screen transitions to the sort setting screen of FIG. 10. However, instead of transitioning to the sort setting screen of FIG. 10, the screen may first transition to the score sheet display screen of FIG. 11, and then transition to the sort setting screen of FIG. 10 only when the sort icon srt is pressed on the score sheet display screen of FIG. 11. In this way, by making the screen transitions more flexible, the scoring support method becomes even more user-friendly.

[0119] <<Second embodiment>> The first embodiment described above is based on the premise that the test to be graded is conducted on paper, and the answer sheets of the answerers are converted into image data using an external input / output device 50 (step S51 in FIG. 3). Specifically, the paper answer sheets are scanned using a scanner or the like to generate digital images such as PDF files, and image processing is performed on the digital images. Such paper-based tests are sometimes referred to as PBT (Paper Based Testing).

[0120] In contrast, the second embodiment assumes a computer-based test, as opposed to a paper-based test. Specifically, the second embodiment applies to a test conducted online via a network such as the Internet, in which test takers enter their answers using their own devices. Such computer-based tests are sometimes referred to as CBT (Computer Based Testing). CBT allows tests to be easily created and administered using web forms, and the marking process for answer sheets can be performed automatically. In other words, CBT is advantageous for graders because it reduces the time required to grade tests. On the other hand, the simplicity of CBT may make it more difficult for graders to grasp test takers' learning comprehension.

[0121] FIG. 14 shows an example of an operational flow executed between the user terminal 50a and the scoring support device 1a in this embodiment (corresponding to FIG. 3 relating to the first embodiment). FIG. 15 shows an example of a CBT test screen created in the form of a Web form. FIG. 16 shows an example of a scoring information input screen in the CBT scoring process. FIG. 17 shows an example of a scoring sheet page display screen in the CBT preview display process.

[0122] In this embodiment, as shown in FIG. 14, there is interaction between a user terminal 50a and the scoring support device 1a. The user terminal 50a and the scoring support device 1a are connected via a network such as the Internet. Information 31 on each question of the test to be scored, which is created in the form of a web form as shown in FIG. 15, and correct answer information 35 for each question are stored in advance in the storage unit 30. The scoring support device 1a is operated by an application unit (for convenience, referred to as "application unit 10a").

[0123] In this embodiment, the operational flow of Fig. 14 begins with multiple contestants inputting answer information to each question using their own user terminals 50a (step S51a). The contestants input answers to each question using an input device such as a keyboard in a web form such as that shown in Fig. 15.

[0124] In response to this, the marking support device 1a receives answer information for each question for each student from the user terminal 50a of each answerer via the network. Next, in response to this reception, the application unit 10a generates image data by rendering the correct answer sheet and the answer sheet for each answerer, as if the paper answer sheets were converted into image data (step S12a). Note that the generation of image data for the answer sheets may be performed collectively after receiving answer information from multiple answerers. Image data for the graded answer sheets is generated based on the image data of the rendered answer sheets.

[0125] Next, the application unit 10a uses the correct answer information for each question to determine the marking information for the answer to each question that has been received. Then, using the answer information and marking information for each question, image data of a marked answer sheet is generated for each solver (step S13a). In this embodiment, unlike step S13 in FIG. 3 relating to the first embodiment, the marking process can be executed automatically. That is, the application unit 10a can automatically execute the marking process by using computer processing to compare the correct answer information for each question that has been stored in advance in the storage unit with the answer information for each question that has been received for each solver.

[0126] As a result, for example, a marking information input screen such as that shown in Fig. 16 is automatically generated for each question. That is, in this embodiment, the marking process is automated, which is advantageous in that it does not require the marking person to do anything. The marking person can efficiently generate marked answer sheet image data by inputting comments to the marking information input screen automatically generated for each question, as explained in step S13 of Fig. 4 (particularly step S35 of Fig. 5, and Figs. 5 and 6) in relation to the first embodiment.

[0127] 17, the image of the graded answer sheet generated as a result of step S13a is preferably an image of the answer sheet rendered in step S12a to which the result of the correct answer for each question has been added. For example, the result of the correct answer includes an image of "○" or "×". In this way, the evaluated correct answer sheet and the graded answer sheet for each answerer are rendered and image data is generated as if the paper answer sheet had been converted into image data.

[0128] That is, in this embodiment, the image of the graded answer sheet in CBT looks as if it were a paper answer sheet converted into image data, that is, an image of the graded answer sheet similar to that in PBT is generated. This makes it possible to apply the preview displays I and II described in the first embodiment to this embodiment as well, thereby enabling flexible and efficient grading work and effective understanding of the learner's learning comprehension status.

[0129] Returning to Fig. 14, when the marking process is executed in step S13a, the application unit 10a determines the sort order of the images of the multiple marked answer sheets for the test to be marked based on the marking information. Accordingly, in response to a request from the marker, the images of the marked answer sheets are individually displayed page by page in the determined sort order (step S14a (preview display I')). Note that step S14a may be similar to step S14 (steps S41 to S45) in the first embodiment. In particular, the algorithm for rearranging the images of the marked answer sheets when displaying the pages in preview display I' may be similar to that described for preview display I in the first embodiment.

[0130] An example of a page display of images of multiple graded answer sheets is shown in Figure 17, where images of the graded answer sheets for "Yamada Taro" (75 points), "Tanaka Jiro" (65 points), and "Nakayama Kazuo" (55 points) are individually sorted in descending order of score and displayed.

[0131] After step S14a, the application unit 10a finalizes the marking, outputs image data of the marked answer sheet as a marked answer sheet image file, and stores it in the storage unit 30 as marking sheet image information 36 (step S15a). Also, the answer sheet image information 33 is finalized at this time, and the answer sheet image file with comments is also output.

[0132] The image file of the graded answer sheet and the image file of the correct answer sheet with comments are returned to the user terminal 50a and displayed on the screen of the user terminal 50a (step S56). That is, in step S56a, the image of the answer sheet with the correct answers added is returned to the answerer as image data, just as if the paper answer sheet had been converted into image data.

[0133] After the marking is finalized in step S15a, the application unit 10a can display images of the marked answer sheets collectively as thumbnails in a list in a desired sort order (step S16a (preview display II')). Step S16a may be performed in preview display I' as part of the above-mentioned step S14. Step S16a may be similar to step S16 (steps S61 to S65) in the first embodiment. In particular, the algorithm for sorting and filtering the images of the marked answer sheets may be similar to that described for preview display II in the first embodiment.

[0134] Furthermore, as explained in the first embodiment with respect to steps S64 and S65 in FIG. 12, it is preferable that, among the images of multiple graded answer sheets displayed as thumbnails in a list, images of graded answer sheets that satisfy predetermined highlighting conditions are highlighted.

[0135] An example of the thumbnail display in step S16a may be similar to that shown in Fig. 13 for the first embodiment. Here, images of the graded answer sheets of "Yamada Taro" (75 points), "Tanaka Jiro" (65 points), and "Nakayama Kazuo" (55 points) are displayed as thumbnails in a list in descending order of score. In addition, the image of the graded answer sheet of "Nakayama Kazuo" that meets the highlighting conditions is highlighted with a bold frame hlt.

[0136] In this way, in this embodiment, the image of the graded answer sheet appears as if it were a paper answer sheet converted into image data, and thus the same scoring support method can be applied to CBT as to PBT. In other words, the preview displays I and II described in the first embodiment can be applied in the same way even when a test is conducted by CBT. In other words, this enables flexible and efficient scoring work and effective understanding of the answerers' learning and understanding status, and allows graders to more effectively understand the answerers' learning and understanding status.

[0137] It is also possible to administer both a paper-based test (PBT) and a computer-based test (CBT) for one test, and apply a combination of the marking support method according to the first embodiment and the marking support method according to the second embodiment to each of them. This allows for even more flexible and efficient marking work and effective understanding of the learners' learning comprehension status, allowing the markers to understand the learners' learning comprehension status even more effectively.

[0138] At least the following technical ideas can be understood from the above-described embodiment.

[0139] [Mode 1] According to Mode 1, there is provided a scoring support method. In this scoring support method, image data of a plurality of answer sheets, on which answers are written in answer columns for questions in a first test to be scored for a plurality of answerers, is stored in a storage unit, and this scoring support method includes the steps of, for the first test, displaying partial images of each answer sheet in an order for each answer area set in the answer column and individually accepting scoring information for each partial image, generating image data of a graded answer sheet corresponding to each answer sheet for each answerer using the scoring information, accepting specification of predetermined sorting conditions, determining the sorting order of the images of the graded answer sheets based on the scoring information and the sorting conditions, and displaying the images of the graded answer sheets in the order mentioned above.

[0140] [Form 2] According to Form 2, in the marking support method of Form 1, the step of displaying in the above order includes at least one of displaying images of multiple marked answer sheets individually in the above order, and displaying images of multiple marked answer sheets in a list arranged in the above order.

[0141] [Form 3] According to Form 3, in the scoring support method of Form 1 or 2, the scoring information for a second test that has already been scored and is to be compared with the first test for a plurality of respondents is stored in a memory unit, the second test is selected in chronological order from those associated with the same unit as the first test, and the above order is determined based on the scoring information for the first test and the scoring information for the second test.

[0142] [Form 4] According to form 4, in the scoring support method of form 3, the scoring information for the first test and the scoring information for the second test each have a score for each answerer, and the above order is determined based on the score difference between the score for the first test and the score for the second test for each answerer.

[0143] [Form 5] According to form 5, in the scoring support method of form 4, the score difference is the score difference between the score of the first test and the moving average of the scores of multiple second tests administered over a predetermined period in chronological order.

[0144] [Mode 6] According to mode 6, in the marking support method of mode 1 or 2, the step of determining the order is applied to the image of the marked answer sheet extracted according to a predetermined filter condition.

[0145] [Form 7] According to Form 7, in the scoring support method of Form 6, the scoring information for the first test includes the correct answers of multiple respondents for each question in the first test, and the scoring support method further includes a step of determining the correct answer rate for each question in the first test according to the scoring information for the first test, and the filter condition is associated with the correct answer results of multiple respondents for specific questions selected according to the correct answer rate.

[0146] [Eighth Mode] According to an eighth mode, in the marking support method of the second mode, images of the marked answer sheets that satisfy a predetermined highlighting condition are highlighted in the images of the marked answer sheets displayed in a list.

[0147] [Form 9] According to Form 9, in the scoring support method of Form 8, for a plurality of contestants, scoring information for a scored second test to be compared with a first test is stored in a memory unit, the scoring information for the second test includes the rankings of the plurality of contestants, the second tests are selected in chronological order from those associated with the same unit as the first test, the scoring support method further includes a step of determining the rankings of the plurality of contestants based on the scoring information for the first test, and the highlighting condition is associated with the change in ranking between the first test and the second test.

[0148] [Form 10] According to form 10, in the scoring support method of form 8 or 9, the scoring information for the first test includes the results of correct answers given by multiple respondents for each question in the first test, and the scoring support method further includes a step of determining the correct answer rate for each question in the first test in accordance with the scoring information for the first test, and the highlight conditions are associated with the correct answer results of the multiple respondents based on the correct answer rate for the selected specific question and a predetermined expected correct answer rate.

[0149] [Mode 11] According to mode 11, there is provided a scoring support method, in which correct answer information for each question in a test is stored in a storage unit, and the scoring support method includes the steps of: accepting answer information for each question from a terminal of each answerer; determining scoring information for the accepted answers to each question using the correct answer information for each question; generating image data of a graded answer sheet for each answerer using the answer information and the scoring information for each question; determining a sorting order for the images of the graded answer sheets for the test based on the scoring information; and displaying the images of the graded answer sheets in the order mentioned above.

[0150] [Form 12] According to form 12, in the marking support method of form 11, the step of displaying in the above order includes at least one of displaying images of multiple marked answer sheets individually in the above order, and displaying images of multiple marked answer sheets in a list arranged in the above order.

[0151] [Mode 13] According to mode 13, in the marking support method of mode 11 or 12, images of marked answer sheets that satisfy a predetermined highlighting condition are highlighted from among the images of a plurality of marked answer sheets displayed in a list.

[0152] The scoring support method of the present disclosure has been described above, but the technical scope of the present disclosure is not limited to the scope of the above description. It will be apparent to those skilled in the art that various modifications and improvements can be made to the above description. It is clear from the claims that such modifications may also be included in the technical scope of the present disclosure. [Explanation of symbols]

[0153] 1... Marking support device, 2... Processor, 3... Memory, 4... Storage, 5... Input / output interface, 6... Communication interface, 10... Application unit, 11... Area setting unit, 12... Display unit, 13... Marking unit, 14... Marking information addition unit, 15... File output unit, 16... Preview unit, 30... Memory unit, 31... Test information, 32... Student information, 33... Answer sheet image information, 34... Answer sheet image information, 35, 35'... Marking information, 36... Marking sheet image information, 50... External input / output device, 50a... User terminal

Claims

1. A marking support method, comprising: storing image data of a plurality of answer sheets, each of which has an answer written in an answer column for a question in a first test to be marked, for a plurality of test takers; and a step of displaying partial images of each of the answer sheets side by side for each answer area set in the answer column, and receiving marking information for each of the partial images individually; generating image data of a marked answer sheet corresponding to each answer sheet for each answerer using the marking information; A step of accepting a designation of a predetermined sorting condition; determining a sorting order for the images of the marked answer sheets based on the marking information and the sorting conditions; displaying the images of the marked answer sheets in the order; Including, score information for a second test that has already been scored and is to be compared with the first test for the plurality of test takers is stored in the storage unit; The second test is selected in chronological order from among tests associated with the same unit as the first test; A scoring support method, wherein the order is determined based on scoring information for the first test and scoring information for the second test.

2. The scoring support method according to claim 1, The step of displaying in the order individually displaying the images of the plurality of marked answer sheets in the order; and Displaying the images of the plurality of marked answer sheets in a list in the order; A scoring support method including at least one of the above.

3. The scoring support method according to claim 1 or 2, the first test score information and the second test score information each include a score for each of the answerers; A scoring support method, wherein the order is determined based on the difference in score between the score of the first test and the score of the second test for each of the answerers.

4. The scoring support method according to claim 3, A scoring support method, wherein the score difference is the score difference between the score of the first test and a moving average of the scores of multiple second tests administered over a predetermined period according to the time series.

5. A scoring support method, comprising: storing image data of a plurality of answer sheets, each of which has an answer written in an answer column for a question in a first test to be scored, for a plurality of test takers; and the scoring support method, for the first test, a step of displaying partial images of each of the answer sheets side by side for each answer area set in the answer column, and receiving marking information for each of the partial images individually; generating image data of a marked answer sheet corresponding to each answer sheet for each answerer using the marking information; A step of accepting a designation of a predetermined sorting condition; determining a sorting order for the images of the marked answer sheets based on the marking information and the sorting conditions; displaying the images of the marked answer sheets in the order; Including, A marking support method, wherein the step of determining the order is applied to the images of the marked answer sheets extracted according to predetermined filter conditions.

6. In the scoring support method according to claim 5, The step of displaying in the order individually displaying the images of the plurality of marked answer sheets in the order; and Displaying the images of the plurality of marked answer sheets in a list in the order; A scoring support method including at least one of the above.

7. 7. The scoring support method according to claim 5, the score information of the first test includes the results of correct answers of the plurality of test takers for each question of the first test, The scoring support method further includes a step of determining a correct answer rate for each question of the first test according to the scoring information of the first test, A scoring support method, wherein the filter condition is associated with the results of the correct answers of the multiple answerers for a specific question selected according to the correct answer rate.

8. A scoring support method, wherein image data of a plurality of answer sheets on which answers are written in answer columns for questions in a first test to be scored for a plurality of test takers is stored in a memory unit, and the scoring support method, with respect to the first test, a step of displaying partial images of each of the answer sheets side by side for each answer area set in the answer column, and receiving marking information for each of the partial images individually; generating image data of a marked answer sheet corresponding to each answer sheet for each answerer using the marking information; A step of accepting a designation of a predetermined sorting condition; determining a sorting order for the images of the marked answer sheets based on the marking information and the sorting conditions; displaying the images of the marked answer sheets in the order; Including, The step of displaying in the order individually displaying the images of the plurality of marked answer sheets in the order; and Displaying the images of the plurality of marked answer sheets in a list in the order; and A marking support method in which, among the images of the plurality of marked answer sheets displayed in the list, images of marked answer sheets that satisfy predetermined highlighting conditions are highlighted.

9. The scoring support method according to claim 8, score information for a second test that has been scored and is to be compared with the first test for the plurality of contestants is stored in the storage unit, and the score information for the second test includes rankings of the plurality of contestants; The second test is selected in chronological order from among tests associated with the same unit as the first test; The scoring support method further includes a step of determining a ranking of the plurality of answerers based on the scoring information of the first test, The scoring support method, wherein the highlighting condition is associated with a change in ranking between the first test and the second test.

10. The scoring support method according to claim 8 or 9, The scoring information of the first test includes correct answers of the plurality of answerers for each question of the first test, and the scoring support method further comprises: determining a correct answer rate for each question of the first test according to the score information of the first test; A scoring support method in which the highlighting conditions are associated with the correct answer results of the multiple answerers based on the correct answer rate for the selected specific question and a predetermined expected correct answer rate.

Images