Automatic rating device, program, and system

Abstract

To provide an automatic rating device, a program, and a system that can further improve the degree of freedom in rating in relation to a question involving drawing a figure.SOLUTION: An automatic rating device 18 executes acquisition processing of acquiring answer data D2 including stroke data 80 that indicates an answer to a question involving drawing a figure, and that indicates collection of strokes formed through stroke operations using an electronic pen 14, and use data 82 that indicates a use action of a stationery 16 to assist drawing of a figure; and rating processing of calculating the score of the answer on the basis of the answer data D2 acquired through the acquisition processing.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] The present invention relates to an automatic scoring device, a program, and a system. [Background technology]

[0002] Automatic marking technology has been known for some time, in which answer data showing answers to given questions is analyzed and the answers are automatically marked. In particular, the efficiency of marking is expected to improve by "digitalization," in which the person being marked writes the answer by hand using an electronic pen.

[0003] Patent Document 1 discloses an automatic scoring method in which an input pattern and a standard pattern expressed by coordinate information of the strokes of the character to be scored and the sub-strokes that make up the strokes are given, and the input pattern is evaluated for correctness through comparison with the standard pattern. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Publication No. 2017-037125 Summary of the Invention [Problem to be solved by the invention]

[0005] The types of problems vary widely depending on the subject. For arithmetic or mathematics, one example of a problem is a "geometry problem" that requires students to draw using stationery other than the electronic pen (for example, a compass or straightedge).

[0006] However, according to the automatic marking method of Patent Document 1, since marks are given based on the result figure as an answer, it may be difficult to evaluate the level of understanding of the person being marked in stages. For example, even if the person being marked uses stationery correctly in the process of creating the answer, this is not reflected in the marking result because it is not explicitly expressed in the result figure.

[0007] The present invention has been made in consideration of the above-mentioned problems, and its purpose is to provide an automatic marking device, program, and system that can further improve the degree of freedom in marking questions that involve drawing. [Means for solving the problem]

[0008] An automatic scoring device in a first aspect of the present invention includes one or more processors, and the one or more processors execute an acquisition process to acquire answer data indicating an answer to a question involving drawing, the answer data including stroke data indicating a collection of strokes formed through stroke operations using an electronic pen and usage data indicating stationery usage actions to assist in the drawing, and a scoring process to calculate a score for the answer based on the answer data acquired through the acquisition process.

[0009] The automatic marking program in the second aspect of the present invention causes one or more processors to execute an acquisition process to acquire answer data indicating an answer to a question involving drawing, the answer data including stroke data indicating a collection of strokes formed through stroke operations using an electronic pen and usage data indicating stationery usage actions to assist in the drawing, and a marking process to calculate a score for the answer based on the answer data acquired through the acquisition process.

[0010] The automatic marking system in a third aspect of the present invention comprises an electronic pen, stationery for assisting in drawing with the electronic pen, a terminal of a person to be graded that generates answer data indicating an answer to a question that involves drawing through operation of the electronic pen and the stationery, and an automatic marking device configured to be able to communicate with the terminal of the person to be graded, the automatic marking device executing an acquisition process for acquiring the answer data including stroke data indicating a collection of strokes formed through stroke operation using the electronic pen and usage data indicating usage actions of the stationery, and a marking process for calculating a score for the answer based on the answer data acquired through the acquisition process. [Effects of the Invention]

[0011] According to the present invention, it is possible to increase the degree of freedom in scoring questions that involve drawing. [Brief explanation of the drawings]

[0012] [Figure 1] 1 is a diagram illustrating the overall configuration of an automatic scoring system according to an embodiment of the present invention. [Figure 2] FIG. 2 is a diagram showing a first example of the stationery item in FIG. [Figure 3] FIG. 2 is a diagram showing a second example of the stationery item in FIG. [Figure 4] FIG. 2 is a diagram showing an example of the configuration of the automatic marking device shown in FIG. [Figure 5] FIG. 5 is a functional block diagram of the scoring processing unit shown in FIG. 4. [Figure 6] FIG. 2 is a sequence diagram relating to the operation of the automatic marking system shown in FIG. [Figure 7] FIG. 10 is a diagram showing an example of a question screen before an answer is given. [Figure 8] FIG. 10 is a diagram showing an example of a question screen after an answer has been given. [Figure 9] 7 is a detailed flowchart relating to step SP30 in FIG. 6. [Figure 10] FIG. 10 is a diagram illustrating an example of a data structure of answer data. [Figure 11]FIG. 6 is a diagram showing an example of a decision tree structure specified by the point allocation information of FIG. 5. [Figure 12] FIG. 6 is a first diagram showing an example of a decision tree structure specified by the detection condition information of FIG. 5. [Figure 13] FIG. 6 is a second diagram showing an example of a decision tree structure specified by the detection condition information of FIG. 5. [Figure 14] FIG. 6 is a third diagram showing an example of a decision tree structure specified by the detection condition information of FIG. 5. [Figure 15] FIG. 10 is a functional block diagram relating to a scoring processing unit in the first modified example. [Figure 16] 16 is a diagram showing an example of a data structure of the adjustment information of FIG. 15. FIG. [Figure 17] FIG. 10 is a diagram showing an example of a question screen in the second modified example. [Figure 18] FIG. 5 is another functional block diagram of the scoring processing unit shown in FIG. [Figure 19] FIG. 11 is a diagram showing a first resultant graphic in the third modified example. [Figure 20] FIG. 10 is a diagram showing a second resultant graphic in the third modified example. DETAILED DESCRIPTION OF THE INVENTION

[0013] The automatic scoring device, program, and system of the present invention will be described with reference to the accompanying drawings. To facilitate understanding of the description, the same components in each drawing are denoted by the same reference numerals whenever possible, and duplicate descriptions will be omitted. Furthermore, the term "part" may be replaced with other terms such as unit, module, device, or element.

[0014] [Explanation of Automated Scoring System 10] <Overall structure> FIG. 1 is a diagram illustrating the overall configuration of an automatic marking system 10 according to one embodiment of the present invention. The automatic marking system 10 is provided to provide an "automatic marking service" for automatically marking answer sheets prepared by students. This marking is performed for various purposes, including confirming the effectiveness of learning, assessing aptitude, or assessing ability. The following description will be given taking as an example a case in which the students (e.g., junior high school students) are working on mathematical geometry problems (i.e., output learning).

[0015] Specifically, the automatic grading system 10 comprises one or more learner terminals 12 (corresponding to "gradee terminals"), one or more electronic pens 14, one or more stationery items 16, and an automatic grading device 18. Each learner terminal 12 is connected to the automatic grading device 18 via a network NT so as to be able to communicate with the automatic grading device 18.

[0016] The learner terminal 12 is a computer owned by a user (here, a learner) who uses the automatic marking service, and is configured as a tablet, smartphone, personal computer, wearable device, etc. Specifically, the learner terminal 12 is configured to include a processor 21, a memory 22, a communication unit 23, a display unit 24, a touch sensor 25, and an EMR (Electro-Magnetic Resonance) sensor 26.

[0017] The processor 21 is configured with an arithmetic processing device including a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), and an MPU (Micro-Processing Unit). The processor 21 reads out programs and data stored in the memory 22, and performs a generation process to generate ink data (hereinafter also referred to as digital ink) that describes handwritten content, a rendering process to display content indicated by the digital ink, and the like.

[0018] The memory 22 stores programs and data necessary for the processor 21 to control each component. The memory 22 is non-transitory and is composed of a computer-readable storage medium. Here, the computer-readable storage medium is composed of [1] a storage device such as a hard disk drive (HDD) or solid state drive (SSD) built into the computer system, or [2] a portable medium such as a magneto-optical disk, a read-only memory (ROM), a compact disc (CD)-ROM, or a flash memory.

[0019] The communication unit 23 has a communication function for performing wired or wireless communication with an external device, allowing the learner terminal 12 to exchange various data, including question data D1, answer data D2, and marking result data D3, which will be described later in FIG. 2, with the automatic marking device 18, for example.

[0020] The display unit 24 is capable of visibly displaying content including images or videos, and is configured, for example, with a liquid crystal panel, an organic EL (Electro-Luminescence) panel, or electronic paper. By making the display unit 24 flexible, the learner can perform various handwriting operations while keeping the touch surface of the learner terminal 12 curved or bent.

[0021] The touch sensor 25 is a capacitance-type sensor having a plurality of sensor electrodes arranged in a plane. The touch sensor 25 includes, for example, a plurality of X-line electrodes for detecting the position of the X axis of a sensor coordinate system and a plurality of Y-line electrodes for detecting the position of the Y axis. Note that the touch sensor 25 may be a self-capacitance-type sensor in which block-shaped electrodes are arranged in a two-dimensional lattice pattern, instead of the mutual capacitance-type sensor described above.

[0022] The EMR sensor 26 is an electromagnetic induction type (hereinafter also referred to as "EMR type") planar sensor that receives an induction signal generated by the electronic pen 14 or the stationery 16 under the action of an alternating magnetic field from a plurality of detection coils arranged in a planar shape. In the example of Fig. 1, the EMR sensor 26 is provided separately from other components, but a part or all of the EMR sensor 26 may be provided integrally with other components (for example, the display unit 24).

[0023] The electronic pen 14 is a pen-type pointing device configured to be capable of one-way or two-way communication with the learner terminal 12. The electronic pen 14 is, for example, an active electrostatic coupling type (AES type) or electromagnetic induction type (EMR type) stylus. A learner can write pictures, characters, etc. on the learner terminal 12 by holding the electronic pen 14 and moving the pen tip while pressing it against the touch surface of the learner terminal 12.

[0024] The stationery 16 is a tool to assist in drawing using the electronic pen 14. Examples of the stationery 16 include a compass 16a, a straightedge 16b, a ruler 16c, set squares 16d and 16e, and a protractor 16f. The configuration of the stationery 16 will be described in detail with reference to FIGS. 2 and 3.

[0025] The automatic marking device 18 is a server computer that performs overall control over the marking of the answer data D2, and may be either a cloud-based or on-premise type. Here, the automatic marking device 18 is illustrated as a single computer, but the automatic marking device 18 may instead be a group of computers that form a distributed system.

[0026] FIG. 2 is a diagram showing a first example of the stationery 16 in FIG. 1. The compass 16a includes two legs 30, 31, a central mechanism 32, a fixed portion 33, an indicator 34, and a pen portion 35. The central mechanism 32 is provided to adjust the angle at which the two legs 30, 31 meet. The fixed portion 33 is provided to fix the compass 16a to the touch surface of the learner's terminal 12, with the tip of the leg 30 (a needle portion, not shown) serving as the center of rotation. The indicator 34 is, for example, an AES or EMR stylus, and is provided to indicate the position of the fixed portion 33. The pen portion 35 is, for example, an AES or EMR stylus, and is detachably provided at the tip of the leg 31.

[0027] The straight ruler 16b and the ruler 16c each include a main body 36 and two resonant circuits 37 and 38. The main body 36 is a long, rectangular plate member. A scale 39 for measuring length is printed on the main body 36 of the ruler 16c. The resonant circuits 37 and 38 are fixed at different positions on one main surface of the main body 36. The resonant circuits 37 and 38 are LC resonant circuits including a coil and a capacitor, and generate induced signals with different resonant frequencies. If the relative positional relationship between the resonant circuits 37 and 38 is known, the position and orientation of the straight ruler 16b or the ruler 16c, as well as their front and back orientations, can be uniquely determined from the detected positions of the resonant circuits 37 and 38.

[0028] FIG. 3 shows a second example of the stationery item 16 in FIG. 1. The triangular ruler 16d includes a main body 40 and three resonant circuits 41a, 41b, and 41c. The main body 40 is a triangular plate member with angles of 45°, 45°, and 90°, and has a scale 42 printed on it for measuring angles. The resonant circuits 41a to 41c are fixed at different positions on one of the main surfaces of the main body 40. The resonant circuits 41a to 41c are LC resonant circuits including a coil and a capacitor, and generate induced signals with different resonant frequencies. As with the straight set square 16b, this allows not only the position and orientation of the triangular ruler 16d but also its front and back orientation to be uniquely identified.

[0029] The triangular ruler 16e includes a main body 43 and three resonant circuits 44a, 44b, and 44c. The main body 43 is a triangular plate member with angles of 30°, 45°, and 90°, and a scale 45 is printed on the edge of one side. The resonant circuits 44a to 44c are fixed at different positions on one of the main surfaces of the main body 40. The resonant circuits 44a to 44c are LC resonant circuits including a coil and a capacitor, and generate induced signals with different resonant frequencies. As with the straight set square 16b, this makes it possible to uniquely identify not only the position and orientation of the triangular ruler 16e, but also its front and back orientation.

[0030] The protractor 16f includes a main body 46 and three resonant circuits 47a, 47b, and 47c. The main body 46 is a semicircular plate member with a scale 48 printed on it. The resonant circuits 47a to 47c are fixed at different positions on one main surface of the main body 46. The resonant circuits 47a to 47c are LC resonant circuits including a coil and a capacitor, and generate induced signals with different resonant frequencies. This makes it possible to uniquely identify not only the position and orientation of the protractor 16f, but also its front and back orientation, just like the straightedge 16b. Note that the protractor 16f may be a "circular protractor" in which the main body 46 is a circular plate member.

[0031] Furthermore, by adopting the EMR system as the means for detecting the stationery 16, there is no need to provide a power source in the stationery 16, and the stationery 16 can be made lighter, resulting in higher usability.

[0032] <Block diagram of automatic scoring device 18> Fig. 4 is a block diagram showing an example of the configuration of automatic marking device 18 in Fig. 1. Automatic marking device 18 specifically includes a communication unit 50, a control unit 52, and a storage unit 54.

[0033] The communication unit 50 is an interface for transmitting and receiving electrical signals to and from an external device, which allows the automatic marking device 18 to provide the question data D1 or marking result data D3 that it has generated to the learner terminal 12, and to obtain the answer data D2 from the learner terminal 12.

[0034] The control unit 52 is configured with a processor including a CPU or a GPU. The control unit 52 reads and executes the programs and data stored in the storage unit 54, thereby functioning as a data acquisition unit 60, a question processing unit 62, a scoring processing unit 64, and an output instruction unit 66.

[0035] The data acquisition unit 60 acquires various data related to question setting or grading. The data acquisition unit 60 may acquire data via communication from an external device including the learner terminal 12, or may acquire data by reading it from the storage unit 54.

[0036] The question setting processing unit 62 performs various information processing to set questions to the learner. Specifically, the question setting processing unit 62 refers to the question DB 70 in response to a request from the learner, and generates question data D1 to be provided to the learner terminal 12.

[0037] The marking processing unit 64 performs various information processes for marking the answer sheet indicated by the answer sheet data D2 acquired by the data acquisition unit 60. The specific configuration of the marking processing unit 64 will be described in detail with reference to FIG.

[0038] The output instruction unit 66 instructs the output device to present the scoring results obtained by the scoring processing unit 64 to the user. This "output device" may be a device provided in an external device including the learner terminal 12, or may be an output device (not shown) provided in the automatic scoring device 18. The output mode may be, for example, one that stimulates the five senses, such as sight, hearing, and touch.

[0039] The memory unit 54 stores programs and data necessary for the control unit 52 to control each component. The memory unit 54 is composed of a non-transitory, computer-readable storage medium. Here, the computer-readable storage medium is composed of [1] a storage device such as an HDD or SSD built into a computer system, or [2] a portable medium such as a magneto-optical disk, a ROM, a CD-ROM, or a flash memory. In the example of FIG. 4, the memory unit 54 has constructed a database related to questions (hereinafter referred to as "question DB70"), and stores a personal data group 70, question data D1, answer data D2, marking result data D3, and marking criteria data D4.

[0040] The question DB70 stores and registers the following items: [1] learning content that constitutes part of the question; [2] learning content that indicates model answers or instructional content; [3] question conditions including question format and time limit; or [4] analysis results of answers (e.g., statistical data such as average scores and percentage of correct answers).

[0041] The personal data group 72 corresponds to a data group customized for each individual user who sets and grades the questions. Specifically, the personal data group 72 includes user information 74 and learning outcome information 76.

[0042] The user information 74 includes various information related to the service user. Examples of the user information 74 include: [1] "user identification information" including name, nickname, user ID, password, and email address; [2] "attribute information" including school name, organization name, grade, and gender; and [3] "device identification information" including identification information of the electronic pen 14 (i.e., pen ID), identification information of the stationery 16 (i.e., stationery ID), and identification information of the learner terminal 12 (i.e., terminal ID).

[0043] The learning outcome information 76 includes various information related to the learning outcome. Examples of the learning outcome information 76 include [1] "history information" including the duration of the learning, the number of times the learning was performed, and the total duration of the learning, or [2] "grade information" including the score calculated through the practice problems and the degree of retention.

[0044] The question data D1 includes information about the questions (hereinafter, "question information"). This "question information" includes [1] the subject, format, and time of the question, [2] a set of questions and answers, or [3] the number, type, question order, and point allocation of the question content. This "question content" corresponds to at least one learning content that is the subject of the question. Examples of question content include graphic elements, images, illustrations, characters, words, phrases, phrases, and sentences.

[0045] The answer data D2 includes information about the answer to the question (hereinafter, "answer information"). When a question is asked that requires drawing, this "answer information" includes, in addition to the above-mentioned user information 74, stroke data 80 and usage data 82. The usage data 82 may be configured as a data file separate from the stroke data 80, or may be configured as a data file integrated with the stroke data 80.

[0046] The stroke data 80 includes information (hereinafter referred to as stroke information) indicating a collection of strokes formed through stroke operations using the electronic pen 14. The stroke data 80 constitutes, for example, a part of digital ink for expressing handwritten content. Examples of "ink description languages" for describing digital ink include WILL (Wacom Ink Layer Language), InkML (Ink Markup Language), and ISF (Ink Serialized Format).

[0047] Hereinafter, the figure formed by the entire set of strokes will be referred to as the "result figure." The elements that make up the result figure will be referred to as "graphic elements." Specifically, these graphic elements are composed of points, lines, curves, symbols, marks, characters, or combinations of these.

[0048] The usage data 82 includes information indicating usage actions of the stationery 16 to assist in drawing. The usage data 82 may constitute part of the digital ink, similar to the stroke data 80. Examples of "usage actions" include: [1] placing, adjusting the position, and rotating the compass 16a; [2] placing and adjusting the position of the straightedge 16b or ruler 16c; [3] placing and adjusting the position of the set squares 16d and 16e; or [4] placing and adjusting the position of the protractor 16f.

[0049] The grading result data D3 includes information about the grading result of the answer sheet indicated by the answer sheet data D2 (hereinafter, grading information). Examples of the grading information include the calculated score, the basis for the grading, the model answer, or the content of instruction to the learner.

[0050] The scoring criteria data D4 includes information regarding the scoring criteria for the question specified by the question data D1, specifically, point allocation information 98 and detection condition information 100 (FIG. 5). The scoring criteria data D4 may be unstructured data, structured data, or semi-structured data. Examples of semi-structured data include XML (Extensible Markup Language) or JSON (JavaScript (registered trademark) Object Notation).

[0051] <Functional block diagram of the scoring processing unit 64> Fig. 5 is a functional block diagram of the scoring processor 64 shown in Fig. 4. Specifically, the scoring processor 64 includes a data interpretation unit 90, an action detection unit 92, a graphic element detection unit 94, and a score calculation unit 96.

[0052] The data interpretation unit 90 acquires the scoring criteria data D4 corresponding to the question data D1 and performs an "interpretation operation" to interpret the content of the scoring criteria data D4. Through this interpretation operation, the scoring criteria data D4 is separated into scoring information 98 and detection condition information 100.

[0053] The score information 98 includes a score or a decision rule for score allocation. The data structure of the decision rule may be in a table format or a tree format. For example, in the case of a tree format, the score information 98 is described by a decision tree in which at least one of the action used, the graphic element, and the result graphic is used as an explanatory variable and the score is used as an objective variable. The score to be awarded may be set not only in two levels, full score and zero score, but also in at least one level of intermediate score or partial score.

[0054] The detection condition information 100 includes multiple detection conditions and condition branching rules. A detection condition may consist of a single condition or multiple sub-conditions. The sub-conditions are classified into "action conditions" related to the action to be used and "element conditions" related to the graphic elements. An example of an action condition is the type and positional relationship of the stationery 16. An example of an element condition is the shape and positional relationship of the stroke. The positional relationship may be either [1] a condition indicating the relative position and orientation with respect to a specific graphic element (hereinafter referred to as the first condition) or [2] a condition indicating the relative position and orientation with respect to a specific type of stationery 16 (hereinafter referred to as the second condition).

[0055] An example of a first condition in an action condition is [1] that the reference position of the stationery 16 to be detected coincides with a specific point, or [2] that the orientation of the stationery 16 to be detected coincides with the orientation of a specific line segment. An example of a first condition in an element condition is [1] that one end of a stroke to be detected coincides with a specific point, [2] that a specific point exists on the stroke to be detected, or [3] that the stroke to be detected is parallel or perpendicular to the specific stroke. An example of a second condition in an element condition is [1] that one end of a stroke to be detected coincides with a specific point, [2] that a specific point exists on the stroke to be detected, or [3] that the stroke to be detected is parallel or perpendicular to the specific stroke.

[0056] The action detection unit 92 performs an analysis process on the answer sheet data D2 to be graded, and performs a "first detection calculation" to detect a specific use action in the stationery 16. Through this first detection calculation, information indicating the detection result of the use action included in the detection condition information 100 (hereinafter referred to as primary detection information) is obtained.

[0057] The graphic element detection unit 94 performs an analysis process on the answer data D2 to be graded, and performs a "second detection calculation" to detect graphic elements corresponding to the used actions detected by the action detection unit 92. Through this second detection calculation, information indicating the detection results of the combinations of the used actions and graphic elements included in the detection condition information 100 (hereinafter referred to as secondary detection information) is obtained.

[0058] The score calculation unit 96 performs a "calculation operation" to calculate the score of the answer sheet according to the detection results of the used actions and / or graphic elements. In the example of Fig. 5, the score calculation unit 96 refers to the point allocation information 98 from the data interpretation unit 90 and the secondary detection information from the graphic element detection unit 94, and calculates the score of the answer sheet according to predetermined decision rules. Through this calculation operation, scoring result data D3 is generated.

[0059] [Automatic Scoring System 10 Operation] <1. Overall operation> The automatic marking system 10 according to one embodiment of the present invention is configured as described above. Next, an overview of the operation of the automatic marking system 10 will be described with reference to Figs. 6 to 8. Fig. 6 is a sequence diagram relating to the operation of the automatic marking system 10 shown in Fig. 1. Each step in this sequence diagram is executed by the learner terminal 12 and the automatic marking device 18 working together.

[0060] In step SP10 of FIG. 6, the learner terminal 12 accepts a predetermined operation for requesting a question (that is, a question request operation).

[0061] In step SP12, the learner terminal 12 transmits a signal requesting a question (that is, a question request signal) to the automatic marking device 18 in response to the question request operation performed in step SP10.

[0062] In step SP14, the automatic marking device 18 receives the question request signal sent in step SP12 from the learner terminal 12, and thereby acquires the user information 74 corresponding to the learner.

[0063] In step SP16, the automatic marking device 18 (more specifically, the question processing unit 62) uses the user information 74 acquired in step SP14 to generate question data D1 in response to a request from the learner.

[0064] In step SP18, the automatic marking device 18 (more specifically, the output instruction section 66) transmits the question data D1 generated in step SP16 to the corresponding learner terminal 12.

[0065] In step SP20, the learner terminal 12 receives the question data D1 transmitted in step SP18, generates a display signal showing the question screen 110 using the question data D1, and then supplies the display signal to the display unit 24. As a result, the question screen 110 is displayed on the display unit 24 of the learner terminal 12.

[0066] 7 is a diagram showing an example of a question screen 110 before an answer is provided. Specifically, the question screen 110 has a question field 112 showing the content of the question, an answer field 114 for inputting an answer by hand, and a button 116 labeled "Score." The question field 112 displays the question, "Use a compass to draw an equilateral triangle ABC with line segment AB as its base." The answer field 114 displays an initial figure 118 consisting of three geometric elements, specifically, points A, B, and line segment AB.

[0067] 8 is a diagram showing an example of the question screen 110 after an answer has been given. A result graphic 120 indicating the final result of the answer is drawn in the answer field 114. The result graphic 120 is composed of, in addition to the initial graphic 118, a plurality of graphic elements drawn using the electronic pen 14 and the stationery 16. The graphic elements consist of [1] three arcs drawn using the compass 16a, [2] two straight lines drawn using the electronic pen 14 and the straightedge 16b, [3] one point drawn using the electronic pen 14, and [4] one letter (C) drawn using the electronic pen 14.

[0068] 6, the learner terminal 12 receives a predetermined operation for requesting grading of the answer sheet (i.e., a grading request operation). In the example of FIG. 8, the grading request operation corresponds to a tap operation or a click operation for selecting the [GRADING] button 116.

[0069] In step SP26, the learner terminal 12 transmits a marking request signal including the accumulated answer sheet data D2 to the automatic marking device 18 at the time when the marking request operation is received in step SP24.

[0070] In step SP28, the automatic marking device 18 receives the marking request signal sent in step SP26 from the learner terminal 12, and thereby acquires the answer data D2 corresponding to the question data D1.

[0071] In step SP30, the automatic marking device 18 (more specifically, the marking processing unit 64) performs marking processing on the answer data D2 received in step SP28 to calculate the score of the answer, and generates marking result data D3.

[0072] In step SP32, the scoring processing unit 64 of the automatic scoring device 18 uses the scoring result data D3 obtained through the scoring process in step SP30 to update the learning outcome information 76. This learning outcome information 76 is stored in the memory unit 54 of the automatic scoring device 18 in a state associated with the learner.

[0073] In step SP34, the output instruction section 66 of the automatic marking device 18 transmits the marking result data D3 generated in step SP30 to the corresponding learner terminal 12.

[0074] In step SP36, the learner terminal 12 receives the grading result data D3 transmitted in step SP34, generates a display signal indicating the contents of the grading result using the grading result data D3, and then supplies the display signal to the display unit 24. This allows the learner to understand the grading result of the answer indicated by the answer data D2 through the display unit 24 of the learner terminal 12.

[0075] Thereafter, the automatic marking system 10 can automate the marking of answer sheets by repeatedly executing steps SP10 to SP36 in FIG.

[0076] <2. Details of scoring> Next, the scoring operation by the automatic scoring device 18 will be described in detail with reference to Figures 9 to 14. Figure 9 is a detailed flowchart relating to step SP30 in Figure 6.

[0077] 9, the automatic marking device 18 checks whether the timing for marking the answer sheet has arrived. If the timing for marking has not yet arrived (step SP40: NO), the automatic marking device 18 remains in step SP40 until the timing for marking arrives. If the timing for marking has arrived (step SP40: YES), the automatic marking device 18 proceeds to the next step SP42.

[0078] In step SP42, the automatic marking device 18 (more specifically, the data acquisition unit 60 in FIG. 4) acquires the answer data D2 to be marked. This answer data D2 includes stroke data 80 and usage data 82.

[0079] 10 is a diagram showing an example of the data structure of answer data D2. Answer data D2 has a data structure in which [1] document metadata, [2] ink semantics, [3] devices, [4] strokes, [5] groups, and [6] contexts are sequentially arranged.

[0080] The stroke data 80 is data for describing the individual strokes that make up the resulting graphic by handwriting, and indicates the shape of the strokes that make up the resulting graphic and the order in which the strokes are written. As can be seen from FIG. 10, one stroke is <trace>The stroke is described by a plurality of point data arranged sequentially within a tag. Each point data consists of at least a designated position (X coordinate, Y coordinate) and is separated by a delimiter such as a comma. For convenience of illustration, only the point data indicating the start and end points of the stroke is shown, and point data indicating multiple intermediate points is omitted. In addition to the designated position described above, this point data may also include the order in which the stroke was generated or edited, the pressure and posture of the electronic pen 14, etc.

[0081] The usage data 82 is described in the same data structure as the stroke data 80, as shown in Fig. 9. In this case, by managing the data using a stationery ID instead of a pen ID, it is possible to identify the time sequence between the stroke writing order and the usage action.

[0082] In step SP44 of FIG. 9, the data acquisition section 60 of the automatic marking device 18 acquires data (ie, marking standard data D4) for marking the answer sheet indicated by the answer sheet data D2 acquired in step SP42.

[0083] In step SP46, the automatic marking device 18 (more specifically, the data interpretation unit 90 in FIG. 5) interprets the marking standard data D4 acquired in step SP44, and extracts the marking information 98 and the detection condition information 100.

[0084] FIG. 11 is a diagram showing an example of a decision tree structure specified by the point allocation information 98 of FIG. 5. In the example of FIG. 11, the explanatory variables are "detection condition" and "degree of completion of the result figure," and the objective variable is "score." There are three types of detection conditions: A1, A2, and A3. "A1" is a condition item related to whether the length of one side of the equilateral triangle has been measured using the compass 16a. "A2" is a condition item related to whether the position of point C has been identified using the compass 16a. "A3" is a condition item related to whether the line segments AC and BC have been drawn using the straightedge 16b.

[0085] First, if the detection condition A1 is met and the result figure is "correct," partial points are awarded and it is determined that instruction regarding the detection condition A1 is necessary. On the other hand, if the detection condition A1 is not met and the result figure is "incorrect," no points are awarded (0 points). If the detection condition A1 is met, the process proceeds to the next scoring step.

[0086] Next, if the detection condition A2 is not met and the result figure is "correct," partial points are awarded and it is determined that instruction regarding the detection condition A2 is necessary. On the other hand, if the detection condition A2 is not met and the result figure is "incorrect," no points are awarded (0 points). If the detection condition A2 is met, the process proceeds to the next scoring step.

[0087] Next, if the detection condition A3 is not met and the resultant figure is "correct," partial points are awarded and it is determined that instruction regarding the detection condition A3 is necessary. On the other hand, if the detection condition A3 is not met and the resultant figure is "incorrect," no points are awarded (0 points). Furthermore, if the detection condition A3 is met but the resultant figure is "incorrect," partial points are awarded and it is determined that instruction regarding the resultant figure is necessary. Finally, if the detection condition A3 is met and the resultant figure is "correct," full points are awarded.

[0088] 12 is a first diagram showing an example of a decision tree structure specified by the detection condition information 100 of FIG. 5. This decision tree describes a detection condition A1 regarding whether the length of one side of an equilateral triangle has been measured using the compass 16a. Specifically, if either the first condition A11 or the second condition A12 is met, a value of "True" (held) is returned, and if neither the first condition A11 nor the second condition A12 is met, a value of "False" (not held) is returned.

[0089] The first condition A11 is a set of five sub-conditions: [1] the type of stationery 16 is a "compass," [2] the position of the needle (or indicator 34) is "point A," [3] the position of the pen 35 is "point B," [4] the stroke shape is "point or arc," and [5] the positional relationship is "including point B." The above-mentioned [1] to [3] are sub-conditions related to the action of use, and the above-mentioned [4] to [5] are sub-conditions related to the graphic element.

[0090] The second condition A12 is a set of five sub-conditions: [1] the type of stationery 16 is a "compass," [2] the position of the needle is "point B," [3] the position of the pen 35 is "point A," [4] the stroke shape is "point or arc," and [5] the positional relationship is "including point A." The above-mentioned [1] to [3] are sub-conditions related to the action of use, and the above-mentioned [4] to [5] are sub-conditions related to the graphic element.

[0091] 13 is a second diagram showing an example of a decision tree structure specified by the detection condition information 100 of FIG. 5. This decision tree describes a detection condition A2 regarding whether the position of point C has been identified using the compass 16a. Specifically, if both the first condition A21 and the second condition A22 are satisfied, a value of "True" (held) is returned, and if at least one of the first condition A21 and the second condition A22 is not satisfied, a value of "False" (not held) is returned.

[0092] The first condition A21 is a set of five sub-conditions: [1] the type of stationery 16 is a "compass," [2] the position of the needle (or indicator 34) is "point A," [3] the position of the pen 35 is "point C," [4] the stroke shape is an "arc," and [5] the positional relationship is "passes through point C." The above-mentioned [1] to [3] are sub-conditions related to the action of use, and the above-mentioned [4] to [5] are sub-conditions related to the graphic element.

[0093] The second condition A22 is a set of five sub-conditions: [1] the type of stationery 16 is a "compass," [2] the position of the needle is "point B," [3] the position of the pen 35 is "point C," [4] the stroke shape is "arc," and [5] the positional relationship is "passes through point C." The above-mentioned [1] to [3] are sub-conditions related to the action of use, and the above-mentioned [4] to [5] are sub-conditions related to the graphic element.

[0094] 14 is a third diagram showing an example of a decision tree structure specified by the detection condition information 100 of FIG. 5. This decision tree describes a detection condition A3 regarding whether or not line segments AC and BC have been drawn using a straightedge 16b. Specifically, if both the first condition A31 and the second condition A32 are satisfied, a value of "True" (held) is returned, and if at least one of the first condition A31 and the second condition A32 is not satisfied, a value of "False" (not held) is returned.

[0095] The first condition A31 is a set of five sub-conditions: [1] the type of stationery 16 is a "straight ruler," [2] the reference direction of the straight ruler 16b is "parallel to line segment AC," [3] the stroke shape is "straight line," [4] the positional relationship is "the line end coincides with point A," and [5] the positional relationship is "the line end coincides with point C." The above-mentioned [1] to [2] are sub-conditions related to the action of use, and the above-mentioned [3] to [5] are sub-conditions related to the graphic element.

[0096] The second condition A32 is a set of five sub-conditions: [1] the type of stationery 16 is a "straight ruler," [2] the reference direction of the straight ruler 16b is "parallel to the line segment BC," [3] the stroke shape is "straight line," [4] the positional relationship is "the end of the line coincides with point B," and [5] the positional relationship is "the end of the line coincides with point C." The above-mentioned [1] to [2] are sub-conditions related to the action of use, and the above-mentioned [3] to [5] are sub-conditions related to the graphic element.

[0097] 9, the scoring processing unit 64 performs a scoring process on the answer data D2 acquired in step SP42. This scoring process includes: [1] a first detection step (SP48a) by the action detection unit 92; [2] a second detection step (SP48b) by the graphic element detection unit 94; and [3] a calculation step (SP48c) by the score calculation unit 96.

[0098] In the example of Fig. 8, all three detection conditions A1, A2, and A3 shown in Fig. 11 are satisfied, and the result graphic 120 matches or resembles the graphic of the model answer, so the answer is scored full marks. In this way, the automatic scoring device 18 completes the scoring process for the answer data D2 in accordance with the flowchart shown in Fig. 9.

[0099] [Variations] Next, first to third modified examples of the automatic scoring device 18 in the above embodiment will be described with reference to FIGS.

[0100] <First Modification> First, a first modified example of the automatic scoring device 18 will be described with reference to FIGS.

[0101] 15 is a functional block diagram of a scoring processor 140 in the first modified example. In addition to the data interpreter 90, action detector 92, and graphic element detector 94, this scoring processor 140 further includes a graphic scoring unit 142 and a score adjuster 144. Note that the operations of the action detector 92 and graphic element detector 94 are the same as those in FIG. 5, and therefore will not be described here.

[0102] The data interpretation unit 90 acquires the scoring criteria data D4 corresponding to the question data D1 and performs an "interpretation operation" to interpret the content of the scoring criteria data D4. Through this interpretation operation, the scoring criteria data D4 is separated into the adjustment information 146 and the detection condition information 100.

[0103] The adjustment information 146 includes rules for determining adjustment points. The data structure of the decision rules may be in a table format or a tree format. For example, in the case of a tree format, the adjustment information 146 is described by a decision tree in which at least one of the actions used and the graphic elements is used as an explanatory variable and the adjustment points are used as an objective variable. Types of adjustment points include [1] "addition" which adds points to the raw score, [2] "deduction" which subtracts points from the raw score, and [3] "invalidation" which cancels the raw score and sets it to zero.

[0104] Examples of points awarded include satisfying [1] the "first point condition" regarding the state in which the function of stationery 16 is exercised, or [2] the "second point condition" regarding the drawing element showing the result of exercise of the function of stationery 16. Examples of the first point condition include [1] the shape of a stroke drawn using compass 16a being an arc, or [2] the shape of a stroke drawn using straightedge 16b or set square 16c being a straight line. Examples of the second point condition include [1] the drawing of a symbol indicating that two angles or line segments are equal in length, or [2] the drawing of a symbol indicating that two line segments are parallel or perpendicular.

[0105] An example of a point deduction would be when the "point deduction condition" regarding the state where the use of stationery 16 that should have been used when drawing a drawing is not detected. Examples of point deduction conditions include [1] drawing a circular stroke freehand without using the compass 16a, [2] drawing a straight stroke freehand without using the straightedge 16b, [3] measuring a length or angle using another stationery 16 instead of the compass 16a, or [4] estimating a length or angle by eye without using the stationery 16 with a scale.

[0106] An example of invalidation is when an "invalidation condition" is met regarding the state in which the use of stationery 16 that is prohibited for use when drawing is detected. Examples of invalidation conditions include [1] using stationery 16 that is explicitly prohibited in the question text, or [2] using stationery 16 that is not explicitly prohibited in the question text but that would easily lead to the answer if used.

[0107] The figure scoring unit 142 performs an analysis process on the stroke data 80 of the answer data D2, and performs a calculation (corresponding to the "first calculation") to calculate a raw score for the answer based on the content of the drawing indicated by the stroke data 80. For example, the figure scoring unit 142 compares a result figure identified from the stroke data 80 with a figure indicating the model answer (for example, a final figure or a figure in the process of being created), and calculates a raw score according to the similarity between the two. Through this first calculation, a raw score is assigned to the result figure.

[0108] The score adjustment unit 144 performs a calculation (corresponding to a "second calculation") to calculate the score by adjusting the raw score according to a decision rule from the detection results of the used actions and / or graphic elements. In the example of Figure 15, the score adjustment unit 144 refers to the adjustment information 146 from the data interpretation unit 90 and the secondary detection information from the graphic element detection unit 94, and performs an adjustment to add, subtract, or invalidate the raw score of the answer sheet according to a predetermined decision rule. Through this adjustment calculation, scoring result data D3 is generated.

[0109] Fig. 16 is a diagram showing an example of the data structure of the adjustment information 146 in Fig. 15. This adjustment information 146 is, for example, data in a table format showing the relationship between detection conditions and adjustment contents. For example, if detection condition A1 is met, two points are added (+2 points). If both detection conditions A1 and A2 are met, four points are added (+4 points).

[0110] On the other hand, points may be added, subtracted, or invalidated depending on how the stationery 16 is used. For example, if the compass 16a is used correctly, one point is added (+1 point). If the straightedge 16b is not used when drawing the lines BC and AC, one point is deducted (-1 point). If the compass 16a is not used or the protractor 16f is used improperly in the process of creating the answer sheet, the points are invalidated.

[0111] In this way, the scoring processor 140 may calculate the raw score of the answer based on the contents of the drawing indicated by the stroke data 80, and adjust the raw score according to the determination rules based on the results of detecting the actions and / or graphic elements used. This configuration also allows for more precise scoring depending on the presence or absence of the actions and / or graphic elements used, as in the case of the scoring processor 64 (FIG. 5).

[0112] <Second Modification> Next, a second modified example of the automatic scoring device 18 will be described with reference to FIGS.

[0113] 17 is a diagram showing an example of a question screen 150 in the second modified example. Specifically, the question screen 150 has a question field 152 showing the content of the question, an answer field 154 for inputting an answer by hand, and a button 156 labeled "Score." The question field 152 displays the question: "Using stationery, draw a line segment that passes through point C, is parallel to line segment AB, and has the same length." The answer field 154 has drawn an initial figure 158 consisting of four geometric elements, specifically, points A, B, C, and line segment AB.

[0114] This problem does not specify the type of stationery 16 to be used, so there are several possible solutions. For example, there are solutions such as [1] using two straightedges 16b together, [2] using a compass 16a and a straightedge 16b together, and [3] using a compass 16a and set squares 16d and 16e together.

[0115] Fig. 18 is another functional block diagram relating to the control unit 52 shown in Fig. 4. The control unit 52 further includes a selection unit 68 in addition to the scoring processing unit 64.

[0116] The scoring processing unit 64 performs a scoring process to score the answer sheet indicated by the answer sheet data D2 acquired by the data acquisition unit 60. However, multiple pieces of scoring standard data D4 are associated with one piece of question data D1. Through this scoring process, a score X corresponding to solution X, a score Y corresponding to solution Y, and a score Z corresponding to solution Z are calculated.

[0117] The selection unit 68 performs a selection process to select the maximum score from the multiple scores supplied from the scoring processing unit 64. Through this selection process, scoring result data D3 is generated.

[0118] In this way, the control unit 52 may calculate multiple scores according to multiple types of decision rules corresponding to one piece of question data D1 and select the maximum score from the multiple scores, thereby enabling questions to be posed with multiple possible solutions in mind.

[0119] <Third Modification> 19 is a diagram showing a first result figure 160 in the third modified example. Result figure 160 shows a solution to the problem of drawing a perpendicular bisector of line segment AB. Result figure 160 is composed of a first line segment 161, a pair of arc lines 162, a pair of arc lines 163, a second line segment 164, and a right-angle symbol 165.

[0120] The first line segment 161 is a line segment connecting two points A and B. The pair of arc lines 162 are two arcs that have their centers at the two points A and B, respectively, and intersect above the first line segment 161. The pair of arc lines 163 are two arcs that have their centers at the two points A and B, respectively, and intersect below the first line segment 161. The second line segment 164 is a line segment that connects an intersection P of the pair of arc lines 162 and an intersection Q of the pair of arc lines 163. The right-angle symbol 165 is an L-shaped mark that indicates that the first line segment 161 and the second line segment 164 are perpendicular to each other.

[0121] 20 is a diagram showing a second result figure 170 in the third modified example. Result figure 170 shows a solution to the problem of drawing a bisector for ∠BAC. Result figure 170 is composed of a first line segment 171, a second line segment 172, an arc line 173, a pair of arc lines 174, a third line segment 175, and an isometric symbol 176.

[0122] The first line segment 171 is a line segment connecting two points A and B. The second line segment 172 is a line segment connecting two points A and C. The arc line 173 is a single arc centered at point A that intersects with both the first line segment 171 and the second line segment 172. The pair of arc lines 174 are two arcs centered at intersection points U and V, respectively, that intersect between the first line segment 171 and the second line segment 172. The third line segment 175 is a line segment connecting point A and intersection point W of the pair of arc lines 174. The equal angle symbol 176 is a dot-like mark indicating that the angle ∠BAW is equal to the angle ∠CAW.

[0123] For example, when the drawings shown in Figures 19 and 20 are made, the graphic element detection unit 94 (Figure 5 or Figure 15) may detect the graphic elements of the right angle symbol 165 (Figure 19) or the equal angle symbol 176 (Figure 20) and add them to the scoring elements. This allows the basis for understanding the use of the stationery 16 to be detected and scoring to be performed according to that understanding.

[0124] [Summary of the embodiment] As described above, the automatic marking system 10 in this embodiment comprises an electronic pen 14, stationery 16 for assisting drawing with the electronic pen 14, a graded person terminal (here, a learner terminal 12) that generates answer data D2 indicating answers to questions that involve drawing through operation of the electronic pen 14 and the stationery 16, and an automatic marking device 18 configured to be able to communicate with the learner terminal 12.

[0125] Furthermore, according to the automatic scoring device 18, automatic scoring program, and automatic scoring method in this embodiment, one or more processors (here, the control unit 52) ​​execute an acquisition process (step S42) to acquire answer data D2 including stroke data 80 indicating a collection of strokes formed through stroke operations using the electronic pen 14 and usage data 82 indicating the usage action of the stationery 16, and a calculation process (step S48) to calculate the score of the answer based on the answer data D2 acquired through the acquisition process.

[0126] In this way, the score for the answer is calculated based on the answer data D2 including the stroke data 80 and the usage data 82, so that the scoring is performed by reflecting not only the result figures 120, 160, 170 finally indicated by the stroke data 80 but also the action of using the stationery 16. This allows for greater flexibility in scoring questions that involve drawing.

[0127] The calculation process (SP48) may also include a detection calculation (SP48a, SP48b) that analyzes the acquired answer data D2 to detect the use actions of the stationery 16 and the graphic elements corresponding to the use actions in the process of creating the answer, and a scoring calculation (SP48c) that calculates the score according to the detection results of the use actions and / or the graphic elements by the detection calculation. This allows for more precise scoring according to the presence or absence of the use actions and / or the graphic elements.

[0128] The scoring calculation (SP48c) may also include a calculation for calculating a score from the results of detecting the used actions and / or the graphic elements according to a decision rule. In particular, the decision rule may be described by a decision tree with at least one of the used actions, the graphic elements, and the degree of completion of the drawing as an explanatory variable and the score as an objective variable. This allows for fair and highly reproducible scoring according to a predetermined decision tree.

[0129] The scoring calculation (SP48c) may also include a first calculation for calculating a raw score for the answer based on the content of the drawing indicated by the stroke data 80, and a second calculation for calculating a score by adjusting the raw score according to a decision rule based on the detection results of the used actions and / or graphic elements. In particular, the decision rule may be described by a decision tree with the used actions and / or graphic elements as explanatory variables and adjustment points acting on the raw score as objective variables. This allows for fair and highly reproducible scoring according to a predetermined decision tree.

[0130] The decision rule may also be to add points to the raw score when the exertion of the function of the stationery 16 is detected, or to add points to the raw score when a drawing element showing the result of exerting the function of the stationery 16 is detected. This allows the degree of understanding of how to use the stationery 16 to be reflected in the scoring.

[0131] The decision rule may also be to deduct points from the raw score when the use of the stationery 16 that should be used when drawing is not detected, thereby encouraging the proper use of the stationery 16 through scoring.

[0132] The decision rule may also be to invalidate the raw score if the use of prohibited stationery 16 is detected during drawing. This will act as a deterrent to the unauthorized use of stationery 16, making it easier to ensure fairness in grading.

[0133] The graphic element may also include a stroke that satisfies a specific positional relationship with the position of the stationery 16 that is placed when the use action is performed. This makes it possible to detect a drawing state in which a stroke is formed while using the stationery 16.

[0134] The graphic elements may also include marks that indicate the results of the functions of the stationery 16. This allows the basis for understanding the use of the stationery 16 to be detected.

[0135] Furthermore, the learner terminal 12 may have an electromagnetic induction type planar sensor (here, EMR sensor 26), and the stationery item 16 may have a main body 36 (40, 43, 46) and a plurality of resonant circuits 37-38 (41a-41c, 44a-44c, 47a-47c) that are provided at different positions on the main body 36 (40, 43, 46) and that generate induction signals having different resonant frequencies. When the relative positional relationships of the resonant circuits 37-38 (41a-41c, 44a-44c, 47a-47c) are known, not only the position and orientation of the stationery item 16 but also its front and back orientation can be uniquely identified from the detected positions of the resonant circuits 37-38 (41a-41c, 44a-44c, 47a-47c). Furthermore, by adopting the EMR system, there is no need to provide a power source in the stationery 16, and the stationery 16 can be made lighter, resulting in higher usability.

[0136] The present invention is not limited to the above-described embodiment and the first to third modifications, and can be freely modified without departing from the spirit of the present invention. Alternatively, the respective configurations may be arbitrarily combined within the scope of no technical contradiction. Alternatively, the execution or execution order of each step constituting the flowchart or sequence diagram may be changed within the scope of no technical contradiction.

[0137] In the above embodiment, an example has been described in which the state of the stationery 16 is detected using the position detection function (i.e., AES method or EMR method) provided in the learner terminal 12, but the state detection method is not limited to this configuration. For example, a camera may be provided that points toward the touch surface of the learner terminal 12, and the learner terminal 12 or another analysis device may analyze the captured image to detect the position or movement of the stationery 16. Alternatively, a pressure sensor may be provided on the touch surface of the learner terminal 12, and the learner terminal 12 or another analysis device may analyze the pressure distribution to detect the position or movement of the stationery 16. [Explanation of symbols]

[0138] 10. Automatic marking system, 12. Learner terminal (marked person terminal), 14. Electronic pen, 16. Stationery, 37-38, 41a-41c, 44a-44c, 47a-47c. Resonance circuit, 50. Communication unit, 52. Control unit, 54. Memory unit, 60. Data acquisition unit, 62. Question processing unit, 64, 140. Marking processing unit, 66. Output instruction unit, 68. Selection unit, 80. Stroke data, 82. Usage data, 9 0: Data interpretation section, 92: Action detection section, 94: Graphic element detection section, 96: Score calculation section, 98: Score information, 100: Detection condition information, 110, 150: Question screen, 118, 158: Initial shapes, 120, 160, 170: Resulting shapes, 142: Shape scoring section, 144: Score adjustment section, 146: Adjustment information, D1: Question data, D2: Answer data, D3: Scoring result data, D4: Scoring criteria data< / trace>

Claims

1. one or more processors; The one or more processors: an acquisition process for acquiring answer data indicating an answer to a question involving drawing, the answer data including stroke data indicating a set of strokes formed through stroke operations using an electronic pen, and usage data indicating a use action of stationery to assist in the drawing; A grading process for calculating a score for the answer sheet based on the answer sheet data acquired through the acquisition process; An automatic scoring device that performs the following.

2. The scoring process includes: A detection calculation for analyzing the answer data acquired through the acquisition process to detect the stationery use actions and graphic elements corresponding to the use actions during the creation of the answer; a calculation operation for calculating the score in accordance with a result of detection of the action to be used and / or the graphic element by the detection operation, The automatic scoring device according to claim 1.

3. the calculation operation includes an operation of calculating the score from the detection result of the used action and / or the graphic element according to a decision rule; The automatic scoring device according to claim 2.

4. the decision rule is described by a decision tree having at least one of the action to be used, the graphic element, and the degree of completion of the drawing as an explanatory variable and the score as an objective variable; The automatic scoring device according to claim 3.

5. The calculation operation is a first calculation for calculating a raw score for the answer sheet based on the content of the drawing indicated by the stroke data; a second calculation of adjusting the raw score from the detection result of the used action and / or the graphic element according to a decision rule to calculate the score; The automatic scoring device according to claim 2.

6. The decision rule is described by a decision tree in which the action to be used and / or the graphic element are used as explanatory variables and an adjustment point acting on the raw score is used as a target variable. The automatic scoring device according to claim 5.

7. The decision rule is to add a point to the raw score when the performance of the function of the stationery is detected, or to add a point to the raw score when the drawing element showing the result of the performance of the function of the stationery is detected. The automatic scoring device according to claim 5.

8. The decision rule is to deduct points from the raw score when the use of the stationery that should have been used in the drawing is not detected. The automatic scoring device according to claim 5.

9. The decision rule is to invalidate the raw score when the use of the stationery that is prohibited for use is detected during the drawing. The automatic scoring device according to claim 5.

10. the graphic element includes a stroke that satisfies a specific positional relationship with the position of the stationery item that will be placed when the use action is performed; The automatic scoring device according to claim 1.

11. The graphic element includes a mark indicating the result of the function of the stationery being exerted. The automatic scoring device according to claim 1.

12. an acquisition process for acquiring answer data indicating an answer to a question involving drawing, the answer data including stroke data indicating a set of strokes formed through stroke operations using an electronic pen, and usage data indicating a use action of stationery to assist in the drawing; A grading process for calculating a score for the answer sheet based on the answer sheet data acquired through the acquisition process; An automatic scoring program that causes one or more processors to execute the above.

13. An electronic pen and Stationery for assisting drawing with the electronic pen; a terminal for a person to be graded that generates answer data indicating an answer to a question that requires drawing through operation of the electronic pen and the stationery; An automatic scoring device configured to be able to communicate with the graded person terminal, An acquisition process for acquiring the answer data including stroke data indicating a set of strokes formed through stroke operations using the electronic pen and usage data indicating a usage action of the stationery; A grading process for calculating a score for the answer sheet based on the answer sheet data acquired through the acquisition process; an automatic scoring device that executes the above; An automatic scoring system.

14. The graded person terminal has an electromagnetic induction type surface sensor, The stationery includes a main body and a plurality of resonant circuits provided at different positions on the main body for generating induction signals having different resonant frequencies. The automatic scoring system of claim 13.

Images