Program point rating system and program point rating method

Abstract

To easily recognize a faulty portion in a program for a learner.SOLUTION: A program point rating system includes: an acquisition unit for acquiring a learner program coded according to a subject; a control unit for executing an execution process for executing the learner program by inputting scoring input information including data which is data corresponding to the subject used for execution of the learner program, a scoring process for scoring the learner program by comparing output information outputted by execution of the learner program and scoring output information including a correct answer for the output; and a visualizing process for inputting output information and the scoring information to a simulator simulating an execution action of the learner program for the subject and visualizing the passage and result, having simulation carried out, and generating visualized information for visualizing a process and result in the execution action of the learner program based on the output of a simulator, and an output unit for outputting the visualized information and the result of the scoring.SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] The present invention relates to a program scoring system and a program scoring method. [Background technology]

[0002] In recent years, learning institutions such as universities have begun to assign students to code programs as part of their learning. Programs are intended to solve specific problems, such as making a robot perform various movements or search for a specific path.

[0003] For example, there is a program marking system installed on the web to mark a learner's program. A program marking system is a system that accepts a learner's program, executes it, marks it based on the execution result, and displays (outputs) the marking result.

[0004] The program marking system marks the program by, for example, having a computer execute the program using marking input data prepared in advance for a specific task, and judging whether the execution result matches the marking output data prepared in advance. The marking result fed back to the learner is, for example, the degree of match with the marking output data, or a pass / fail judged based on the degree of match.

[0005] Techniques relating to program evaluation are disclosed, for example, in the following: [Prior art documents] [Patent documents]

[0006] [Patent Document 1] JP 2022-32364 A [Patent Document 2] JP 2015-18467 A Summary of the Invention [Problem to be solved by the invention]

[0007] However, even if a student checks the pass / fail results or the degree of agreement, he or she cannot recognize which part of the program he or she created is defective. Some program grading systems display the processor execution time and memory usage, but even with these, it is difficult to recognize the defective part.

[0008] Therefore, one object of the present invention is to provide a programming grading system and a programming grading method that make it easy to recognize defects in a learner's program. [Means for solving the problem]

[0009] One disclosure relates to a program grading system having an acquisition unit that acquires a learner program coded according to an assignment, an execution process that inputs grading input information, which corresponds to the assignment and includes data used to execute the learner program, into the learner program and executes the learner program, a grading process that compares output information output in the execution of the learner program with grading output information including a correct answer to the output, and grades the learner program, and a visualization process that inputs the output information and the grading input information into a simulator that simulates the execution operation of the learner program in the assignment, executes the simulation, and generates visualization information that visualizes the progress and results of the execution operation of the learner program based on the output of the simulator, and an output unit that outputs the visualization information and the grading results.

[0010] One disclosure relates to a program scoring system in which the learner program and the control unit program are coded in a programming language having the same standard input / output, and input to and output from the learner program are provided using standard input / output commands.

[0011] One disclosure relates to a program scoring system in which the control unit and the simulator programs are coded in a programming language having the same standard input / output, and input to and output from the simulator are performed using standard input / output commands.

[0012] One disclosure relates to a program scoring system in which, when the task is a task regarding robot movement, the visualization information includes a diagram of the movement path of the robot from the start of the movement to the end of the movement.

[0013] One disclosure relates to a program scoring system in which the control unit executes the learner program and the simulator in parallel when the learner program is a program to which information is input during execution.

[0014] One disclosure relates to a program scoring system in which the control unit, in the parallel execution, inputs output information to the simulator when the output information is output during execution of the learner program, and inputs the input information to the simulator when the scoring input information is input during execution of the learner program. Effect of the Invention

[0015] This disclosure makes it easier for learners to identify flaws in their programs. [Brief description of the drawings]

[0016] [Figure 1] FIG. 1 is a diagram showing an example of the configuration of a program scoring system 10. As shown in FIG. [Diagram 2] FIG. 2 is a diagram illustrating an example of the configuration of the program scoring server 100. As shown in FIG. [Diagram 3] FIG. 3 is a diagram showing an example of the scoring result. [Figure 4] FIG. 4 is a diagram illustrating an example of visualization information. [Diagram 5] FIG. 5 is a diagram showing an example of the software configuration of the program scoring system 10. As shown in FIG. [Figure 6] FIG. 6 is a diagram showing an example of a process flowchart of the program scoring process S10. [Figure 7] FIG. 7 is a diagram showing an example of a process flowchart of the scoring process S20 for each test case. [Figure 8] FIG. 8 is a diagram illustrating an example of visualization information. [Figure 9] FIG. 9 is a diagram showing an example of the software configuration of the program scoring system 10. As shown in FIG. [Figure 10] FIG. 10 is a diagram showing an example of a process flowchart of the scoring process S21 for each test case. [Figure 11] FIG. 11 is a diagram showing an example of the parallel execution process S30. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0017] [First embodiment] A first embodiment will be described.

[0018] <Example of Program Scoring System 10 Configuration> 1 is a diagram showing an example of the configuration of a program marking system 10. The program marking system 10 is a system that marks a program to be marked (hereinafter, may be referred to as a learner program) and displays the evaluation result. The program marking system 10 is, for example, a system constructed on the web.

[0019] The program marking system 10 includes a program marking server 100, subject PCs 200-1 to 200-n, a simulator 300, and a network 400.

[0020] Subject PCs 200-1 to n (hereinafter sometimes referred to as subject PCs 200) are communication devices that submit (transmit) a learner program, and are, for example, a personal computer, tablet terminal, or smartphone owned by the subject. The subject is, for example, a learner such as a student who has been instructed to create a program corresponding to a certain assignment. The subject uses his / her own subject PC 200 to submit the learner program via a specified URL (Uniform Resource Locator).

[0021] Simulator 300 is a device that simulates a learner's program, and for example, executes a program or object that runs on a server machine to perform a simulation. Simulator 300 is placed in a different location from program marking server 100 in Fig. 1, but it may be placed within program marking server 100 (may be owned by program marking server 100), for example. Simulator 300 inputs a program (or object) to be simulated, execution conditions, test cases, output from the learner's program, etc., and outputs a simulation result.

[0022] A network 400 is a network that connects each device, and is, for example, the Internet or an intranet.

[0023] The program grading server 100 is a device that grades a learner's program, and is, for example, a server machine. The program grading server 100 accepts the learner's program from the subject PC 200. Then, the program grading server 100 executes the learner's program and grades it in cooperation with the simulator 300.

[0024] The program grading server 100 outputs the grading results of the learner's program. The output is executed, for example, by displaying it on the screen of the subject PC 200 or by creating a file. The grading results include, for example, visualization information generated based on the operation or output of the learner's program or the simulator 300, and a performance evaluation value for the task of the learner's program.

[0025] The visualization information is information that includes display methods other than text, such as figures and graphs, that show the execution results and grading results of a learner's program. For example, the visualization information is the operation results including intermediate steps such as the progress from the start to the end of execution of a learner's program in a certain test case, and the operating status.

[0026] The performance evaluation value is a value of the degree of coverage or performance of the learner's program with respect to the task, such as how well the learner's program meets the requirements of the given task or how efficiently the task is cleared.

[0027] <Example of the configuration of the program scoring server 100> 2 is a diagram illustrating an example of the configuration of the program scoring server 100. The program scoring server 100 includes a CPU (Central Processing Unit) 110, a storage 120, a memory 130, a communication circuit 140, and a display 150.

[0028] The storage 120 is an auxiliary storage device such as a flash memory, a hard disk drive (HDD), or a solid state drive (SSD) that stores programs and data. The storage 120 stores a grader program 121, a learner program grading program 122, and a visualization library 123. The visualization library 123 is a library used to generate visualization information, and includes, for example, programs and templates.

[0029] The memory 130 is an area into which the programs stored in the storage 120 are loaded. The memory 130 may also be used as an area in which the programs store data.

[0030] Communication circuitry 140 is a device that connects to and communicates with subject PC 200 and simulator 300. Communication circuitry 140 may be connected to subject PC 200 and simulator 300 via network 400.

[0031] Display 150 is a screen and display unit that displays the grading results, etc. A grader (a person who has presented the assignment, such as a teacher or professor) looks at display 150, starts grading, and inputs data into Grader program 121, etc. Display 150 may be, for example, an integrated type or an externally connected type.

[0032] The CPU 110 is a processor that loads a program stored in the storage 120 into the memory 130, executes the loaded program, configures each unit, and realizes each process.

[0033] The CPU 110 executes the Grader program 121 to construct a Grader. The Grader grades a learner's program. The Grader generates visualization information based on the operation or output of the learner's program, and generates a variety of performance evaluation values ​​set for the tasks of the learner's program. The Grader, for example, links the simulator 300 with the learner's program, and inputs and outputs data to and from the simulator 300. The Grader program 121 may be considered, for example, as a grading program or a part of a grading program.

[0034] It should be noted that the Grader program 121 exists for each assignment, for example. The question setter creates the Grader program 121 using a programming language having standard input and standard output, which will be described below. The Grader program 121 is created, for example, in the same or a different programming language as the learner program. The programming language of the learner program and the programming language of the Grader 121 may be different programming languages ​​as long as they have the same standard input and standard output.

[0035] The CPU 110 executes the learner program marking program 122 to construct a control unit and perform a learner program marking process. The learner program marking process is a process for executing the learner program and the simulator 300, generating visualization information, and calculating a performance evaluation value. Note that the learner program marking program 122 may be understood as, for example, a marking program or a part of a marking program.

[0036] The CPU 110 executes a learner program execution module 1221 included in the learner program marking program 122 to construct a control unit and perform a learner program execution process. The learner program execution process is a process for executing a learner program in cooperation with the simulator 300. The control unit controls, for example, a grader and performs the learner program execution process.

[0037] The CPU 110 executes the simulator cooperation module 1222 included in the learner program marking program 122 to construct a control unit and perform simulator cooperation processing. The simulator cooperation processing is processing for inputting data to the simulator and receiving output from the simulator. The control unit controls, for example, the Grader and performs the simulator cooperation processing.

[0038] The CPU 110 executes a visualization information generation module 1223 included in the learner program marking program 122 to construct a control unit and perform visualization information generation processing. The visualization information generation processing is processing for generating visualization information from outputs and operations from a simulator and a learner program. The control unit, for example, controls a Grader and performs visualization information generation processing.

[0039] The CPU 110 executes the performance evaluation value calculation module 1224 of the learner program marking program 122 to construct a control unit and perform a performance evaluation value calculation process. The performance evaluation value calculation process is a process for calculating a performance evaluation value from the output and operation from the simulator and the learner program. The control unit controls, for example, a Grader and performs the performance evaluation value calculation process.

[0040] <Display score results> An example of the grading result to be output will be described below. Fig. 3 is a diagram showing an example of the grading result. The grading result is, for example, displayed on a display unit of the subject PC 200 or the program grading server 100, or is saved as a file.

[0041] The scoring results include, for example, judgment results, feedback, judge input, judge output, user output, visualization information, and performance evaluation.

[0042] The judgment result may be, for example, whether the learner program has passed or failed. Alternatively, the judgment result may be, for example, a score for the learner program.

[0043] The feedback is, for example, a critique of the learner's program. The feedback may include, for example, good points, bad points, and points for improvement in the learner's program. The feedback may be written by, for example, a grader.

[0044] The judge input is, for example, data to be input to the learner program. The judge input may include, for example, conditions and response contents in a test case.

[0045] The judge output is, for example, correct answer data (sometimes called a scoring output) created by the creator (question setter) of the task.

[0046] The performance evaluation value is a value of the degree of coverage or performance of the learner's program for the task, such as how well the learner's program matches the given task or how efficiently the task is cleared. The performance evaluation value is displayed for each performance in the execution of the learner's program, such as CPU usage, work efficiency, time, coverage rate, etc.

[0047] Visualization information is information that includes a display method other than text, such as a diagram or graph, of the execution results and the grading results of the learner's program. For example, visualization information may be elements of a one-dimensional array and their movements, elements of a two-dimensional array and their movements, structural values ​​and paths such as trees, graphs, grids, and networks, geometric objects (points, lines, rectangles, polygons, circles) on a two-dimensional plane and their movements, structures (including point clouds) in three-dimensional space and their movements, nodes and links of dynamic data structures and their deletion / addition / movement, still images and videos obtained from a robot simulator, and charts (graphs, plots), etc. Visualization information includes information that allows the progress of the execution of a learner's program from start to finish, such as the process, progress, and results. Examples of visualization information are shown below.

[0048] Fig. 4 is a diagram showing an example of visualized information: Fig. 4 is an example of visualized information of a learner's program for the following task 1.

[0049] Task 1: Given the shape and motion characteristics of the robot, the workspace, obstacles, and the initial and target positions and orientations of the robot on a two-dimensional plane, create a program that outputs the speeds of the left and right wheels so that the robot reaches the target position and orientation without hitting any obstacles.

[0050] The visualized information for the task 1 shows the positions of obstacles and the trajectory of the robot. For the task 1, for example, the total moving distance (efficiency), the total rotation angle (stability), the distance from the obstacle (safety), etc. are evaluated as performance evaluation values.

[0051] Fig. 4(A) is a diagram showing an example of visualization information in a successful case. The visualization information shows the positions and shapes of obstacles 1 and 2. The visualization information also shows robot trajectory 1. According to trajectory 1, it can be seen that the robot moves from the bottom of obstacle 1 to the top of obstacle 2 while avoiding obstacles 1 and 2. In other words, it can be seen that the robot reaches the target position without colliding with obstacles 1 and 2.

[0052] Fig. 4(B) is a diagram showing an example of visualization information in the case of failure. The visualization information shows the positions and shapes of obstacles 1 and 2. The visualization information also shows trajectory 2 of the robot. According to trajectory 2, it can be seen that the robot has collided with obstacle 2. In other words, it can be seen that the robot has collided with the obstacle and has not yet reached the target position (the position reached by trajectory 1 in Fig. 4(A)).

[0053] By displaying visualized information in this way, the subject can not only determine whether or not the robot has reached the target position, but also recognize detailed progress along the way, such as, for example, that the robot did not turn when it should have, that it did not (presumably) recognize an obstacle, and that it collided with the left side of obstacle 2.

[0054] <Example of software configuration for Program Scoring System 10> Fig. 5 is a diagram showing an example of the software configuration of the program marking system 10. In Fig. 5, the execution of the learner code C1, the control of the Grader G1, and the control of the simulator S1 are controlled by a control unit (not shown) of the program marking server 100. In addition, when the simulator 300 exists outside the program marking server 100, the control unit controls the interface with the simulator S1.

[0055] The program grading system 10 includes a learner code C1, a Grader G1, a simulator S1, a visualization library L1, a grading input T1, a grading output T2, and a learner output O1.

[0056] The learner code C1 is a learner program, for example, a text code.

[0057] GraderG1 is the above-mentioned Grader, and is, for example, a processing unit constructed by a CPU executing the Grader program 121.

[0058] The simulator S1 is a processing unit that is constructed by the CPU of the simulator 300 executing a program.

[0059] The scoring input T1 is, for example, a precondition that exists for each test case of the task. For example, in the case of Task 1, the scoring input T1 includes the shape and motion characteristics of the robot, the working space and obstacles, the initial position and orientation of the robot, and the target position and orientation.

[0060] The marking output T2 is, for example, correct answer data that exists for each test case of the assignment, or data for calculating / generating / determining correct answer data. For example, in the case of assignment 1, the marking output T2 includes the trajectory of the robot. Note that if Grader G1 can generate data equivalent to the marking output T2 using the marking input T1, the marking output T2 may not be necessary.

[0061] The learner output O1 is data that is output by execution of the learner code C1.

[0062] In addition, standard input and output may be used for input and output of each part. By using standard input and output, any programming language that has standard input and output can be used as a target for grading as a learner's program, making it possible to build a program grading system that is less dependent on the language.

[0063] The control unit inputs the marking input T1 to the learner code C1 using the standard input, and executes the learner code C1 (In1). The standard input indicates, for example, a standard input command that a program has. For example, the standard input is "fgets" in the C language. For example, the standard input for inputting the marking input T1 is written in the learner code C1.

[0064] The control unit inputs the marking input T1 to the Grader G1 using a standard input (In2). For example, a standard input for inputting the marking input T1 is written in the Grader program 121. The control unit inputs, for example, a file including the marking input T1 (such as a text file in which the contents of the marking input are written) to the Grader G1 using a file input.

[0065] The control unit executes the learner code C1 to output the learner output O1 using the standard output (Out1). The standard output indicates, for example, a standard output command that the program has. The standard output is, for example, "fprintf" in the C language. For example, the standard output for outputting the learner output O1 is written in the learner code C1.

[0066] The control unit inputs the learner output O1 to the Grader G1 using the standard input (In3). For example, the standard input for inputting the learner output O1 is written in the Grader program 121.

[0067] The control unit inputs the grading output T2 to the Grader G1 using a standard input (In4). For example, a standard input for inputting the grading output T2 is written in the Grader program 121. The control unit inputs, for example, a file including the grading output T2 (such as a text file in which the contents of the grading output are written) to the Grader G1 using a file input.

[0068] The control unit links the GraderG1 with the simulator S1. The GraderG1 inputs the marking input T1 and the learner output O1, or data generated based on the marking input T1 and the learner output O1, to the simulator S1 using a standard input (In5). For example, when displaying examples of correct answers in the visualization information, the marking output T2 or data generated based on the marking output T2 may be input to the simulator S1.

[0069] In addition, GraderG1 obtains the output from simulator S1 in response to the input using the standard output (Out2).

[0070] For example, in the case of Task 1, the learner output O1 includes speed information of the robot (degree of rotation of the wheels). Furthermore, for example, in the case of Task 1, the learner output O1 may include position information of the robot for each predetermined time (or predetermined distance). The simulator S1 performs a simulation using this position information, and outputs data necessary for the visualization information. The data necessary for the visualization information includes, for example, information on whether the robot has collided with an obstacle, information on the positional relationship with the obstacle, etc.

[0071] In addition, if the simulator S1 exists externally, it is not known what programming language it is written in. The simulator S1 may be written in a programming language that does not have standard input / output. In this case, the interface between the Grader G1 and the simulator S1 may use input / output other than the standard input / output.

[0072] For example, if the simulator S1 is written in a programming language different from that of the Grader G1, the Grader G1 converts the data and information acquired through standard input and output into a format that can be input to the simulator S1, and inputs the converted data and information to the simulator S1.

[0073] Also, for example, if simulator S1 is written in a programming language different from that of Grader G1, Grader G1 converts the data and information obtained from simulator S1 into a format that can be used with standard input / output, and inputs it into the learner program using standard input / output.

[0074] The control unit causes the GraderG1 to output visualization information and a performance evaluation value (Out3). The GraderG1 generates visualization information by referring to the output from the simulator S1 and the visualization library L1 (R1). The visualization library L1 has programs and templates for illustrating the robot and obstacles by inputting, for example, information on the position and size of the robot, and information on the position and size of the obstacle.

[0075] Also, the GraderG1 compares the graded output T2 with the learner output O1 to calculate a performance evaluation value for the learner code C1. For example, the smaller the difference between the graded output T2 and the learner output O1, the higher the performance evaluation value. Also, the performance evaluation value may be a value including the CPU processing time, memory usage, etc.

[0076] <Program scoring process> 6 is a diagram showing an example of a processing flowchart of the program scoring process S10. The control unit of the program scoring server 100 executes the program scoring process S10.

[0077] The control unit presents an assignment to a learner (subject) (S101). The assignment is presented, for example, by presenting a file in which the assignment is written, or by presenting the assignment on the subject PC 200.

[0078] The control unit waits for acceptance of the learner program (No in S102). When the control unit acquires the learner program from the learner (Yes in S102), it executes and scores the learner program for each test case.

[0079] If all test cases have not been completed (No in S103), the control unit executes a grading process for each test case S20.

[0080] When all test cases have been completed (Yes in S103), the control unit presents the program grading results (processor execution time, memory usage, visualized image, execution results of the grading program, etc.) to the learner (S104), and ends the process.

[0081] 7 is a diagram showing an example of a processing flowchart of the grading process for each test case S20. Note that the grading process for each test case S20 targets a learner's program that can be executed to the end by being given conditions (such as the space, the shape of the robot, and the positions of obstacles in the case of Task 1) that are determined before execution, such as Task 1.

[0082] The control unit inputs marking input data as standard input to the learner's program and executes it (S201). The marking input data is, for example, data that has been prepared in advance as a corresponding test case.

[0083] The control unit converts the standard output of the learner program into a file as the learner output (S202). The file is, for example, a text file.

[0084] The control unit calculates the processor execution time and memory usage due to the execution of the learner program (S203). The control unit, for example, stores the calculated values ​​or outputs them as a file.

[0085] The control unit inputs the marking input data, the marking output data, and the learner output file to the grader as standard inputs, and executes the marking program (S204). Note that the marking program is a process for appropriately operating the simulator 300, such as a process for appropriately inputting the marking input data, the marking output data, and the learner output to the simulator 300, and acquiring the output from the simulator.

[0086] The control unit checks whether the marking program has ended (S205). If the marking program has not ended (No in S205), the control unit inputs input data to the simulator 300 as standard input and executes the simulator (S206). The input data is, for example, marking input data, marking output data, learner output, and data obtained by processing these data.

[0087] Then, the control unit generates a visualized image using the standard output of the simulator 300 and the visualization library 123 (S207). Note that there are cases where a visualized image already exists (for example, a visualized image has already been generated on the No path of branching process S205). In this case, the control unit updates only the changes from the existing visualized image. For example, in the case of problem 1, obstacles and previously traveled routes are displayed in the existing visualized image. Therefore, the control unit need only add (update) a new route, which is a change corresponding to the data input this time.

[0088] On the other hand, if the scoring program has ended (Yes in S205), the control unit ends the process. The determination of the end of the scoring program may be made, for example, when there is no more input data to the simulator 300, or when output from the simulator 300 has ended (when all data to be output has been output), etc.

[0089] [Second embodiment] The second embodiment is an example of a case where a learner program needs to communicate with a marking program (or a Grader). Communication means, for example, the marking program responding to a request or inquiry from the learner program. Communication includes, for example, the robot of the learner program sending out an echo and the marking program replying with a response to this.

[0090] Communication may also refer to the learner program responding to requests or queries from the marking program.

[0091] Additionally, communication may be data input from the marking program to the learner program at a certain time. For example, communication may include inputting information that changes over time (such as a disaster situation) at an appropriate time (such as an appropriate step in the learner program).

[0092] <Display score results> Fig. 8 is a diagram showing an example of visualized information: Fig. 8 is an example of visualized information of a learner's program for the following task 2.

[0093] Task 2: Write a program to control an indoor field robot. However, you will not be given a map of the field. Your program will communicate with the grading program as follows: The field consists of N rows in the north-south direction and N columns in the east-west direction, totaling N x N squares. Each cell is represented by its row number y (1, 2, ..., N) and column number x (1, 2, ..., N), and each square is either a wall or a path. The robot can enter a path square, and will become uncontrollable if it enters a wall square or outside the field. Your program must first get the size of the field N, the start point (sy, sx), and the end point (gy, gx) from standard input in the format "N sy sx gy gx". After that, the program can repeatedly perform the following two types of operations:

[0094] 1. Survey operation Send out ultrasonic waves in the specified direction. Your program should send out standard output in the form of "search dir", where dir can be E, N, W, or S, to send ultrasonic waves in the East, North, West, or South direction, respectively. Immediately after this, your program should receive the result on standard input in the form of "L", where L is an integer representing the number of path squares from the robot's current position in the specified direction before the wall square is reached.

[0095] 2. Movement Move the robot one square in the specified direction. Your program should output to standard output in the form [move dir], where dir can be E, N, W, or S, which means to move the robot one square in the east, north, west, or south direction, respectively. Immediately after this, your program should receive a result on standard input in the form of "state", which can be moved, success, or failed, meaning that the robot has moved in the specified direction, reached its destination, or failed to move, respectively.

[0096] Thus, in Task 2, communication with the scoring program occurs (the robot emits ultrasound and receives a response).

[0097] The visualized information for Task 2 shows spaces (movable spaces) such as walls (wall spaces) and paths that the robot has found, spaces that the robot has moved through (movable spaces), and spaces that are unclear as to whether they are paths or walls (unknown spaces). Note that in Fig. 8, movable spaces are shown in white, movable spaces in dark grey, wall spaces in black, and unknown spaces in light grey. Note that in Fig. 8, movable spaces are considered to be part of the movable spaces.

[0098] Fig. 8(A) is a diagram showing an example of visualized information when work efficiency is high. The visualized information shows the ranges of 1 movable square, 1 wall square, 1 moveable square, and 1 unknown square. Fig. 8(B) is a diagram showing an example of visualized information when work efficiency is low. The visualized information shows the ranges of 2 movable squares, 2 wall squares, 2 moveable squares, and 2 unknown squares.

[0099] It can be seen that the number of moving squares is smaller in Fig. 8(A) than in Fig. 8(B). This shows that the learner program in Fig. 8(A) is able to reduce the robot's moving distance during route search more than the learner program in Fig. 8(B). In other words, the learner program in Fig. 8(A) is more efficient in its work.

[0100] By displaying visualized information in this way, the subject can recognize how far the robot has moved and how many known squares it has discovered.

[0101] <Example of software configuration for Program Scoring System 10> FIG. 9 is a diagram showing an example of the software configuration of the program scoring system 10. As shown in FIG.

[0102] The program grading system 10 includes a learner code C1, a Grader G1, a simulator S1, a visualization library L1, and a grading input T1.

[0103] The scoring input T1 includes map information of the field, for example, in the case of Task 2. The map information includes information such as the positions of walls in the field (positions where ultrasonic waves are reflected).

[0104] The control unit inputs the marking input T1 to the Grader G1 using a standard input (In10).

[0105] The control unit inputs the marking input T1 or data generated by the GraderG1 based on the marking input T1 to the learner code C1 using the standard input (In11). For example, in the case of Task 2, this input includes information about the response to the ultrasonic waves emitted by the robot.

[0106] The control unit executes the learner code C1 to output, for example, data equivalent to the learner output O1 in the first embodiment using the standard output (Out10). For example, in the case of Task 2, this output includes information about the ultrasonic waves emitted by the robot.

[0107] The control unit inputs the data output by the learner code C1 to the Grader G1 using the standard input (In12). The control unit does not output the data as the learner output O1 as in the first embodiment, but inputs the output of the learner code C1 to the Grader G1 using, for example, pipe processing.

[0108] The control unit links GraderG1 with simulator S1. GraderG1 controls simulator S1 in parallel with execution of learner code C1. GraderG1 inputs the output from learner code C1, data generated by GraderG1 from marking input T1, and the like to simulator S1 using standard input (In13). Then, GraderG1 obtains the output from simulator S1 in response to the input using standard output (Out11).

[0109] For example, in the case of Task 2, the data input to the simulator S1 includes information regarding the direction in which the ultrasonic waves are emitted from the robot, the position of the robot, the position of the walls, the response to the ultrasonic waves, etc. The simulator S1 uses this data to perform a simulation and outputs data necessary for visualization information. The data necessary for visualization information includes, for example, information on the identified squares and the moved squares.

[0110] The control unit causes the GraderG1 to output the visualization information and the performance evaluation value (Out12). The GraderG1 refers to the output from the simulator S1 and the visualization library L1 (R2) and generates the visualization information.

[0111] Moreover, the GraderG1 compares the output of the learner code C1 with the data generated from the marking input T1, and calculates a performance evaluation value for the learner code C1.

[0112] <Program scoring process> In the second embodiment, the program marking process S10 in FIG. 6 is replaced by a marking process for each test case S20, and a marking process for each test case S21 is executed.

[0113] 10 is a diagram showing an example of the marking process for each test case S21. Note that the marking process for each test case S21 targets a learner's program involving parallel processing, such as Exercise 2.

[0114] The control unit inputs the marking input data to the Grader as standard input, and executes the marking program (S211).

[0115] The control unit executes a parallel execution process S30. The parallel execution process S30 is a process for executing the simulator 300 in parallel with a process for executing the learner program while communicating with the Grader.

[0116] The control unit generates a visualized image using the standard output of the simulator 300 and the visualization library 123 (S212). Note that there are cases where a visualized image already exists (for example, a visualized image has already been generated on the No path of branching process S213). In this case, the control unit updates only the changes from the existing visualized image. For example, in the case of assignment 2, the determined squares and moved squares are displayed in the existing visualized image.

[0117] The control unit checks whether the learner program and the marking program have been completed (S213). If either the learner program or the marking program has not been completed (No in S213), the control unit executes the parallel execution process S30.

[0118] The reason for the expression "whether the learner program and the marking program have finished" is that which of the learner program and the marking program finishes first differs depending on the task. For example, in the case of Task 2, which program finishes first differs depending on how the termination condition is set. For example, if the termination condition is the passage of a certain amount of time, and the time is calculated from the number of steps in the learner program, the learner program will detect the termination condition and therefore will finish first. On the other hand, if the termination condition is the discovery of a certain wall, the marking program will recognize the termination condition first and therefore will finish first.

[0119] Incidentally, even with regard to the input of step S211, depending on the task and the end conditions, the learner's program may be executed in advance.

[0120] If the learner program and the marking program have ended (Yes in S213), the control unit calculates the processor execution time and memory usage due to the execution of the learner program (S214), and ends the process.

[0121] 11 is a diagram showing an example of the parallel execution process S30. By appropriately performing the parallel execution process S30, the Grader can cooperate with the simulator 300 and generate visualization information of the learner's program that requires communication.

[0122] Process S31 is a process in which the Grader generates further data in response to an output from the learner program. Process S31 is a process used when, for example, in the case of Task 2, an ultrasonic wave is emitted and a response is returned.

[0123] In process S31, the Grader acquires learner output from the learner program as standard output (S311). The Grader inputs the learner output or data generated based on the learner output to the simulator 300 as standard input (S312). The Grader generates data such as a response to the learner output (S313) and inputs it to the learner program and simulator 300 as standard input (S315, S314).

[0124] Process S32 is a process triggered by data generation by a grader. Process S32 is used in processes that the learner program cannot predict (the learner program cannot generate a trigger), such as the occurrence of a fault or the passage of time.

[0125] In process S32, the grader generates data (S321) and inputs the data to the learner program and simulator 300 via standard input (S323, S322).

[0126] Process S33 is a process in which the Grader generates further data in response to an output from the simulator 300. For example, in the case of Problem 2, process S33 is a process used when there is an output from the simulator after ultrasonic wave and response information are input to the simulator.

[0127] In process S33, the Grader obtains the simulation results from the simulator 300 as standard output (S331). The Grader inputs the simulation results to the learner's program as standard input (S332). The Grader, for example, generates data based on the simulation results (S333) and inputs the data to the learner's program and the simulator 300 as standard input (S335, S334).

[0128] In addition, in the case where data generation is not performed in processes S31 to S33, the following input may not be performed. Furthermore, the generated data is not necessarily input to both the simulator 300 and the learner's program, and may be input to only one of them. Furthermore, processes S31 to S33 are an example of the parallel execution process S30, and other processes may also be present depending on the assignment. The parallel execution process S30 may be any process that can appropriately control the simulator so that information required for visualization information and performance evaluation information can be obtained from the simulator 300 while appropriately executing the learner's program.

[0129] [Other embodiments] The first and second embodiments may be combined. For example, the program marking server 100 may use the marking process S20 or S21 for each test case depending on the assignment.

[0130] The function of the simulator may be incorporated in a grading program (or a Grader). The visualization library may be included in the simulator, for example, and in this case, the simulator may output visualization information.

[0131] In addition, the standard input and standard output between the Grader and the simulator are standard input on the side that receives the data and standard output on the side that sends the data. For example, the simulator receives the data that the Grader outputs as standard output through standard input. Also, for example, the Grader receives the data that the simulator outputs as standard output through standard input. Input and output between the Grader and the simulator may be performed, for example, through pipe processing. [Explanation of symbols]

[0132] 10: Program Scoring System 100: Program scoring server 110: CPU 120: Storage 121: Learner Program 1221: Student program execution module 1222: Simulator link module 1223: Visualization information generation module 1224: Performance evaluation value calculation module 122: Student Program Grading Program 123: Visualization Library 130: Memory 140: Communication circuit 150: Display 200: Subject PC 300: Simulator 400 : Network

Claims

1. an acquisition unit that acquires a learner program coded according to a task; an execution process of inputting marking input information, including data corresponding to the assignment and used to execute the learner program, into the learner program and executing the learner program; a marking process for marking the learner program by comparing output information outputted during execution of the learner program with marking output information including a correct answer to the output; a control unit that executes a visualization process of inputting the output information and the marking input information into a simulator that simulates the execution operation of the learner program in the task, executing a simulation, and generating visualization information that visualizes the progress and results of the execution operation of the learner program based on an output of the simulator; an output unit that outputs the visualization information and the result of the scoring. Program scoring system.

2. The learner program and the control unit program are coded in a programming language having the same standard input and output, Input to and output from the learner program are performed using standard input / output commands.

2. The program scoring system of claim 1.

3. The programs of the control unit and the simulator are coded in a programming language having the same standard input and output, Input to and output from the simulator are performed using standard input / output commands.

2. The program scoring system of claim 1.

4. When the task is related to the movement of a robot, the visualization information includes a diagram of a movement path of the robot from the start of the movement to the end of the movement.

2. The program scoring system of claim 1.

5. When the learner program is a program to which information is input during execution, the control unit Executing the student program and the simulator in parallel 2. The program scoring system of claim 1.

6. The control unit, in the parallel execution, When the output information is output during execution of the learner program, input the output information to the simulator; When the marking input information is input during execution of the learner program, the marking input information is input to the simulator.

6. A program scoring system according to claim 5.

7. an acquisition process for acquiring a learner program coded according to the task; an execution process of inputting marking input information, including data corresponding to the assignment and used to execute the learner program, into the learner program and executing the learner program; a marking process for marking the learner program by comparing output information outputted during execution of the learner program with marking output information including a correct answer to the output; a visualization process of inputting the output information and the marking input information into a simulator that simulates the execution operation of the learner program in the task, executing a simulation, and generating visualization information that visualizes the progress and results of the execution operation of the learner program based on an output of the simulator; an output process for outputting the visualization information and the scoring result; A program scoring method characterized by being executed by a computer.

Images