Programmatic thinking evaluation device, method, and program

Abstract

One aspect of this invention displays, as a pair, information representing a first state and information representing a second state that has changed from the first state, and displays multiple options representing commands or values used in at least one process necessary to change from the first state to the second state. The invention acquires, for each process, answers created by a respondent who has sequentially selected and input multiple options for the commands or values, evaluates the acquired answers by comparison with correct information stored in advance, and outputs information representing the evaluation result.

Description

Computational thinking assessment device, method, and program

[0001] One aspect of the present invention relates to an evaluation device, an evaluation method, and a program used to evaluate a person's level of computational thinking.

[0002] In recent years, software that achieves automation without directly writing source code in a programming language (low-code / no-code tools, etc.) has been developed. This software is extremely effective in improving business efficiency.

[0003] By the way, in order to improve business processes using this type of software, computational thinking is required. Computational thinking refers to the ability to logically think through the actions required to achieve a desired activity and the corresponding combinations of symbols, or how to combine those symbols. However, there are differences in the level of computational thinking among people, and it is necessary to hold training and seminars or create lecture materials tailored to each level.

[0004] Therefore, a conventional method has been proposed for evaluating the level of understanding of a programming language by using a test to rearrange divided source code into the correct order (see, for example, Non-Patent Document 1).

[0005] Another method has been proposed in which respondents write their own conceptual diagrams and source code for a program, and the evaluator evaluates the code against criteria to assess the degree of understanding of the algorithm (see, for example, Non-Patent Document 2).

[0006] Dale Parsons and Patricia Haden, "Parson's Programming Puzzles: A Fun and Effective Learning Tool for First Programming Courses," ACE '06: Proceedings of the 8th Australasian Conference on Computing Education - Volume 52, January 2006, pp. 157-163. Tetsuya Bando, Yoshiaki Kawashima, Akira Kikuchi, Masahiro Kabe, and Jun Moriyama, "Practice and Evaluation of Elementary School Programming Education Using PIC-GPE Built-in LED Light-Emitting Materials," Journal of the Japan Society for Industrial Technology Education, Vol. 59, No. 3, 2017, pp. 187-197.

[0007] However, the method described in Non-Patent Document 1 is capable of evaluating the level of understanding of programming languages, but is unable to evaluate programming thinking.

[0008] On the other hand, the method described in Non-Patent Document 2 is capable of evaluating the level of understanding of algorithms related to computational thinking. However, since the respondent's answers are evaluated by an evaluator in light of evaluation criteria, there is a possibility that the evaluator's subjectivity may be reflected in the evaluation results, which poses a challenge in maintaining the objectivity of the evaluation results.

[0009] The present invention has been made in light of the above circumstances, and aims to provide a technique that enables objective and effective evaluation of a respondent's programming thinking.

[0010] In order to solve the above problems, one aspect of the evaluation device or evaluation method for computational thinking according to the present invention displays a pair of information representing a first state and information representing a second state after a change from the first state, and displays a plurality of options representing commands or values ​​required to realize a process required to change from the first state to the second state, and obtains an answer created by a respondent by selecting and inputting options in order for each process, compares the obtained answer with pre-stored correct answer information to evaluate it, and outputs information representing the evaluation result.

[0011] According to one aspect of the present invention, a respondent is prompted to create the process required to change from a first state to a second state by selecting and inputting options in order, and the respondent's computational thinking is evaluated by comparing the answer with the correct answer. This makes it possible to objectively evaluate the respondent's computational thinking without involving the subjectivity of an evaluator or the like.

[0012] Furthermore, the questionnaire format presents multiple options for each process and the respondent must select an answer from these options. This means that respondents can answer the questionnaire without needing any special expertise in programming languages ​​or the like, and without having to learn any new operations. This makes it possible to effectively evaluate the respondent's basic programming thinking without being influenced by other evaluation factors.

[0013] That is, according to one aspect of the present invention, it is possible to provide a technology that enables objective and effective evaluation of a respondent's computational thinking.

[0014] FIG. 1 is a diagram illustrating an example of the configuration of a system including an evaluation device according to an embodiment of the present invention. FIG. 2 is a block diagram illustrating an example of the hardware configuration of the evaluation device according to an embodiment of the present invention. FIG. 3 is a block diagram illustrating an example of the software configuration of the evaluation device according to an embodiment of the present invention. FIG. 4 is a flowchart illustrating an example of the overall processing procedure and processing content of the evaluation process executed by the control unit of the evaluation device shown in FIG. 3. FIG. 5 is a flowchart illustrating an example of the processing procedure and processing content of the answer input processing executed on the user terminal. FIG. 6 is a flowchart illustrating an example of the processing procedure and processing content of the evaluation and tallying processing of the evaluation process shown in FIG. 4. FIG. 7 is a diagram illustrating an example of information representing a state change used to test the "sequential" processing process of the basic unit of programmatic thinking. FIG. 8 is a diagram illustrating an example of an answer field presented together with the information representing the state change shown in FIG. 7. FIG. 9 is a diagram illustrating an example of options representing commands presented together with the information representing the state change shown in FIG. 7. FIG. 10 is a diagram illustrating an example of information representing a state change used to test the "iterative" processing process of the basic unit of programmatic thinking. FIG. 11 is a diagram illustrating an example of an answer field presented together with the information representing the state change shown in FIG. 10. FIG. 12 is a diagram showing an example of options expressing commands to be presented together with information expressing state changes shown in FIG. 10 . FIG. 13 is a diagram showing an example of information expressing state changes used to test the "branch" processing process in the basic unit of programmatic thinking. FIG. 14 is a diagram showing an example of an answer field to be presented together with the information expressing state changes shown in FIG. 13 . FIG. 15 is a diagram showing an example of options expressing commands to be presented together with the information expressing state changes shown in FIG. 13 . FIG. 16 is a diagram showing an example of information expressing state changes used to test the "variable" processing process in the basic unit of programmatic thinking. FIG. 17 is a diagram showing an example of an answer field to be presented together with the information expressing state changes shown in FIG. 16 . FIG. 18 is a diagram showing an example of options expressing commands to be presented together with the information expressing state changes shown in FIG. 16 . FIG. 19 is a diagram showing an example of an answer result to the information expressing state changes shown in FIG. 7 . FIG. 20 is a diagram showing an example of an answer result to the information expressing state changes shown in FIG. 10 .Fig. 21 is a diagram showing an example of a response result to the information showing the status change shown in Fig. 13. Fig. 22 is a diagram showing an example of a response result to the information showing the status change shown in Fig. 16.

[0015] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0016] [One embodiment] (Configuration example) (1) System FIG. 1 is a diagram showing an example of the configuration of a system including an evaluation device used to evaluate computational thinking according to one embodiment of the present invention.

[0017] In Fig. 1, SV denotes an evaluation device, UT1 to UTn denote user terminals used by respondents to be evaluated, and the evaluation device SV is capable of data communication with the user terminals UT1 to UTn via a network NW. Note that the evaluation device SV is also capable of data transmission via the network NW with, for example, an administrator terminal (not shown) used by an evaluator.

[0018] The user terminals UT1 to UTn are, for example, general-purpose personal computers equipped with an input device and a display device, and by executing a pre-installed application program, receive presented information such as questions related to the evaluation test transmitted from the evaluation device SV and display it on the display device, and also perform the operation of returning answer information entered by the respondent in the input device in response to the displayed information such as the questions to the evaluation device SV. Note that, in addition to personal computers, tablet terminals and smartphones may also be used as the user terminals UT1 to UTn.

[0019] The network NW includes a wide area network with the Internet at its core, and an access network for accessing the wide area network. Examples of the access network include, but are not limited to, a public communication network using wired or wireless connections, a local area network (LAN) using wired or wireless connections, and a cable television (CATV) network.

[0020] (2) Evaluation Device SV FIGS. 2 and 3 are block diagrams showing an example of the hardware configuration and software configuration, respectively, of an evaluation device SV according to an embodiment of the present invention.

[0021] The evaluation device SV is configured by, for example, a server computer or a personal computer located on the web or cloud. The evaluation device SV includes a control unit 1 using a hardware processor such as a central processing unit (CPU), and a storage unit having a program storage unit 2 and a data storage unit 3, and a communication interface (hereinafter, the interface will be referred to as I / F) unit 4 connected to the control unit 1 via a bus 5.

[0022] The communication I / F unit 4 has the function of accessing the network NW and sending and receiving data, and under the control of the control unit 1, sends question information etc. for evaluation to the user terminals UT1 to UTn, and receives answer information returned from the user terminals UT1 to UTn.

[0023] The program storage unit 2 is configured, for example, by combining a non-volatile memory such as a HDD (Hard Disk Drive) or SSD (Solid State Drive) as a storage medium that can be written to and read at any time, with a non-volatile memory such as a ROM (Read Only Memory), and stores various programs necessary to execute various control processes according to one embodiment of the present invention, in addition to middleware such as an OS (Operating System).

[0024] The data storage unit 3 is configured, for example, by combining a non-volatile memory such as an HDD or SSD, which can be written to and read from at any time, with a volatile memory such as a RAM (Random Access Memory), as a storage medium, and includes a question information storage unit 31, an option storage unit 32, an answer information storage unit 33, and an evaluation aggregation information storage unit 34 in the storage area.

[0025] The problem information storage unit 31 stores a plurality of pieces of information representing state changes as information representing problems used for evaluation. The information representing state changes is composed of a first state indicating the state before the change and a second state after the change from the first state through at least one process requiring computational thinking.

[0026] The option storage unit 32 stores a plurality of options prepared in advance in association with each of the above-mentioned question information and information representing an answer field. A plurality of options are prepared in association with commands or values ​​used in a process required to change from a first state to a second state, and are written in natural language. The answer field is used by a respondent to input an answer selected from the above options in response to the above-mentioned question information, and is prepared in association with each of the above-mentioned question information.

[0027] The answer information storage unit 33 stores answer information created by a respondent selecting multiple options in order in response to presented question information when it is received from the user terminals UT1 to UTn. The evaluation aggregation information storage unit 34 stores information representing the results of evaluation aggregation processing of the answer information as evaluation aggregation information.

[0028] The control unit 1 has, as processing functions according to one embodiment of the present invention, a question information presentation processing unit 11, an option presentation processing unit 12, an answer information acquisition processing unit 13, an evaluation aggregation processing unit 14, and an evaluation aggregation information output processing unit 15.

[0029] These processing units 11 to 15 are all realized by causing a hardware processor in the control unit 1 to execute application programs stored in the program storage unit 2. Note that some or all of the processing units 11 to 15 may be realized using hardware such as an LSI (Large Scale Integration) or an ASIC (Application Specific Integrated Circuit).

[0030] When performing the evaluation process, the problem information presentation processing unit 11 reads out problem information consisting of a pair of a state before the change and a state after the change from the problem information storage unit 31, and transmits the read out problem information from the communication I / F unit 4 to the user terminals UT1 to UTn used by the respondents to be evaluated.

[0031] The option presentation processing unit 12 reads out options written in natural language and information representing answer fields from the option storage unit 32, which are associated with the identification information of the transmitted question information, and transmits the read-out information representing the options and answer fields from the communication I / F unit 4 to the respondent's user terminals UT1 to UTn.

[0032] The answer information acquisition processing unit 13 receives answer information created by respondents in the user terminals UT1 to UTn by selecting options in order in response to the presented question information via the communication I / F unit 4. Then, the received answer information is stored in the answer information storage unit 33 in a state where it is associated with the identification information of the respondent and the identification information of the question information.

[0033] The evaluation aggregation processing unit 14 reads the answer information to be aggregated for evaluations from the answer information storage unit 33 in accordance with a preset aggregation processing rule, for example, each time the process of acquiring answer information for one piece of question information is completed, or when the process of acquiring answer information for all the question information is completed. The evaluation aggregation processing unit 14 then executes an evaluation process and an aggregation process for the read answer information, and stores information representing the results of the evaluation process and the aggregation process in the evaluation aggregation information storage unit 34 in association with the respondent's identification information and the question information's identification information.

[0034] In response to a request from a respondent or an evaluator, the evaluation tally information output processing unit 15 selectively reads out information representing the evaluation results and information representing the tally results from the evaluation tally information storage unit 34. Then, the read-out information representing the evaluation results or information representing the tally results is transmitted from the communication I / F unit 4 to the user terminals UT1 to UTn used by the requesting respondent or to an administrator terminal (not shown) used by the evaluator.

[0035] (Example of Operation) Next, an example of operation of the evaluation device SV configured as above will be described.

[0036] FIG. 4 is a flowchart showing an example of the overall processing procedure and processing contents of the evaluation processing executed by the control unit 1 of the evaluation device SV.

[0037] (1) Presentation of Question Information and Information Representing Choices and Answer Fields: Suppose a communication link is established between the user terminal (e.g., UT1) of the respondent to be evaluated and the evaluation device SV, and the respondent inputs a request to start evaluation in this state. In response to this, when the control unit 1 of the evaluation device SV receives the request to start evaluation in step S10, it subsequently executes the presentation process of the question information, its choices, and information representing the answer fields under the control of the question information presentation processing unit 11 and the option presentation processing unit 12, respectively, in steps S11 and S12 as follows:

[0038] That is, first, the question information presentation processing unit 11 reads out a plurality of pieces of pre-prepared question information from the question information storage unit 31 in any order, and transmits the read out question information from the communication I / F unit 4 to the user terminals UT1 to UTn used by the respondents to be evaluated.

[0039] Next, the option presentation processing unit 12 reads out information representing the options and answer fields associated with the identification information of the transmitted question information from the option storage unit 32. Then, the read-out information representing the options and answer fields is transmitted from the communication I / F unit 4 to the user terminal UT1 of the respondent.

[0040] Incidentally, the problem information storage unit 31 stores in advance problem information corresponding to, for example, the four basic units of programming thinking, namely, "sequential," "repetitive," "branching," and "variables." The option storage unit 32 stores in advance information representing options and answer columns corresponding to each of the above-mentioned problem information of "sequential," "repetitive," "branching," and "variables."

[0041] The question information presentation processing unit 11 and the option presentation processing unit 12 each selectively read out question information, its options, and information representing the answer field from the above four units in any order, and transmit the read out question information, its options, and information representing the answer field to the user terminal UT1.

[0042] An example of the process of presenting information representing question information, options, and answer columns for each of the four units will be described below.

[0043] (1-1) When "Sequential" is selected When "Sequential" is selected, the problem information presentation processing unit 11 transmits, for example, the problem information shown in Fig. 7 to the user terminal UT1. As a result, the user terminal UT1 displays on the display device a pair of a first state (a) representing the state before the change and a second state (b) resulting from the change after the execution of a plurality of instructions, which is a process corresponding to "sequential."

[0044] The option presentation processing unit 12 also transmits to the user terminal UT1 information representing the options and answer fields corresponding to the "sequential" question information, for example, as shown in Figures 8 and 9. As a result, the display device of the user terminal UT1 further displays information representing the options and answer fields in addition to the presented question information. Note that the list of options shown in Figure 9 also includes options representing commands that are not necessary to realize the state change process shown in Figure 7.

[0045] For example, in this case, the answer field displays an answer input field for command statements corresponding to each of the commands corresponding to the multiple processes in the problem information corresponding to the above "sequential." Also, the options display a list of multiple options expressing the commands necessary to realize the above processes corresponding to the above "sequential."

[0046] When the respondent selects an option in the answer field, the next answer field is displayed, allowing the respondent to select the next command. By repeating this operation and selecting the option that indicates the end of the answer, the next answer field is no longer displayed, and the answering operation is completed. Note that, in the above operation, if an option that includes a blank space is selected, a question and answer field are displayed immediately afterwards, requiring the respondent to select or enter the appropriate content to fill in the blank. Examples of answering methods at this time include selecting from a list, selecting using check boxes, and free description.

[0047] (1-2) When "Repeat" is selected When "Repeat" is selected, the problem information presentation processing unit 11 transmits, for example, the problem information shown in Fig. 10 to the user terminal UT1. As a result, the user terminal UT1 displays on its display device a pair of a first state (a) representing the state before the change and a second state (b) resulting from the change after the execution of a plurality of instructions representing the process corresponding to "Repeat."

[0048] The option presentation processing unit 12 also transmits to the user terminal UT1, for example, information representing the answer fields and options corresponding to the "repeat" question information as shown in Figures 11 and 12. As a result, the display device of the user terminal UT1 further displays information representing the options and answer fields in addition to the presented question information. Note that the list of options shown in Figure 12 also includes options representing commands that are not necessary to realize the state change process shown in Figure 10.

[0049] For example, in this case, the answer field displays answer input fields for command statements corresponding to each of the commands representing the multiple processes in the problem information corresponding to the above "repetition." Also, the options display a list of multiple options representing the commands necessary to realize the process corresponding to the above "repetition."

[0050] When the respondent selects an option in the answer field, the next answer field is displayed, allowing the respondent to select the next command. By repeating this operation and selecting the option that indicates the end of the answer, the next answer field is no longer displayed, and the answering operation is completed. Note that, in the above operation, if an option that includes a blank space is selected, a question and answer field are displayed immediately afterwards, requiring the respondent to select or enter the appropriate content to fill in the blank. Examples of answering methods at this time include selecting from a list, selecting using check boxes, and free description.

[0051] (1-3) When "Branch" is Selected When "Branch" is selected, the problem information presentation processing unit 11 transmits, for example, the problem information shown in Fig. 13 to the user terminal UT1. As a result, the display device of the user terminal UT1 displays a first state (a) representing the state before the change, and second states (b1), (b2) and (c1), (c2) after the branch that have changed after executing multiple instructions representing a process equivalent to "branch."

[0052] Furthermore, the option presentation processing unit 12 transmits to the user terminal UT1, for example, information representing the answer field and options corresponding to the "branch" question information as shown in Figures 14 and 15. As a result, the display device of the user terminal UT1 further displays information representing the options and answer field in addition to the presented question information. Note that the list of options shown in Figure 15 also includes options representing commands that are not necessary to realize the state change process shown in Figure 13.

[0053] For example, in this case, the answer field displays an answer input field and check boxes for command statements representing each of the commands corresponding to the multiple processes in the problem information corresponding to the above "branch." Also, the options display a list of multiple options representing the commands required to realize the process corresponding to the above "branch."

[0054] When the respondent selects an option in the answer field, the next answer field is displayed, allowing the respondent to select the next command. By repeating this operation and selecting the option that indicates the end of the answer, the next answer field is no longer displayed, and the answering operation is completed. Note that, in the above operation, if an option that includes a blank space is selected, a question and answer field are displayed immediately afterwards, requiring the respondent to select or enter the appropriate content to fill in the blank. Examples of answering methods at this time include selecting from a list, selecting using check boxes, and free description.

[0055] (1-4) When "Variable" is selected When "Variable" is selected, the problem information presentation processing unit 11 transmits, for example, the problem information shown in Fig. 16 to the user terminal UT1. As a result, the display device of the user terminal UT1 displays a first state (a) representing the state before the change and a second state (b) resulting from the change after the execution of a plurality of instructions representing the process corresponding to the "variable."

[0056] The option presentation processing unit 12 also transmits to the user terminal UT1 information representing the options and answer fields corresponding to the "variable" question information, for example, as shown in Figures 17 and 18. As a result, the display device of the user terminal UT1 further displays information representing the options and answer fields in addition to the presented question information.

[0057] For example, in this case, the answer field displays answer input fields and check boxes for command statements corresponding to each of the commands representing the multiple processes in the problem information corresponding to the above "variables." Also, the options display a list of multiple options representing the commands required to realize the processes corresponding to the above "variables."

[0058] When the respondent selects an option in the answer field, the next answer field is displayed, allowing the respondent to select the next command. By repeating this operation and selecting the option that indicates the end of the answer, the next answer field is no longer displayed, and the answering operation is completed. Note that, in the above operation, if an option that includes a blank space is selected, a question and answer field are displayed immediately afterwards, requiring the respondent to select or enter the appropriate content to fill in the blank. Examples of answering methods at this time include selecting from a list, selecting using check boxes, and free description.

[0059] (2) Entering an answer The respondent selects an answer to the command required for the process of realizing the displayed problem information from the list of options for each command, for example by clicking, on his / her user terminal UT1, and enters the answer in the corresponding input field in the answer column.

[0060] FIG. 5 is a flowchart showing an example of the procedure and content of the response input process executed in the user terminal UT1.

[0061] That is, when the user terminal UT1 starts the process of inputting an answer to an instruction representing a process, first in step S21, the answer selected by the answerer from among multiple options for one of the displayed instructions is entered into the corresponding answer field. Then in step S22, it determines whether or not the answer end flag has been selected, and if the answer end flag has not been selected, it then determines in step S23 whether or not there is a blank in the sentence of the option. If there is a blank as a result of this determination, in step S24, the user terminal UT1 enters the answer selected by the answerer from among the multiple options into the blank in the sentence of the option.

[0062] Similarly, the user terminal UT1 repeatedly executes the processes from step S20 to step S24, and when answers have been entered into all answer fields and the answer end flag is finally selected and input, the answer input process ends.

[0063] The answer input process will be explained below using examples of each of the four units "sequential", "repetitive", "branch" and "variable" mentioned above.

[0064] (2-1) Entering an answer to problem information corresponding to “sequential” Figure 19 shows an example of an answer result selected and entered for each process command included in problem information corresponding to “sequential”, where (a) shows the first state before the change, (b) shows the second state after the change, and (c) shows the entered answer result.

[0065] This example shows a case where the answer to the command to realize the process required to enter data for the "Employee Number," "Department Code," and "Branch Name" displayed in the first state (a), and finally click the "Create" button, is selected and entered from a list of options. Also, in this example, the "fields" and "values" included in the command content are entered by checking the corresponding checkboxes. Finally, by answering the option "End answer," the input end flag is set and the answer is completed.

[0066] When the process of inputting answers into all the above-mentioned fields is completed, the user terminal UT1 returns the answer information input into each of the above-mentioned answer fields to the evaluation device SV via the network NW.

[0067] (2-2) Entering answers to problem information corresponding to “repetition” Figure 20 shows an example of answer information entered for each process command of problem information corresponding to “repetition,” where (a) shows the first state before the change, (b) shows the second state after the change, and (c) shows the entered answer result.

[0068] This example shows a case where the answer to the command to realize the process required to sequentially check the checkboxes corresponding to multiple "products" displayed in the first state (a) and finally click the registration button is selected and input from a list of options. Also, in this example, the number of times the process is repeated is input by selecting a number displayed in the input field. Then, "Click the registration button" is selected and input as the command to be executed after all checkboxes have been checked. Finally, by answering the option "End answer," the input end flag is set and the answer is completed.

[0069] In this case as well, when the process of inputting answers into all the above-mentioned fields is completed, the user terminal UT1 returns the answer information input into each of the above-mentioned answer fields to the evaluation device SV via the network NW.

[0070] (2-3) Entering answers to problem information corresponding to "branch" Figure 21 shows an example of answer information entered for the instructions of each process included in problem information corresponding to "branch", where (a) shows the first state before the change, (b1), (b1) and (c1), (c1) show the second state after the change in each branched process, and (c) shows the entered answer result.

[0071] This example shows a case where an answer to a command for realizing a branching process required to input "yen" and "JPY" and "US dollar" and "USD" respectively into the fields corresponding to "Currency (Japanese)" and "Currency (abbreviation)" displayed in the first state (a) is selected and input from a list of options. Also, in this example, the "value" and "condition" included in the command content are input by checking the corresponding checkboxes. Finally, by answering the option "End answer," the input end flag is set and the answer is completed.

[0072] In this case as well, when the process of inputting answers into all the above-mentioned fields is completed, the user terminal UT1 returns the answer information input into each of the above-mentioned answer fields to the evaluation device SV via the network NW.

[0073] (2-4) Entering an answer to problem information corresponding to a "variable" Figure 22 shows an example of answer information entered for the command of each process included in the problem information corresponding to a "variable", where (a) shows the first state before the change, (b) shows the second state after the change, and (c) shows the entered answer.

[0074] This example shows a case where an answer to a command to realize the process required for the virtual robot to mutually exchange the information written in the "Recipient Information" and "Sender Information" columns displayed in the first state (a) is selected and input from a list of options. Also, in this example, the "value" included in the content of the command for the robot to store the value is input by checking the corresponding checkbox. Finally, by answering the option "End answer," the input end flag is set and the answer is completed.

[0075] In this case as well, when the process of inputting answers into all the above-mentioned fields is completed, the user terminal UT1 returns the answer information input into each of the above-mentioned answer fields to the evaluation device SV via the network NW.

[0076] (3) Acquisition of Answer Information After completing the process of transmitting the question information and the information representing the options and answer fields, the control unit 1 of the evaluation device SV waits for the return of answer information from the destination user terminal UT1. When the user terminal UT1 returns the answer information in this state, under the control of the answer information acquisition processing unit 13, in step S13, the answer information returned from the user terminal UT1 is received via the communication I / F unit 4, and the received answer information is stored in the answer information storage unit 33 in a state where it is associated with the identification information of the question information.

[0077] Therefore, when question information corresponding to each of the above-mentioned "sequential," "repetition," "branching," and "variable" is sequentially sent to the user terminal UT1, answer information for each of these question information is sequentially acquired and stored in the answer information storage unit 33 in a state where it is associated with the respondent's identification information and the identification information of the question information.

[0078] (4) Evaluation Counting of Response Information Next, in step S14, the control unit 1 of the evaluation device SV executes evaluation counting processing of the response information under the control of the evaluation counting processing unit 14 as follows.

[0079] FIG. 6 is a flowchart showing an example of the processing procedure and processing contents of the evaluation compilation process executed by the evaluation compilation processing unit 14.

[0080] That is, for example, when the evaluation tabulation processing unit 14 has finished obtaining answer information for all question information, first in step S141, it reads, for each question information, answers included in the answer information corresponding to that information one by one from the answer information storage unit 33. Then, in step S142, the evaluation tabulation processing unit 14 compares each of the read answers with the corresponding correct answers included in the correct answer information stored in advance in the answer information storage unit 33, and determines whether or not there is a complete match.

[0081] If the result of this judgment is that each answer completely matches the correct answer, the evaluation aggregation processing unit 14 determines in step S143 that the respondent understands the content of the basic unit of programming thinking for the problem information being evaluated, and stores the evaluation result in the evaluation aggregation information storage unit 34 in association with the respondent's identification information and the problem information identification information.

[0082] In response to this, suppose that there is an answer that does not match the correct answer. In this case, in step S144, the evaluation compilation processing unit 14 determines whether the answer information includes the necessary options. If the result of this determination is that the necessary options are included, in step S145, the evaluation compilation processing unit 14 determines that the respondent understands part of the content of the basic unit of computational thinking for the problem being evaluated, and stores the evaluation result in the evaluation compilation information storage unit 34 in association with the respondent's identification information and the problem information identification information.

[0083] On the other hand, if the answer information does not include any of the necessary options, in step S146, the evaluation tabulation processing unit 14 determines that the respondent does not understand the content of the basic unit of computational thinking for the question being evaluated, and stores the evaluation result in the evaluation tabulation information storage unit 34 in association with the respondent's identification information and the question information's identification information.

[0084] Each time the evaluation process for answer information corresponding to one piece of question information is completed, the evaluation tabulation processing unit 14 determines in step S147 whether the number of evaluations has reached or exceeded the number corresponding to the question information being evaluated. That is, it determines whether the evaluation process for answers included in the answer information corresponding to all pieces of question information has been completed. If, as a result of this determination, there is unevaluated answer information remaining, the evaluation tabulation processing unit 14 returns to step S141 and executes the evaluation process for the next piece of answer information in steps S141 to S147.

[0085] Thereafter, in the same manner, the evaluation tabulation processing unit 14 repeatedly executes the above evaluation process for each answer included in each piece of answer information.

[0086] When the evaluation process for the answer information for all the question information is completed, the evaluation counting unit 14 performs counting based on the evaluation results for the answer information in step S148. The counting results are stored in the evaluation counting information storage unit 34 in association with the respondent's identification information.

[0087] The aggregation process may be performed according to the following aggregation process rules, for example.

[0088] (1) Counting of correct answer rates The counting process of correct answer rates includes a counting process that evaluates all problem information corresponding to each unit and calculates the percentage of correct answers, and a counting process that calculates the percentage of options selected that correspond to the process required to process the unit for each problem information to be evaluated. Note that the percentage of incorrect answers may be counted instead of the percentage of correct answers.

[0089] (2) A process of aggregating the correct answer rate for each respondent and evaluating the respondent's level of understanding of computational thinking. In this case, the incorrect answer rate may be aggregated instead of the correct answer rate.

[0090] (3) A process of aggregating the correct answer rate or incorrect answer rate for each piece of problem information and evaluating the difficulty level of each piece of problem information.

[0091] (4) A process of aggregating responses by respondent attributes, such as age group, occupation, and affiliated organization (including, for example, type of school and grade), and evaluating the relationship between response trends and attributes.

[0092] (5) For example, based on the response information for two pieces of question information A and B, the number of respondents who answered question A correctly but question B incorrectly and the number of respondents who answered question A incorrectly but question B correctly are tallied, and the order of understanding of questions A and B is evaluated based on this.

[0093] (5) Evaluation Completion Determination The control unit 1 of the evaluation device SV determines in step S15 whether the evaluation process for all the answers to the question information has been completed, for example, each time the presentation of one piece of question information, the acquisition of answer information for that piece of question information, and the evaluation aggregation process are completed. If, as a result of this determination, there are answers corresponding to unevaluated question information remaining, the process returns to step S11, selects an answer to the next piece of question information, and executes the series of evaluation processes from step S11 to step S15 for that answer information. Then, when the evaluation process for all the answers to the question information has been completed, the process proceeds to outputting the evaluation aggregation information.

[0094] (6) Output of Evaluation Aggregation Information When the evaluation aggregation process is completed, the control unit 1 of the evaluation device SV proceeds to step S16. Then, under the control of the evaluation aggregation information output processing unit 15, the process of outputting evaluation aggregation information is executed as follows.

[0095] For example, the evaluation tally information output processing unit 15 receives a request to transmit evaluation tally results from one of the respondent's user terminals UT1-UTn or the evaluator's administrator terminal. In this case, the evaluation tally information output processing unit 15 reads information representing the corresponding evaluation results or tally results from the evaluation tally information storage unit 34 in accordance with the content of the received transmission request. The evaluation tally information output processing unit 15 then transmits the read information representing the evaluation results or tally results from the communication I / F unit 4 to the requesting user terminal UT1-UTn or the evaluator's administrator terminal.

[0096] The information representing the evaluation results and the tabulation results may be transmitted automatically after the evaluation tabulation process is completed, without waiting for a transmission request from the respondent or the administrator. Also, only the respondent's own evaluation results may be transmitted to the respondent, and the evaluation results and the tabulation results for all respondents may be transmitted to the administrator.

[0097] (Effects) As described above, in one embodiment, the evaluation device SV transmits and displays to the respondent's user terminals UT1 to UTn problem information consisting of a first state before change and a second state resulting from a change from the first state through processes corresponding to the four basic units of programmatic thinking, as well as information representing a list of options and answer fields corresponding to the problem information. The evaluation device SV then acquires answer information from the user terminals UT1 to UTn that the respondent has input by selecting from the list of options for the processes included in the problem information, and evaluates and tallies the acquired answer information.

[0098] Therefore, it is possible to objectively evaluate the respondent's computational thinking without being influenced by the subjectivity of the evaluator, etc. Moreover, because the questionnaire format presents a list of multiple options written in natural language that correspond to the commands for each process, and the respondent selects an answer from this list of options, the respondent is able to answer without needing special expertise such as knowledge of programming languages, or the need to learn new operations, and this makes it possible to effectively evaluate the respondent's basic computational thinking without being influenced by other evaluation elements.

[0099] In addition, since the question information, options, and answer fields can be configured using only a general questionnaire response format, an evaluation system can be designed using existing questionnaire forms, thereby reducing the cost required for conducting evaluations.Furthermore, by utilizing existing questionnaire forms, it becomes easier to implement an evaluation test function in the evaluation device SV, making it possible to conduct evaluation tests in a wider variety of situations.

[0100] [Other Embodiments] (1) In one embodiment, information representing problem information, options, and answer fields is prepared corresponding to the four basic units of computational thinking, but information representing problem information, options, and answer fields corresponding to units other than the above four units may also be prepared. Furthermore, a common option list may be prepared for all problem information. Furthermore, the specific configuration and display layout of the information representing problem information, options, and answer fields may be configured in any manner.

[0101] (2) In the embodiment, all the processes of the evaluation device SV are executed on a server computer located on the web or in the cloud. However, the processes may be executed on a personal computer used by a system administrator in addition to the server computer. Furthermore, each processing unit of the evaluation device SV may be distributed among multiple server computers or personal computers that can transfer data between them via a network NW.

[0102] (3) In addition, the procedures and contents of the processes executed by the processing units of the evaluation device can be modified in various ways without departing from the spirit of the present invention.

[0103] Although the embodiments of the present invention have been described in detail above, the above description is merely an example of the present invention in every respect. It goes without saying that various improvements and modifications can be made without departing from the scope of the present invention. In other words, when implementing the present invention, specific configurations according to the embodiments may be appropriately adopted.

[0104] In short, this invention is not limited to the above-described embodiments, and the components can be modified and embodied in practice without departing from the spirit of the invention. Furthermore, various inventions can be formed by appropriately combining multiple components disclosed in the above-described embodiments. For example, some components may be omitted from all the components shown in the embodiments. Furthermore, components from different embodiments may be appropriately combined.

[0105] SV: Evaluation device UT1 to UTn: User terminal NW: Network 1: Control unit 2: Program storage unit 3: Data storage unit 4: Communication I / F unit 5: Bus 11: Question information presentation processing unit 12: Choice presentation processing unit 13: Answer information acquisition processing unit 14: Evaluation aggregation processing unit 15: Evaluation aggregation information output processing unit 31: Question information storage unit 32: Choice storage unit 33: Answer information storage unit 34: Evaluation aggregation information storage unit

Claims

1. An evaluation device connected to a display device and an input device, comprising: a first processing unit that displays on the display device a pair of information representing a first state and information representing a second state after a change from the first state; a second processing unit that displays on the display device a plurality of options corresponding to commands or values ​​required to realize a process required to change from the first state to the second state; a third processing unit that acquires, for each of the processes, an answer created by a respondent selecting and inputting in order from among the plurality of options for the command or the value on the input device; and a fourth processing unit that evaluates, for each of the processes, the answer against pre-stored correct answer information and outputs evaluation information representing the evaluation result.

2. The evaluation device according to claim 1, wherein the second processing unit displays a plurality of the options described in a natural language.

3. The evaluation device according to claim 1, wherein the second processing unit generates an answer field corresponding to the command or the value corresponding to the process, and displays a plurality of the options in association with the answer field.

4. The evaluation device of claim 1, wherein the second processing unit defines at least one of sequence, repetition, branching, and variables as the process, and displays a plurality of options representing the instructions or values ​​corresponding to the defined process.

5. The evaluation device described in claim 1, wherein the fourth processing unit further executes at least one of a first aggregation process that aggregates the correct answer rate or the incorrect answer rate for each respondent to generate information representing the ability of the individual respondent, a second aggregation process that aggregates the correct answer rate or the incorrect answer rate for each process to generate information representing the difficulty level of each process, and a third aggregation process that aggregates the correct answer rate or the incorrect answer rate for each attribute of the respondent to generate information representing the relationship between the attributes of the respondent and the answer result, and outputs the aggregated information generated as a result of this execution together with the evaluation information.

6. The evaluation device of claim 1, wherein the fourth processing unit further executes a fourth aggregation process in which, when at least two of sequential, repetitive, branching, and variable processes are prepared as the processes, the correct answer rate or the incorrect answer rate is calculated for each execution order of the prepared processes, thereby generating information representing a degree of understanding of the execution order of the processes, and outputs the aggregated information generated as a result of this execution together with the evaluation information.

7. A method for evaluating computational thinking executed by an evaluation device connected to a display device and an input device, comprising: a first step of displaying on the display device a pair of information representing a first state and information representing a second state after a change from the first state; a second step of displaying on the display device a plurality of options corresponding to commands or values ​​required to realize a process required to change from the first state to the second state; a third step of acquiring, for each of the steps, an answer created by a respondent selecting and inputting in order from among the plurality of options for the command or the value on the input device; and a fourth step of evaluating, for each of the processes, the answer against pre-stored correct answer information and outputting evaluation information representing the evaluation result.

8. A program for causing a processor included in the evaluation device to execute at least one of the processes executed by the first to fourth processing units included in the evaluation device according to any one of claims 1 to 6.

Images