Information processing device and program

Abstract

To provide a technique for reducing a burden of an instructor by rubric evaluation.SOLUTION: According to an aspect of this disclosure, an information processing device includes a question sentence generation unit for performing rubric evaluation of a first learner's answer to generate a question sentence to request for generation of a rubric evaluation result according to evaluation items and an evaluation standard for performing rubric evaluation of the first learner's answer to a first question sentence; and a result output unit for generating and outputting the rubric evaluation result including scores of the first learner's answer by inputting the question sentence to a trained model.SELECTED DRAWING: Figure 15

Description

Technical Field

[0001] The present disclosure relates to an information processing apparatus and a program.

Background Art

[0002] With the increasing importance of active learning such as "discussion" and "experience / demonstration", rubric evaluation, which evaluates the degree of learning achievement of learners from viewpoints such as thinking ability and expression ability, has attracted attention as a method for evaluating learning.

[0003] For example, Patent Document 1 discloses a system for creating test questions including items for diagnosing "knowledge / skills", "thinking ability / judgment ability / expression ability", etc. based on a rubric.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] Generally, a rubric is composed of a plurality of evaluation items and evaluation criteria, and a leader such as a teacher who guides learners subjectively performs rubric evaluation on the answers of learners according to these evaluation items and evaluation criteria. Therefore, the leader is under a high load due to rubric evaluation.

[0006] In view of the above, an object of the present disclosure is to provide a technique for reducing the load on a leader due to rubric evaluation.

Means for Solving the Problems

[0007] An information processing apparatus according to one aspect of the present disclosure includes a question sentence generation unit that generates a question sentence for obtaining a rubric evaluation result by performing a rubric evaluation of a first learner answer according to evaluation items and evaluation criteria for the rubric evaluation of the first question sentence, and a result output unit that inputs the question sentence into a learned model and generates and outputs a rubric evaluation result including a score of the first learner answer.

Effect of the Invention

[0008] According to one aspect of the present disclosure, the load on the instructor by rubric evaluation can be reduced.

Brief Description of the Drawings

[0009]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Figure 18

Mode for Carrying Out the Invention

[0010] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. However, the scope of the invention is not limited to the illustrated examples. In the following description, those having the same functions and configurations are denoted by the same reference numerals, and the description thereof is omitted.

[0011] [Embodiment] [Overview] According to an embodiment of the present disclosure, an information processing apparatus (for example, the user device 10 described later) includes a question sentence generation unit and a result output unit. The question sentence generation unit generates a question sentence that requests to generate a rubric evaluation result of a learner's answer by performing a rubric evaluation of the learner's answer according to evaluation items and evaluation criteria for rubric evaluation of the learner's answer to a problem sentence. The result output unit includes a result output unit that inputs the generated question sentence into a learned model (for example, a large language model) and generates and outputs a rubric evaluation result including a score of the learner's answer.

[0012] In this way, the information processing apparatus can generate and output a rubric evaluation result of the learner's answer by having the learned model perform a rubric evaluation of the learner's answer, so that the burden on the instructor due to the rubric evaluation can be reduced.

[0013] (First Embodiment) <Configuration of Learner Answer Evaluation System> FIG. 1 is a block diagram showing a schematic configuration example of a learner answer evaluation system according to the first embodiment of the present disclosure.

[0014] As shown in FIG. 1, the learner answer evaluation system 1 includes a user device 10, an information management device 20, and an answer generation device 30. The user device 10 and the information management device 20 are communicably connected to each other via a network 40, and the user device 10 and the answer generation device 30 are communicably connected to each other via the network 40. The network 40 may be a LAN (Local Area Network), the Internet, an intranet, a VPN (Virtual Private Network), a mobile communication network, etc. or a combination thereof.

[0015] The learner answer evaluation system 1 presents an answer (an evaluation result of the learner's answer) to an input (an instruction for evaluating the learner's answer) from a user (an instructor such as a teacher) of the user device 10 to the user of the user device 10. Here, the answer is an evaluation result of the learner's answer by rubric evaluation.

[0016] The user device 10 is configured to be connectable to the information management device 20 and the answer generation device 30, for example, via a dedicated application installed in the user device 10 or a web browser.

[0017] The user of the user device 10 pre-registers user information including a user ID and a password (hereinafter also referred to as login information) with the information management device 20 in order to use the learner answer evaluation service provided in the learner answer evaluation system 1. In the following, it is assumed that the user of the user device 10 has registered user information with the information management device 20.

[0018] The user device 10 transmits the user's login information input by the user of the user device 10 to the information management device 20 via the network 40. If the authentication process using the login information by the information management device 20 is successful, the user of the user device 10 can use the learner answer evaluation service. In the following, it is assumed that the user of the user device 10 and the information management device 20 are successful in authentication in the authentication process.

[0019] The user device 10 transmits, via the network 40, to the information management device 20 a problem statement input by the user of the user device 10, and evaluation items and evaluation criteria for rubric evaluation of the learner answer to the problem statement. When the problem statement, the evaluation items, and the evaluation criteria are transmitted to and registered (stored) in the information management device 20, the learner being guided by the user of the user device 10 can register an answer (learner answer) to the problem statement with the information management device 20 through another user device for learners.

[0020] In response to an input (e.g., pressing a predetermined button) from the user of the user device 10 for evaluating the learner's answer from the user of the user device 10, the user device 10 receives, from the information management device 20 via the network 40, the question text, evaluation items, and evaluation criteria registered in the information management device 20, and information regarding the learner being tutored by the user of the user device 10 (e.g., information including the learner's answer stored in the learner answer storage unit 213 shown in FIG. 8 described later).

[0021] After that, the user device 10 receives the learner's answer to be evaluated from the user of the user device 10. In response to an evaluation instruction (e.g., pressing a predetermined button) for the learner's answer from the user of the user device 10, the user device 10 generates a question text (which may be referred to as an instruction text, a prompt, etc.) for requesting an evaluation of the learner's answer based on the received input, the corresponding question text, evaluation items, and evaluation criteria. The user device 10 transmits the generated question text to the answer generation device 30 via the network 40.

[0022] The user device 10 obtains an answer by receiving, from the answer generation device 30 via the network 40, an answer (evaluation result of the learner's answer) to the transmitted question text. The user device 10 outputs the evaluation result of the learner's answer (e.g., displays the evaluation result of the learner's answer on a display device (display unit) provided in the user device 10).

[0023] Although one user device 10 is shown in FIG. 1, a plurality of user devices 10 may exist. The user device 10 may be referred to as an information terminal, a user terminal, a terminal, a client, a client device, etc.

[0024] The information management device 20 is installed, for example, in a cloud environment.

[0025] The information management device 20 receives the login information of the user of the user device 10 from the user device 10 via the network 40. The information management device 20 executes an authentication process using the received login information.

[0026] The information management device 20 receives a problem statement, evaluation items, and evaluation criteria from the user device 10 via the network 40.

[0027] The information management device 20 transmits to the user device 10 via the network 40 the problem statement, evaluation items, and evaluation criteria registered in the information management device 20, and information regarding the learner being tutored by the user of the user device 10.

[0028] The answer generation device 30 is installed, for example, in a cloud environment.

[0029] The answer generation device 30 receives a question from the user device 10 via the network 40. The answer generation device 30 generates an answer (evaluation result of the learner's answer) for the received question. Specifically, the answer generation device 30 uses a learned large language model (LLM) to generate an answer for the received question. The answer generation device 30 transmits the generated answer to the user device 10 via the network 40.

[0030] Note that the answer generation device 30 may be a device that provides existing conversational AI services such as ChatGPT provided by OpenAI or Bard provided by Google (registered trademark). In this case, the user device 10 can transmit a question to the answer generation device 30 using an API for using such a conversational AI service and receive an answer to the question from the answer generation device 30. Alternatively, the answer generation device 30 may provide a conversational AI service by training the above-mentioned LLM using learning data such as text data published on the web.

[0031] <Configuration of User Device> FIG. 2 is a block diagram showing an example of the hardware configuration of a user device according to the first embodiment of the present disclosure. The user device 10 may be implemented by any computer (such as a tablet, a smartphone, or a notebook PC (Personal Computer)) or information processing device.

[0032] For example, as shown in FIG. 2, the user device 10 may include a storage device 11, a processing device 12, a user interface (UI) device 13, a communication device 14, and a bus 15. The storage device 11, the processing device 12, the UI device 13, and the communication device 14 may be connected to each other via the bus 15.

[0033] A program or instruction for realizing the functions and processes described above and below of the user device 10 may be downloaded from some external device (such as a server) via a network or the like. Also, such a program or instruction may be provided from a removable storage medium such as a CD-ROM (Compact Disc Read Only Memory) or a flash memory.

[0034] The storage device 11 is realized by a RAM (Random Access Memory), a flash memory, a hard disk drive, or the like, and stores files, data, etc. used for the execution of the program or instruction together with the installed program or instruction. The storage device 11 may include a non-transitory storage medium.

[0035] The processing device 12 may be realized by, for example, a general-purpose processor or controller (circuit), or may be realized by a dedicated processor or controller (circuit). When realized by a general-purpose processor, the processing device 12 may be realized by one or more CPUs (Central Processing Units), GPUs (Graphics Processing Units), processing circuitry, etc. that can be composed of one or more processor cores. In this case, the processing device 12 executes the functions and processes of the user device 10 described above and below according to data such as programs or instructions stored in the storage device 11 and parameters used to execute the programs or instructions. Such a program may be a program that causes the computer or processing device 12 to function as the user device 10 according to the present disclosure when executed by the computer or processing device 12. Also, such a program may be a program that causes the computer or processing device 12 to execute the method related to the evaluation of the learner's answer according to the present disclosure when executed by the computer or processing device 12.

[0036] The UI device 13 may be composed of input devices such as a keyboard, mouse, camera, microphone, output devices such as a display, speaker, headset, printer, and input / output devices such as a haptic device and a touch panel, and realizes an interface between the user and the user device 10. For example, the user operates the user device 10 by operating the graphical user interface (GUI) displayed on the display or touch panel using a keyboard, mouse, etc.

[0037] The communication device 14 may be realized by various communication circuits that execute communication processing with external devices, communication networks such as the Internet, LAN, and VPN.

[0038] The above-described hardware configuration of the user device 10 is merely an example, and the user device 10 according to the present disclosure may be implemented by other appropriate hardware configurations.

[0039] FIG. 3 is a block diagram showing an example of the functional configuration of a user device according to the first embodiment of the present disclosure.

[0040] For example, as shown in FIG. 3, the user device 10 includes an input reception unit 101, an evaluation-related information output unit 102, a learner answer acquisition unit 103, a question sentence generation unit 104, an answer acquisition unit 105, and a display control unit 106. The input reception unit 101, the evaluation-related information output unit 102, the learner answer acquisition unit 103, the question sentence generation unit 104, the answer acquisition unit 105, and the display control unit 106 may be realized as hardware by the processing device 12, may be realized as software (for example, a dedicated application), or may be realized as a combination of hardware and software.

[0041] The input reception unit 101 receives an input (for example, login information, question sentences, evaluation items, evaluation criteria, learner answers to be evaluated, etc.) from the user of the user device 10. When the input is login information, the input reception unit 101 transmits the received input to the information management device 20 via the communication device 14. When the input is a question sentence, an evaluation item, and an evaluation criterion, the input reception unit 101 outputs the received input to the evaluation-related information output unit 102. When the input is a learner answer to be evaluated, the input reception unit 101 outputs the received input to the question sentence generation unit 104. Regarding the input of the learner answer to be evaluated, the user of the user device 10 may select the learner answer to be evaluated (by checking a checkbox) from a list of learner answers displayed on the display device (display unit) of the user device 10, copy and paste it, etc., and the input reception unit 101 may receive the learner answer to be evaluated. The input reception unit may be referred to as an input acquisition unit.

[0042] The evaluation-related information output unit 102 transmits, via the communication device 14, the problem statement, evaluation items, and evaluation criteria, which are the inputs output by the input reception unit 101, to the information management device 20.

[0043] In response to an input for evaluating a learner's answer (e.g., pressing a predetermined button) from the user of the user device 10, the learner answer acquisition unit 103 receives, from the information management device 20 via the communication device 14, the problem statement, evaluation items, and evaluation criteria associated with the user of the user device 10, and information about each learner (including the learner's answer) that the user of the user device 10 is instructing (associated with the user of the user device 10). By doing so, the learner answer acquisition unit 103 acquires the problem statement, evaluation items, and evaluation criteria, and information about the learner. The learner answer acquisition unit 103 outputs the acquired problem statement, evaluation items, and evaluation criteria to the question statement generation unit 104, and outputs the acquired learner answer (list) to the display control unit 106.

[0044] Based on the input (learner answer to be evaluated) output by the input reception unit 101 and the problem statement, evaluation items, and evaluation criteria output by the learner answer acquisition unit 103, the question statement generation unit 104 generates a question statement including an instruction to obtain an evaluation of the learner answer. In other words, the question statement generation unit 104 generates an instruction to perform a rubric evaluation of the learner answer according to the evaluation items and evaluation criteria for rubric-evaluating the learner answer to the problem statement. The question statement generation unit 104 transmits the generated question statement to the answer generation device 30 via the communication device 14.

[0045] More specifically, the question sentence generation unit 104 generates a question sentence including an instruction to perform a rubric evaluation using the evaluation items and criteria output by the input reception unit 101 on the learner answer to be evaluated output by the input reception unit 101 for the question sentence output by the learner answer acquisition unit 103, and to output the evaluation result. Further, when the question sentence output by the learner answer acquisition unit 103 is accompanied by or includes an image, for example, by using an external image captioning (image caption generation) service, the content represented by the image can be converted into text (referred to as an image description text), and the image description text can be included in (combined with) the question sentence. Image captioning is a technique for explaining the content represented by an image in natural language. For example, in the case of the image 1112 shown in FIG. 11 described later, an image description text such as "Children of different genders and skin colors are standing on the earth, holding hands" can be generated by image captioning. Note that when the answer generation device 30 (the learned model thereof) is capable of interpreting the image, the question sentence generation unit 104 may transmit the image itself to the answer generation device 30 without converting the image into text. As an example, in the case of the question sentence (accompanied by or including an image), evaluation items, and criteria shown in FIG. 11 described later, and in the case of the learner answer shown in FIG. 12 described later, the question sentence generation unit 104 can generate the following question sentence: Please perform a rubric evaluation on the following learner answer based on the evaluation items and criteria. "In the image, children are standing on the earth, holding hands with smiles. It conveys the scene of children from various countries gathering and playing." ■ Evaluation item: Thinking ability · Evaluation criterion: 3 = "Can organize and structure complex events."... ■ Evaluation item: Expressive ability · Evaluation criterion: 3 = "Has uniqueness and can convey a specific answer."... Please refer to the following content when evaluating. ■ Question Please answer by imagining the content conveyed by the image. ※Image prompted: An image of children of different genders and skin colors standing on the earth and holding hands Here, the description "An image of children of different genders and skin colors standing on the earth and holding hands" is the content of the image verbalized through image captioning.

[0046] The answer acquisition unit 105 acquires an answer by receiving, from the answer generation device 30 via the communication device 14, an answer (evaluation result of the learner's answer) to the question sentence transmitted by the question sentence generation unit 104. For example, in the case of the above-described example, the answer acquisition unit 105 may acquire the following answer: 『■Thinking ability: 2 points ·The people and objects shown in the image are captured, and partial organization and structuring of the content have been achieved. ■Expressive ability: 1 point ·It is an abstract answer with low originality. ···』

[0047] The answer acquisition unit 105 outputs the acquired answer to the display control unit 106.

[0048] The display control unit 106 generates a screen including the learner's answer (list) output by the learner answer acquisition unit 103, and causes the display device (display unit) included in the UI device 13 to display the screen (for example, to enable the user of the user device 10 to select the learner's answer to be evaluated). The display control unit 106 generates a screen including the answer output by the answer acquisition unit 105, and causes the display device (display unit) to display the screen. In other words, the display control unit 106 outputs the learner's answer (list) and the answer (to the outside). In addition, the display control unit 106 generates a screen for inputting the question sentence, evaluation items, and evaluation criteria, and causes the display device (display unit) to display the screen.

[0049] Note that the user of the user device 10 can also register, through the user device 10, a comment (evaluator comment) corresponding to the learner's answer in the information management device 20 (see, for example, FIG. 8).

[0050] <Configuration of Information Management Device> FIG. 4 is a block diagram showing a hardware configuration example of an information management device according to the first embodiment of the present disclosure. The information management device 20 may be realized by a computing device (computer, computer) such as a server or an information processing device.

[0051] For example, as shown in FIG. 4, the information management device 20 may include a storage device 21, a processing device 22, a UI device 23, a communication device 24, and a bus 25. The storage device 21, the processing device 22, the UI device 23, and the communication device 24 may be interconnected via the bus 25.

[0052] A program or instruction for realizing the functions and processes described above and below of the information management device 20 may be downloaded from some external device (for example, a server) via a network or the like. Further, such a program or instruction may be provided from a removable storage medium such as a CD-ROM or a flash memory.

[0053] The storage device 21 is realized by a RAM, a flash memory, a hard disk drive, etc., and stores files, data, etc. used for the execution of the installed program or instruction together with the installed program or instruction. The storage device 21 may include a non-transitory storage medium.

[0054] The processing device 22 may be realized by, for example, a general-purpose processor or controller (circuit), or may be realized by a dedicated processor or controller (circuit). When realized by a general-purpose processor, the processing device 22 may be realized by one or more CPUs, GPUs, processing circuits, etc. that may be composed of one or more processor cores. In this case, the processing device 22 executes the functions and processes of the information management device 20 described above and below according to programs or instructions stored in the storage device 21, data such as parameters used to execute the programs or instructions, etc. Such a program may be a program that causes the computer or the processing device 22 to function as the information management device 20 according to the present disclosure when executed by the computer or the processing device 22. Further, such a program may be a program that causes the computer or the processing device 22 to execute the method related to the evaluation of the learner's answer according to the present disclosure when executed by the computer or the processing device 22.

[0055] The UI device 23 may be composed of input devices such as a keyboard, mouse, camera, microphone, etc., output devices such as a display, speaker, headset, printer, etc., input / output devices such as a haptic device, touch panel, etc., and realizes an interface between the user and the information management device 20. For example, the user operates the information management device 20 by operating the GUI displayed on the display or touch panel using a keyboard, mouse, etc.

[0056] The communication device 24 may be realized by various communication circuits that execute communication processing with external devices, communication networks such as the Internet, LAN, VPN, etc.

[0057] The above-described hardware configuration of the information management device 20 is merely an example, and the information management device 20 according to the present disclosure may be realized by other appropriate hardware configurations.

[0058] FIG. 5 is a block diagram showing a functional configuration example of the information management device according to the first embodiment of the present disclosure.

[0059] For example, as shown in FIG. 5, the information management apparatus 20 includes an input acquisition unit 201, an authentication process execution unit 202, a learner response output unit 203, a user information storage unit 211, a problem statement storage unit 212, and a learner response storage unit 213. The input acquisition unit 201, the authentication process execution unit 202, and the learner response output unit 203 may be realized as hardware by the processing device 22, may be realized as software, or may be realized as a combination of hardware and software. The user information storage unit 211, the problem statement storage unit 212, and the learner response storage unit 213 may be realized by the storage device 21.

[0060] The input acquisition unit 201 acquires an input by receiving, from the user device 10 via the communication device 24, an input (login information, problem statement, evaluation item, evaluation criterion, etc.) by the user of the user device 10. When the input is login information, the input acquisition unit 201 outputs the input to the authentication process execution unit 202. When the input is a problem statement, an evaluation item, and an evaluation criterion, the input acquisition unit 201 stores the problem statement, the evaluation item, and the evaluation criterion in the problem statement storage unit 212.

[0061] The authentication process execution unit 202 collates the login information output by the input acquisition unit 201 with the login information among the user information stored in the user information storage unit 211, and determines that the authentication is successful when they match.

[0062] The learner response output unit 203, in response to an input for evaluating the learner response by the user (logged-in user) of the user device 10, acquires from the user information storage unit 211, the problem statement storage unit 212, and the learner response storage unit 213, the problem statement, the evaluation item, and the evaluation criterion associated with the logged-in user, and information (including the learner response) regarding each learner coached by the logged-in user (associated with the logged-in user). The learner response output unit 203 transmits, via the communication device 24, the acquired problem statement, evaluation item, and evaluation criterion, and the information regarding the learner, to the user device 10.

[0063] The user information storage unit 211 stores user information regarding the user of the user device 10.

[0064] FIG. 6 is a diagram showing an example of a table in which the user information stored in the user information storage unit 211 is stored.

[0065] As shown in FIG. 6, the table 250 in which user information is stored may include fields such as a user ID, a password, a surname, a given name, the school in charge (elementary school in the example shown in FIG. 6), and the class in charge. Although not shown in FIG. 6, the users include not only instructors but also learners, and the table 250 in which user information is stored may include a field of information indicating whether the user is an instructor or a learner, and the user ID may be generated using a rule that can distinguish between instructors and learners.

[0066] The problem statement storage unit 212 stores problem statements, evaluation items, and evaluation criteria created by the user of the user device 10.

[0067] FIG. 7 is a diagram showing an example of a table in which the problem statements stored in the problem statement storage unit 212 are stored.

[0068] As shown in FIG. 7, the table 260 in which problem statements are stored may include fields such as a user ID, a problem ID, a problem statement, an image associated with or included in the problem statement (if any), evaluation items, and evaluation criteria. The table 260 and the table 250 are associated with each other by the user ID field in the table 260 and the user ID field in the table 250.

[0069] The learner answer storage unit 213 stores the answers of learners to the problem statements created by the user of the user device 10.

[0070] FIG. 8 is a diagram showing an example of a table in which the learner answers stored in the learner answer storage unit 213 are stored.

[0071] As shown in FIG. 8, the table 270 in which the learner answers are stored may include fields such as a question ID, a user ID (learner identification information), a question sentence, an image associated with or included in the question sentence (if any), an evaluation item, an evaluation criterion, an evaluation result (in the example shown in FIG. 8, evaluation result 1 = the score of the thinking ability described later, evaluation result 2 = the score of the expression ability described later), and an evaluator (instructor) comment. The table 270 and the table 260 are associated with each other by the field of the question ID in the table 270 and the field of the question ID in the table 260 (therefore, the table 270 and the table 250 are also associated with each other). Therefore, the table 270 in which the learner answers are stored may not include fields of the question sentence, the image associated with or included in the question sentence (if any), the evaluation item, and the evaluation criterion.

[0072] <Configuration of Answer Generation Device> FIG. 9 is a block diagram showing a hardware configuration example of an answer generation device according to the first embodiment of the present disclosure. The answer generation device 30 may be realized by a computing device (computer, computer) such as a server or an information processing device.

[0073] For example, as shown in FIG. 9, the answer generation device 30 may include a storage device 31, a processing device 32, a UI device 33, a communication device 34, and a bus 35. The storage device 31, the processing device 32, the UI device 33, and the communication device 34 may be connected to each other via the bus 35.

[0074] The program or instruction for realizing the functions and processes described above and below of the answer generation device 30 may be downloaded from some external device (for example, a server) via a network or the like. Further, such a program or instruction may be provided from a removable storage medium such as a CD-ROM or a flash memory.

[0075] The storage device 31 is implemented by a RAM, a flash memory, a hard disk drive, etc., and stores programs or instructions installed, as well as files, data, etc. used for the execution of the programs or instructions. The storage device 31 may include a non-transitory storage medium.

[0076] The processing device 32 may be implemented by, for example, a general-purpose processor or a controller (circuit), or may be implemented by a dedicated processor or a controller (circuit). When implemented by a general-purpose processor, the processing device 32 may be implemented by one or more CPUs, GPUs, processing circuits, etc. that can be composed of one or more processor cores. In this case, the processing device 32 executes the functions and processes of the answer generation device 30 described above and below according to programs or instructions stored in the storage device 31, data such as parameters used to execute the programs or instructions, etc. Such a program may be a program that causes the computer or the processing device 32 to function as the answer generation device 30 according to the present disclosure when executed by the computer or the processing device 32. Further, such a program may be a program that causes the computer or the processing device 32 to execute the method related to the evaluation of the learner's answer according to the present disclosure when executed by the computer or the processing device 32.

[0077] The UI device 33 may be composed of input devices such as a keyboard, a mouse, a camera, a microphone, output devices such as a display, a speaker, a headset, a printer, and input / output devices such as a haptic device, a touch panel, etc., and realizes an interface between the user and the answer generation device 30. For example, the user operates the answer generation device 30 by operating the GUI displayed on the display or the touch panel using a keyboard, a mouse, etc.

[0078] The communication device 34 may be implemented by various communication circuits that execute communication processing with external devices, communication networks such as the Internet, a LAN, a VPN, etc.

[0079] The hardware configuration of the above-described answer generation device 30 is merely an example, and the answer generation device 30 according to the present disclosure may be implemented by other appropriate hardware configurations.

[0080] FIG. 10 is a block diagram showing a functional configuration example of an answer generation device according to the first embodiment of the present disclosure.

[0081] For example, as shown in FIG. 10, the answer generation device 30 includes a question sentence acquisition unit 301 and an answer processing unit 302. The question sentence acquisition unit 301 and the answer processing unit 302 may be realized as hardware by the processing device 32, may be realized as software, or may be realized as a combination of hardware and software.

[0082] The question sentence acquisition unit 301 acquires a question sentence by receiving the question sentence from the user device 10 via the communication device 34. The question sentence acquisition unit 301 outputs the question sentence to the answer processing unit 302.

[0083] The answer processing unit 302 generates an answer (evaluation result of the learner answer) to the question sentence by inputting the question sentence output by the question sentence acquisition unit 301 into the LLM, which is a learned model. The answer processing unit 302 transmits the generated answer to the user device 10 via the communication device 34.

[0084] <Registration of Question Sentences, Evaluation Items, and Evaluation Criteria> Next, an example of registration of question sentences, evaluation items, and evaluation criteria by the user of the user device 10 will be described.

[0085] FIG. 11 is a diagram showing an example of a screen for registering question sentences, evaluation items, and evaluation criteria in a dialogue (chat) format according to the first embodiment of the present disclosure.

[0086] As shown in FIG. 11, a screen example 1100 is displayed on the display unit of the user device 10. In the screen example 1100, in response to the display of information 1101 prompting the input of a problem statement and information 1102 prompting the input of an image, the user of the user device 10 can input a problem statement (for example, information 1111 such as "Please imagine what the image conveys and answer.") and an image (for example, image 1112 (however, the input of the image is optional)). Also, in the screen example 1100, in response to the display of information 1103 prompting the input of evaluation items and evaluation criteria, the user of the user device 10 can input evaluation items and evaluation criteria 1113. The evaluation items and evaluation criteria 1113 are, as an example, the evaluation items and evaluation criteria shown in Table 1 below. In Table 1, "thinking ability" and "expressive ability" are examples of evaluation items, and "able to organize and structure complex events." (3 points), "lacking originality and being an abstract answer." (1 point), etc. are examples of evaluation criteria.

Table 1

[0087] For example, when the enter key is pressed by the user of the user device 10, the evaluation-related information output unit 102 can transmit the input problem statement, image, evaluation items, and evaluation criteria to the information management device 20 each time.

[0088] <Registration of Learner Answers to Problem Statements> Next, an example of the registration of learner answers to problem statements by learners will be described.

[0089] FIG. 12 is a diagram showing a screen example for registering learner answers to problem statements according to the first embodiment of the present disclosure.

[0090] As shown in FIG. 12, a screen example 1200 is displayed on the display unit of the user device used by the learner. In the screen example 1200, an input field (the "answer field") for the answer to the question sentence (information 1111) and the image (image 1112) illustrated with reference to FIG. 11 is displayed. The learner can register the learner answer by inputting an answer (in the illustrated example, "In the image, children are standing on the earth with smiling faces and holding hands. It shows children from various countries gathering and playing.") in the input field.

[0091] <Instruction to request evaluation of learner answer> Next, an example of an instruction to request the user of the user device 10 to evaluate the learner answer will be described.

[0092] FIG. 13 is a diagram showing a screen example for an instruction to request evaluation of a learner answer in an interactive (chat) format (and including the evaluation result of the learner answer according to the instruction) according to the first embodiment of the present disclosure.

[0093] As shown in FIG. 13, a screen example 1300 is displayed on the display unit of the user device 10. Before the screen example 1300 is displayed, the user of the user device 10 has selected the learner answer illustrated with reference to FIG. 12 as the evaluation target, and the screen example 1300 is displayed according to this selection. In the screen example 1300, a question sentence 1301 including an instruction to rubric-evaluate the learner answer, which is generated by the question sentence generation unit 104 based on the learner answer to be evaluated ("In the image, children are standing on the earth with smiling faces and holding hands. It shows children from various countries gathering and playing."), the question sentence (information 1111), the image (image 1112), the evaluation items, and the evaluation criteria (see Table 1), is displayed.

[0094] For example, when the enter key is pressed by the user of the user device 10, the question sentence generation unit 104 can transmit the generated question sentence 1301 to the answer generation device 30.

[0095] <Evaluation of learner answer> Next, an example of the evaluation of the learner's answer by the user device 10 (and the answer generation device 30) will be described.

[0096] The screen example 1300 in FIG. 13 described above also includes the evaluation result 1311 of the learner's answer according to the above-described instruction (due to the question sentence 1301 being transmitted from the user device 10 to the answer generation device 30). The evaluation result 1311 of the learner's answer is obtained by the answer acquisition unit 105 and is the result of rubric evaluation of the learner's answer considering the content of the question sentence 1301. In the illustrated example, it includes scores for thinking ability and expressive ability (2 points and 1 point respectively).

[0097] <Operation of the Learner Answer Evaluation System> Next, with reference to FIGS. 14 and 15, an operation example of the learner answer evaluation system 1 will be described.

[0098] FIG. 14 is a flowchart showing an operation example of a user device according to the first embodiment of the present disclosure. The flowchart shown in FIG. 14 relates to the registration of question sentences, evaluation items, and evaluation criteria.

[0099] In step S101, the user device 10 (input reception unit 101) receives a question sentence, an evaluation item, and an evaluation criterion, which are inputs from the user of the user device 10.

[0100] In step S102, the user device 10 (evaluation-related information output unit 102) transmits the received question sentence, evaluation item, and evaluation criterion to the information management device 20. As a result, the information management device 20 stores the received question sentence, evaluation item, and evaluation criterion in the question sentence storage unit 212 in association with the user (logged-in user) of the user device 10.

[0101] FIG. 15 is a flowchart showing an operation example of a user device according to the first embodiment of the present disclosure. The flowchart shown in FIG. 15 relates to the acquisition of the evaluation result of the learner's answer.

[0102] In step S201, the user device 10 (learner answer acquisition unit 103) acquires information regarding the learner associated with the user (logged-in user) of the user device 10 from the information management device 20 (question text, evaluation items, evaluation criteria, learner answers, etc.).

[0103] In step S202, the user device 10 (question text generation unit 104) generates a question text regarding the learner answer, including an instruction to obtain an evaluation of the learner answer, based on the learner answer of the evaluation target selected or input by the logged-in user, the question text, the evaluation items, and the evaluation criteria.

[0104] In step S203, the user device 10 (question text generation unit 104) transmits the generated question text to the answer generation device 30. As a result, the answer generation device 30 generates an evaluation result of the received responder answer and transmits it to the user device 10.

[0105] In step S204, the user device 10 (answer acquisition unit 105) acquires an answer (evaluation result of the learner answer) from the answer generation device 30.

[0106] In step S205, the user device 10 (display control unit 106) causes the display unit to display the answer.

[0107] <Effect in the First Embodiment> As described above, the user device 10 generates a question sentence that requests to perform a rubric evaluation of the learner's answer according to the evaluation items and criteria for rubric-evaluating the learner's answer to the problem sentence, and generates a rubric evaluation result. Further, the user device 10 inputs the generated question sentence into the learned LLM, generates and outputs a rubric evaluation result including the score of the learner's answer. The learned LLM may be possessed by the answer generation device 30 as described above, or may be possessed by the user device 10 as described below. In this way, the user device 10 can generate and output the rubric evaluation result of the learner's answer by having the learned LLM perform the rubric evaluation of the learner's answer, so that the burden on the instructor by the rubric evaluation can be reduced.

[0108] (Second Embodiment) Next, a second embodiment of the present disclosure will be described. In the second embodiment, the description will mainly focus on the parts different from the first embodiment. The description of the parts that are the same as or similar to those in the first embodiment will be omitted. In rubric evaluation, there is a possibility that the evaluation varies for each instructor, and the second embodiment also addresses the variation in evaluation for each instructor.

[0109] FIG. 16 is a block diagram showing a functional configuration example of a user device according to a second embodiment of the present disclosure.

[0110] For example, as shown in FIG. 16, the user device 10 according to the second embodiment further includes a similar problem determination unit 107, which is different from the user device 10 according to the first embodiment. The similar problem determination unit 107 may be realized as hardware by the processing device 12, may be realized as software (for example, a dedicated application), or may be realized as a combination of hardware and software.

[0111] In addition to the processing in the first embodiment, when the input is a learner response to be evaluated, the input reception unit 101 outputs the received input, together with the question sentence of the learner response to be evaluated (and, if there is an image associated with or included in the question sentence, the image), to the question sentence generation unit 104 and the similar question determination unit 107.

[0112] In addition to the processing in the first embodiment, the learner response acquisition unit 103 outputs the acquired information about the learner (information stored in the learner response storage unit 213 shown in FIG. 8) to the similar question determination unit 107. Note that the learner response acquisition unit 103 may acquire information about not only the learner associated with the user (logged-in user) of the user device 10 from the information management device 20 but also other learners (for example, all learners) stored in the information management device 20 (that is, the learner response output unit 203 of the information management device 20 may also transmit information about other learners to the user device 10). However, the information about the learner (learner response, etc.) displayed to the user (logged-in user) of the user device 10 may be only the information about the learner associated with the logged-in user.

[0113] The similar question determination unit 107 receives, as inputs, the learner response to be evaluated output by the input reception unit 101 and the information about the learner output by the learner response acquisition unit 103. The similar question determination unit 107 acquires information about a question sentence (referred to as another question sentence) similar to the question sentence (referred to as the target question sentence) of the received learner response to be evaluated from the question sentence (referred to as the comparison target question sentence) included in the received information about the learner. Note that the other question sentence similar to the target question sentence may include the same question sentence as the target question sentence. For example, the similar question determination unit 107 may acquire information about another question sentence similar to the target question sentence (for example, other learner responses to the other question sentence, evaluator comments on the other learner responses, evaluation results (scores, etc.) of the other learner responses, etc.) according to the following procedure: (1) Calculate a feature vector from the target problem sentence using a language processing model such as BERT (Bidirectional Encoder Representations from Transformers) (for example, it may be provided by an external service or the user device 10 itself may be equipped with it). (2) If there is an image associated with or included in the target problem sentence (referred to as the target image), calculate a feature vector from the image using an image analysis model such as VGG (Visual Geometry Group) (for example, it may be provided by an external service or the user device 10 itself may be equipped with it). (3) Calculate a feature vector from the comparison target problem sentence included in the received information about the learner (and, if the above image exists, the comparison target image associated with or included in the comparison target problem sentence) in the same manner as in (1) (and (2)). (4) Calculate the difference between the feature vectors of the target problem sentence (and target image) and the comparison target problem sentence (and comparison target image), and if the difference is less than or equal to the threshold or greater than or equal to the threshold, determine that the comparison target problem sentence (and comparison target image) is another similar problem sentence (and image). (5) Obtain information about other problem sentences determined to be similar (comparison target problem sentences).

[0114] For example, in (4), the similar problem determination unit 107 may determine similarity by calculating a distance such as the Euclidean distance, Manhattan distance, or Hamming distance between the feature vectors calculated in (3) as the difference between the feature vectors (for example, determine similarity when the distance is less than or equal to the threshold). Alternatively, in (4), the similar problem determination unit 107 may determine similarity by calculating the cosine similarity between the feature vectors calculated in (3) as the difference between the feature vectors (for example, determine similarity when the cosine similarity is greater than or equal to the threshold).

[0115] The similar problem determination unit 107 outputs the information about the other problem sentences obtained as described above to the question sentence generation unit 104.

[0116] In this way, by using the information regarding other similar problem statements stored in the learner response memory unit 213, (for example, the more information stored in the learner response memory unit 213,), it becomes possible to suppress the variation in evaluations for each instructor.

[0117] In addition to the processing in the first embodiment, the problem statement generation unit 104 generates a problem statement including an instruction to obtain an evaluation of the learner response based on the input (learner response to be evaluated) output by the input reception unit 101, the problem statement, evaluation items, and evaluation criteria output by the learner response acquisition unit 103, and information regarding other problem statements output by the similar problem determination unit 107. More specifically, the problem statement generation unit 104 generates a problem statement so as to include, in the problem statement, other learner responses and evaluation results for the other problem statements as the information regarding other problem statements output by the similar problem determination unit 107. Further, the problem statement generation unit 104 may generate a problem statement so as to include, in the problem statement, an instruction to obtain expressions leading to high evaluations and answer examples with higher evaluations. For example, the problem statement generation unit 104 may generate a problem statement as follows: "Please perform a rubric evaluation on the following learner response based on the evaluation items and criteria. "In the image, children are standing on the earth with smiles on their faces and holding hands. It conveys the scene of children from various countries gathering and playing." ■ Evaluation item: Thinking ability · Evaluation criterion: 3 = "Can organize and structure complex events."... ■ Evaluation item: Expressiveness · Evaluation criterion: 3 = "Has originality and can convey a specific answer."... Please refer to the following content and the responses and evaluation results of other learners when making the evaluation. Also, please provide feedback on "expressions leading to high evaluations" and "answer examples with higher evaluations". ■ Question Please answer by imagining the content conveyed by the image. ※Image Prompted: An image of children of different genders and skin colors standing on the earth, holding hands ■Answer of Learner 1 "There are many children and they seem to be having fun." ■Evaluation Results Thinking ability: 1 point, Expressive ability: 1 point ■Evaluator's Comment The answer is naive and simple. ■Answer of Learner 2 "In the image, children are standing on the earth, smiling and holding hands. Given the various genders and skin colors of the children, it can be imagined that children from all over the world, regardless of country or race, are gathered together. From the above, it conveys "a peaceful world where there is no discrimination based on country or race and everyone can live with a smile." ■Evaluation Results Thinking ability: 3 points, Expressive ability: 3 points ■Evaluator's Comment The conclusion of "a peaceful world where there is no discrimination based on country or race and everyone can live with a smile" is appropriate considering the world situation. ···』 In the second embodiment, different from the first embodiment, the question text includes instructions to seek other learners' answers and evaluation results, as well as expressions and answer examples leading to high evaluations. By doing so, the answer generation device 30 will perform rubric evaluation by few-shot generation. Note that in the first embodiment as well, the question text generation unit 104 may generate the question text so as to include an instruction to seek expressions leading to high evaluations and answer examples with higher evaluations in the question text.

[0118] Furthermore, the question text generation unit 104 may generate the question text so as to further require, more specifically, identifying the part (expression) that contributed to a score of a predetermined score or higher among the learner answers to be evaluated when the score of the learner answer to be evaluated is equal to or higher than the predetermined score.

[0119] When the above question text is sent to the answer generation device 30, the answer acquisition unit 105 may obtain the following answer: 『■Thinking ability: 2 points · People or objects shown in the image are captured, and the content has been partially organized and structured. ■ Expressiveness: 1 point · It is an abstract answer with low originality. [Expressions Leading to High Evaluation] · Children are smiling and holding hands: The appearance of the characters is captured and is eligible for points in the evaluation of thinking ability. [Examples of Answers with Higher Evaluation] In the image, children are smiling, holding hands, and standing on the earth. Since the genders and skin colors of the children are diverse, it can be imagined that children from all over the world regardless of country or race are gathered. From the above, it conveys "a peaceful world where there is no discrimination based on country or race and everyone can live with a smile". ···』

[0120] FIG. 17 is a diagram showing an example of a screen for obtaining an evaluation of a learner's answer in a dialogue (chat) format according to the second embodiment of the present disclosure and including an evaluation result of the learner's answer according to the instruction.

[0121] As shown in FIG. 17, a screen example 1700 is displayed on the display unit of the user device 10. In the screen example 1700, the user of the user device 10 has selected or input the learner's answer 1701 as an evaluation target. In this example, different from the first embodiment (see FIG. 13), the question sentence generated by the question sentence generation unit 104 is not displayed on the display unit of the user device 10 (of course, the question sentence may be displayed on the display unit of the user device 10). The question sentence is generated by the question sentence generation unit 104 according to the selection or input of the learner's answer 1701 to be evaluated and is transmitted to the answer generation device 30 (for example, in response to the pressing of the enter key).

[0122] The screen example 1700 includes the evaluation results 1711 and 1712 of the learner's answers (due to the question text being sent from the user device 10 to the answer generation device 30 according to the above-described instruction) and a higher-evaluation answer example 1713. The evaluation results 1711 and 1712 of the learner's answers and the higher-evaluation answer example 1713 are the results and answer examples obtained by the answer acquisition unit 105 through rubric evaluation of the learner's answers in consideration of the content of the question text.

[0123] In addition, when there are other question texts similar to the target question text, the answer processing unit 302 of the answer generation device 30 can select an answer with a higher rubric evaluation result from the answers of other learners to the other question texts. On the other hand, when there are no other question texts similar to the target question text, the answer processing unit 302 of the answer generation device 30 can generate an answer that is considered to have a higher evaluation based on the answers and / or evaluation results of other learners.

[0124] FIG. 18 is a flowchart showing an operation example of the user device according to the second embodiment of the present disclosure. The flowchart shown in FIG. 18 relates to obtaining the evaluation results of the learner's answers. In the second embodiment, different from the first embodiment, a new step S301 exists between step S201 and step S202, and step S202 is changed to step S302.

[0125] In step S201, the user device 10 (learner answer acquisition unit 103) obtains information (question text, evaluation items, evaluation criteria, learner answers, evaluation results, evaluator comments, etc.) regarding the learner associated with the user (logged-in user) of the user device 10 from the information management device 20. Note that the information regarding the learner to be obtained may include information regarding the learner associated with the logged-in user.

[0126] In step S301, the user device 10 (similar question determination unit 107) obtains information regarding other question texts similar to the question text of the learner answer to be evaluated selected or input by the logged-in user from the information regarding the learner.

[0127] In step S302, the user device 10 (question sentence generation unit 104) generates a question sentence regarding the learner answer, including an instruction to obtain an evaluation of the learner answer, based on the learner answer to be evaluated, the question sentence, evaluation items and evaluation criteria, and information regarding other question sentences (other learner answers, evaluator comments, evaluation results (scores, etc.)). Note that the generated question sentence may include an instruction to seek expressions that lead to high evaluations or answer examples with higher evaluations.

[0128] In step S203, the user device 10 (question sentence generation unit 104) transmits the generated question sentence to the answer generation device 30. As a result, the answer generation device 30 generates an evaluation result of the received responder answer and transmits it to the user device 10.

[0129] In step S204, the user device 10 (answer acquisition unit 105) acquires an answer (evaluation result of the learner answer) from the answer generation device 30.

[0130] In step S205, the user device 10 (display control unit 106) causes the answer to be displayed on the display unit.

[0131] <Effects in the Second Embodiment> As described above, based on other learner responses to other question texts that are the same as or similar to the question text and the rubric evaluation results of other learner responses, the user device 10 generates a question text that requests to perform a rubric evaluation of the learner response according to the evaluation items and criteria for the learner response to the question text. Further, the user device 10 inputs the generated question text into the learned LLM to generate and output a rubric evaluation result including the score of the learner response. In this way, the user device 10 can generate and output the rubric evaluation result of the learner response by having the learned LLM perform the rubric evaluation of the learner response, so that the burden on the instructor due to the rubric evaluation can be reduced. Furthermore, the user device 10 can suppress the variation in the rubric evaluation for each instructor by having the learned LLM perform the rubric evaluation of the learner response based on other learner responses and the rubric evaluation results of other learner responses.

[0132] <Modification Example> Hereinafter, a modification example of the above embodiment will be described.

[0133] · Modification Example 1 Some functions of the user device 10 may be included in the information management device 20 or the response generation device 30. In this case, the user device 10 may transmit an input from the user to the information management device 20 or the response generation device 30, and the information management device 20 or the response generation device 30 that has received the input may execute processes corresponding to the processes executed by the learner response acquisition unit 103, the question text generation unit 104, the response acquisition unit 105, and / or the similar question determination unit 107, and may transmit the processing result to the user device 10.

[0134] · Modification Example 2 The functions related to answer generation of the answer generation device 30 may be included in the user device 10 or the information management device 20. That is, the LLM, which is a learned model, may be included in the user device 10, or may be arranged on a network to which the user device 10 can be connected, such as the information management device 20 or the answer generation device 30. Therefore, the user device 10 or the information management device 20 may generate a question sentence that requests to perform a rubric evaluation of the learner's answer according to the evaluation items and criteria for rubric-evaluating the learner's answer for the question sentence, input the question sentence into the learned model, and generate and output a rubric evaluation result including the score of the learner's answer.

[0135] · Modification Example 3 Part or all of the various storage units 211 to 213 provided in the information management device 20 may be included in another device (for example, a database device) to which the user device 10 or the information management device 20 can be connected.

[0136] · Modification Example 4 In the above embodiment, it has been described that the user of the user device 10 inputs the evaluation items and criteria. However, it is also possible for the user device 10 or the information management device 20 to automatically specify the evaluation items and criteria from the input question sentence (question content) (see, for example, Japanese Patent Application Laid-Open No. 2023-150275). In this case, the user device 10 or the information management device 20 may control to use the automatically specified evaluation items and criteria as the evaluation items and criteria, or may control to display the automatically specified evaluation items and criteria on the display unit of the user device 10, and cause the user of the user device 10 to input the evaluation items and criteria actually used based on the automatically specified evaluation items and criteria.

[0137] · Modification Example 5 In the above-described embodiment, it has been described that the question sentence generation unit 104 generates a question sentence including an instruction for obtaining an evaluation of the learner's answer. However, the user of the user device 10 may generate a question sentence by freely describing and collectively inputting the content necessary for rubric evaluation such as the learner's answer, evaluation items, and evaluation criteria (for example, the user may input content corresponding to the question sentence 1301 in FIG. 13).

[0138] ·Other Modification Examples The names of the above-described devices (user device 10, information management device 20, etc.) and the functional units constituting the devices (input reception unit 101, evaluation-related information output unit 102, etc.) are merely examples and may be changed as appropriate. Also, two or more of the functional units constituting the device may be combined and integrated, or one functional unit may be divided into functional units. Further, the transmission function and / or reception function possessed by each functional unit may be summarized as a notification unit, a notified unit, a communication unit, a transmission / reception unit, a transmission unit, a reception unit, etc.

[0139] The order of the steps in the flowchart illustrated and described above is merely an example and may be changed as appropriate, or some steps may be executed in parallel.

[0140] As described above, according to one aspect of the present disclosure, the load on the instructor by rubric evaluation can be reduced.

[0141] As described above, the embodiments have been described with reference to the drawings, but the present disclosure is not limited to such examples. It is obvious that those skilled in the art can conceive of various modification examples or correction examples within the scope described in the claims. Such modification examples or correction examples are also understood to belong to the technical scope of the present disclosure. Also, within the scope not departing from the gist of the present disclosure, the components or matters in the embodiments (including modification examples or correction examples) may be arbitrarily combined.

Description of Reference Numerals

[0142] 1 Learner Answer Evaluation System 10 User Device 101 Input Reception Unit 102 Evaluation-related Information Output Unit 103 Learner Answer Acquisition Unit 104 Question Generation Unit 105 Answer Acquisition Unit 106 Display Control Unit 107 Similar Question Judgment Unit 20 Information Management Device 201 Input Acquisition Unit 202 Authentication Processing Execution Unit 203 Learner Answer Output Unit 211 User Information Storage Unit 212 Question Text Storage Unit 213 Learner Answer Storage Unit 30 Answer Generation Device 301 Question Text Acquisition Unit 302 Answer Processing Unit

Claims

1. A question text generation unit that generates a question text for obtaining a rubric evaluation result by performing a rubric evaluation of the first learner's answer according to evaluation items and criteria for rubric-evaluating the first learner's answer to the first question text; A result output unit that inputs the question text into a learned model, generates and outputs a rubric evaluation result including a score of the first learner's answer; An information processing apparatus comprising the above.

2. When there is an image associated with the first question text, the question text generation unit generates the question text by including the first question text and an image description text obtained by verbalizing the content represented by the image in the question text. The information processing apparatus according to Claim 1.

3. The question text generation unit generates the question text for obtaining a rubric evaluation of the first learner's answer according to the evaluation items and criteria based on a second learner's answer to a second question text that is the same as or similar to the first question text and a rubric evaluation result of the second learner's answer. The information processing apparatus according to Claim 1.

4. Further comprising a similar question determination unit that determines that the first question text and the second question text are the same or similar. The information processing apparatus according to Claim 3.

5. When the score of the first learner's answer is equal to or higher than a predetermined score, the question text generation unit generates the question text for further obtaining to identify a portion of the first learner's answer that contributed to the score. The information processing apparatus according to Claim 1.

6. Generating a question text for obtaining a rubric evaluation result by performing a rubric evaluation of a learner's answer to a question text according to evaluation items and criteria for rubric-evaluating the learner's answer; Inputting the question text into a learned model, generating and outputting a rubric evaluation result including a score of the learner's answer; A program for causing an information processing apparatus to execute a process including the above.

Images