Evaluation device, information processing device, evaluation system, evaluation method and program

Abstract

To improve force for correcting a sentence.SOLUTION: A terminal device 10 includes an acquisition unit 111 for acquiring advice inputted by a user about a correction sentence to be a correction object, and a presentation unit 112 for presenting evaluation generated by a generation unit on the basis of prescribed evaluation reference about the validity of the advice to the user.SELECTED DRAWING: Figure 6

Description

[Technical field]

[0001] The present invention relates to a technique for evaluating the appropriateness of advice for a text. [Background technology]

[0002] Writing compositions is directly connected to thinking, and it is important to be able to write compositions in order to think. For this reason, in the Japanese language department, instruction in composition is considered important in order to improve students' thinking ability. In particular, instruction in composition through correction is essential in order to improve students' writing ability and, by extension, their thinking ability. However, for teachers with little experience, their ability to correct (hereafter also referred to as "correction ability") is still insufficient, and instruction through correction is difficult and a heavy burden. If teachers want to improve their correction ability, they have no choice but to practice on students' actual compositions, and there is also the problem that it is difficult to judge whether or not the corrections are appropriate.

[0003] Therefore, various techniques have been disclosed to reduce the burden of correction on teachers. For example, Patent Document 1 discloses a technique for generating corrected sentences by inputting the sentences to be corrected into a trained model. Using the technique of Patent Document 1, it is certainly possible to reduce the time and effort required for teachers to correct sentences, thereby reducing the burden on teachers. However, in actual educational settings, it is also expected that corrections will be made while communicating with students face-to-face. In such situations, the technique of Patent Document 1, which simply generates corrected sentences, is not suitable in the first place. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent Publication No. 2022-088468 Summary of the Invention [Problem to be solved by the invention]

[0005] Furthermore, in the technology of Patent Document 1, the trained model generates the corrected sentences instead of the teacher, so the teacher's correction ability cannot be improved. In consideration of the above circumstances, the present invention aims to improve the ability to correct sentences. [Means for solving the problem]

[0006] [1] The evaluation device of the present invention is a terminal device comprising: an acquisition unit that acquires advice input by a user regarding a text to be corrected; and a presentation unit that presents to the user an evaluation generated by a generation unit based on predetermined evaluation criteria regarding the validity of the advice.

[0007] [2] The terminal device of [1], wherein the presentation unit presents to the user an evaluation generated by the generation unit regarding the validity of the advice and a revised sentence in which the generation unit corrects the corrected sentence based on the advice.

[0008] [3] The terminal device of [2], wherein the presentation unit presents the evaluation generated by the generation unit for the corrected sentence based on the evaluation criteria as an evaluation of the validity of the advice.

[0009] [4] The corrected sentence is a sentence generated based on the evaluation criteria in response to an initial evaluation indicated by the user, and the presentation unit is a terminal device of [1] to [3] that presents the corrected sentence generated by the generation unit to the user.

[0010] [5] The terminal device of [4], wherein the presentation unit presents to the user a plurality of options that are candidates for the initial evaluation, and presents the corrected sentence generated by the generation unit in accordance with an initial evaluation selected by the user from the plurality of options.

[0011] [6] A terminal device according to [1] to [3], wherein the corrected sentence is a sentence input by the user, and the presentation unit presents to the user an initial evaluation of the corrected sentence generated by the generation unit based on the evaluation criteria.

[0012] [7] A terminal device according to [1] to [3], wherein the acquisition unit acquires an image input by the user, and the presentation unit presents to the user the corrected document generated by the generation unit and corresponding to information represented by the image acquired by the acquisition unit.

[0013] [8] A terminal device according to any one of [1] to [7], wherein the generation unit is an interactive large-scale language model.

[0014] [9] An information processing device including a generation unit that generates an evaluation based on predetermined evaluation criteria regarding the validity of advice input by a user regarding a text to be corrected.

[0015]

[10] An evaluation system comprising: an acquisition unit that acquires advice input by a user regarding a text to be corrected; a generation unit that generates an evaluation based on predetermined evaluation criteria regarding the validity of the advice; and a presentation unit that presents the evaluation generated by the generation unit to the user.

[0016]

[11] An evaluation method implemented by a computer system that receives advice input by a user regarding a text to be corrected, and presents to the user an evaluation generated based on predetermined evaluation criteria regarding the validity of the advice.

[0017]

[12] A program that causes a computer system to function as an acquisition unit that acquires advice input by a user regarding a text to be corrected, and a presentation unit that presents to the user an evaluation generated by a generation unit based on predetermined evaluation criteria regarding the validity of the advice. Effect of the Invention

[0018] According to the terminal device, evaluation system, evaluation method, and program of the present invention, an evaluation of the validity of advice input by the user about a text to be corrected is presented to the user, thereby improving the user's ability to correct text. [Brief description of the drawings]

[0019] [Figure 1] 1 is a block diagram illustrating a configuration of an evaluation system according to a first embodiment. [Diagram 2] 1 is a block diagram illustrating a configuration of an information processing device according to a first embodiment. [Diagram 3] 2 is a block diagram illustrating a configuration of a terminal device according to the first embodiment. FIG. [Figure 4] 2 is a block diagram illustrating a functional configuration of a control device in a terminal device according to the first embodiment. FIG. [Diagram 5] 5 is a flowchart showing specific steps of a process in which the evaluation system according to the first embodiment evaluates advice on a corrected text. [Figure 6] 1 is an example of an image displayed on a display device 14 of the terminal device according to the first embodiment. [Figure 7] 13 is a flowchart showing specific steps of a process in which the evaluation system according to the second embodiment evaluates advice on a corrected text. [Figure 8] 13 is a flowchart showing specific steps of a process for the evaluation system according to the third embodiment to evaluate advice on a corrected text. [Figure 9] 1 is an example of an image displayed on a display device 14 of the terminal device according to the first embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0020] First Embodiment FIG. 1 is a block diagram illustrating the configuration of an evaluation system 100 in the first embodiment. The evaluation system 100 is used to improve the ability to correct sentences (correction ability). A user U of the evaluation system 100 is, for example, a school teacher or a cram school instructor who is expected to correct sentences written by actual students. The user U inputs advice for improving the sentence to be corrected (hereinafter referred to as "corrected sentence") to the evaluation system 100. Then, the evaluation system 100 evaluates the validity of the advice input by the user U based on a predetermined evaluation criterion. That is, the user U can improve his / her correction ability by practicing correction (specific advice) using the evaluation system 100.

[0021] In the first embodiment, an example of a text to be corrected is a composition in which a writer freely writes about his / her own impressions and experiences along a preset theme.

[0022] The evaluation system 100 of the first embodiment includes a terminal device 10 and an information processing device 20. The terminal device 10 and the information processing device 20 can communicate with each other via a communication network 300 such as the Internet. The information processing device 20 is realized by, for example, a server system. On the other hand, the terminal device 10 is realized by, for example, a portable information device such as a smartphone or a tablet terminal, or a portable or stationary information device such as a personal computer.

[0023] [Information processing device 20] The information processing device 20 of the first embodiment generates a corrected sentence and an evaluation of the advice input by a user U about the corrected sentence. Fig. 2 is a block diagram illustrating an example of the configuration of the information processing device 20. Specifically, the information processing device 20 includes a control device 21, a storage device 22, and a communication device 23.

[0024] The control device 21 is a single or multiple processors that control the operation of the information processing device 20. Specifically, the control device 21 is configured by one or more types of processors, such as a central processing unit (CPU), a graphics processing unit (GPU), a sound processing unit (SPU), a digital signal processor (DSP), a field programmable gate array (FPGA), or an application specific integrated circuit (ASIC).

[0025] The storage device 22 is one or more memories that store programs executed by the control device 21 and various data used by the control device 21. For example, a known recording medium such as a semiconductor recording medium or a magnetic recording medium, or a combination of multiple types of recording media, is used as the storage device 22. Note that, for example, a portable recording medium that is detachable from the information processing device 20, or a recording medium that the control device 21 can access via a communication network (for example, cloud storage) may be used as the storage device 22.

[0026] The communication device 23 is a communication device that communicates with the terminal device 10 via the communication network 300. Note that the communication device 23, which is separate from the information processing device 20, may be connected to the information processing device 20 by wire or wirelessly.

[0027] The control device 21 executes a program stored in the storage device 22 to realize a function (a generation unit 211) for generating an evaluation by the user U regarding the validity of the advice.

[0028] First, the generating unit 211 generates a corrected text according to the evaluation instructed by the user U. Specifically, the corrected text is a text generated with a predetermined evaluation (hereinafter referred to as "initial evaluation") based on the evaluation criteria. In the first embodiment, the initial evaluation is instructed by the user U. Here, the evaluation of the corrected text is a score given to the corrected text based on the predetermined evaluation criteria, and is, for example, any score from 0 to 100 points (full score). The higher the score, the higher the evaluation of the corrected text. The initial evaluation of the corrected text is an evaluation that does not reach the highest evaluation (full score) based on the evaluation criteria. In other words, the corrected text is a text that has room for correction (improvement). The corrected text generated by the generating unit 211 is transmitted to the terminal device 10 by the communication device 23. In the first embodiment, the evaluation (initial evaluation) of the corrected text is transmitted to the terminal device 10 together with the corrected text.

[0029] Secondly, the generation unit 211 generates an evaluation (hereinafter referred to as "advice evaluation") on the appropriateness of advice for the corrected text. The advice to be evaluated is input by the user U at the terminal device 10 and transmitted to the information processing device 20. The advice evaluation generated by the generation unit 211 is transmitted to the terminal device 10. The generation unit 211 of the first embodiment presents to the user U the advice evaluation and a text in which the generation unit 211 has corrected the corrected text based on the advice (hereinafter referred to as "corrected text").

[0030] In the first embodiment, an evaluation is generated for the corrected sentence based on an evaluation criterion. The evaluation of the corrected sentence is based on the same criteria as the evaluation of the corrected sentence described above, that is, a score. The higher the score of the corrected sentence, the higher the evaluation of the corrected sentence. If the score of the corrected sentence increases significantly as the score of the corrected sentence, this means that the appropriateness of the advice is high (the advice is more appropriate). On the other hand, if the score of the corrected sentence does not increase much compared to the score of the corrected sentence, this means that the appropriateness of the advice is low (the advice is not very appropriate). As can be understood from the above explanation, in the first embodiment, the score of the corrected sentence (specifically, the increase from the score of the corrected sentence to the score of the corrected sentence) can be said to be the advice evaluation.

[0031] The generating unit 211 of the first embodiment is constructed by a generative model. The generative model is configured by, for example, a deep neural network. For example, any type of deep neural network, such as a recurrent neural network (RNN) or a convolutional neural network (CNN), is used as the generative model. The generative model may be configured by combining a plurality of types of deep neural networks.

[0032] It is preferable to use a large-scale language model (LLM) capable of natural language processing, which is trained in advance using a large amount of text data and in which the occurrence probability of words is modeled, as the generation model. In particular, it is preferable to use an interactive large-scale language model (generative artificial intelligence) in which a large-scale language model is fine-tuned using a dialogue data set as the generation model. As the interactive large-scale language model, any known model such as ChatGPT (manufactured by OPEN AI), Bard (manufactured by Google), and Bing AI (manufactured by Microsoft) is used. In the following description, a case where the generation unit 211 is an interactive large-scale language model is exemplified. The interactive language model is capable of generating dialogue-style text.

[0033] An example of the evaluation criteria used to evaluate corrected texts and revised documents will be described in detail. In the first embodiment, the evaluation criteria are composed of the following multiple evaluation items (1) to (4). The following evaluation criteria are based on the 2017 Junior High School Curriculum Guidelines. However, the evaluation criteria used in the present invention are not limited to the following examples, and may be changed as appropriate depending on the type of corrected text.

[0034] (1) Content and organization: Does the composition have a clear theme and a logical structure? 4 points: The topic is clear and the logical structure is solid. 3 points: The topic is relatively clear and there is some degree of logical structure. 2 points: The topic is not very clear and the logical structure is insufficient. Score 1: The topic is unclear and there is little logical structure.

[0035] (2) Sentences and Vocabulary: Does the sentence use appropriate grammar and vocabulary? 4 points: Grammar and vocabulary are appropriate and the writing is natural. 3 points: Grammar and vocabulary are mostly appropriate, and the writing is relatively natural. 2 points: There are some errors in grammar and vocabulary, and the sentences are partially unnatural. 1 point: There are many errors in grammar and vocabulary, and the sentence is generally unnatural.

[0036] (3) Creativity and originality: Is the composition creative and has an original perspective? 4 points: High creativity and unique perspective. Score 3: Some creativity and originality is demonstrated. Score 2: Not much creativity or originality. Score 1: Little to no creativity or originality is shown.

[0037] (4) Cultural and social understanding: Does the essay demonstrate an understanding of culture and society? 4 points: Demonstrates a deep understanding of culture and society. 3 points: Some understanding of culture and society is demonstrated. 2 points: Little understanding of culture and society is shown. Score 1: Little cultural or social understanding is shown.

[0038] The generating unit 211 evaluates (gives scores to) the corrected text and the revised document based on the above evaluation criteria. Specifically, the generating unit 211, for example, specifies a score for each of the above items, converts the total score of each item into a maximum score of 100 points (when the maximum score in the evaluation criteria is 16 points, it converts it into a maximum score of 100 points), and generates the score as the evaluation of the corrected text and the revised document. However, the specific method of evaluating the corrected text and the revised document based on the evaluation criteria is not limited to the above examples.

[0039] [Terminal device 10] 3 is a block diagram illustrating an example of the configuration of the terminal device 10. Specifically, the terminal device 10 includes a control device 11, a storage device 12, a communication device 13, an operation device 15, and a display device 14. The terminal device 10 is realized by, for example, a portable information device such as a smartphone or a tablet terminal, or a portable or stationary information device such as a personal computer.

[0040] The control device 11 is a single or multiple processors that control the operation of the terminal device 10. Specifically, the control device 11 is configured by one or more types of processors, such as a central processing unit (CPU), a graphics processing unit (GPU), a sound processing unit (SPU), a digital signal processor (DSP), a field programmable gate array (FPGA), or an application specific integrated circuit (ASIC).

[0041] The storage device 12 is one or more memories that store the programs executed by the control device 11 and various data used by the control device 11. For example, a known recording medium such as a semiconductor recording medium or a magnetic recording medium, or a combination of multiple types of recording media, is used as the storage device 12. Note that, for example, a portable recording medium that is detachable from the terminal device 10, or a recording medium that the control device 11 can access via a communication network (for example, cloud storage) may be used as the storage device 12.

[0042] The communication device 13 is a communication device that communicates with the information processing device 20 via the communication network 300. Note that the communication device 13, which is separate from the terminal device 10, may be connected to the terminal device 10 by wire or wirelessly.

[0043] The display device 14 displays various images under the control of the control device 11. For example, various display panels such as a liquid crystal display panel or an organic EL panel are used as the display device 14. Note that an operation device 15 or the display device 14 separate from the information processing device 20 may be connected to the information processing device 20 by wire or wirelessly.

[0044] The operation device 15 is an input device that accepts an operation by the user U. For example, an operator operated by the user U or a touch panel that detects contact by the user U is used as the operation device 15. In the first embodiment, the user U inputs various information (for example, an initial evaluation of a corrected text generated by the information processing device 20 and advice on the corrected text) by operating the operation device 15. The information input by the user U is transmitted from the communication device 13 to the information processing device 20. The information transmitted to the information processing device 20 may be the information input by the user U itself, or may be some data including the information input by the user U. In the first embodiment, since the generation unit 211 is an interactive language model, a prompt including the information input by the user U is transmitted to the information processing device 20. For example, if the user U inputs "30 points" as the initial evaluation, a prompt instructing the creation of a corrected text worth 30 points may be transmitted to the information processing device 20.

[0045] 4 is a block diagram illustrating an example of the functional configuration of the control device 11 of the terminal device 10. The control device 11 executes a program stored in the storage device 12 to realize a plurality of functions (an acquisition unit 111, a presentation unit 112) for presenting an evaluation of advice given to the user U on the corrected text.

[0046] The acquisition unit 111 acquires advice input by the user U regarding the corrected text. Specifically, the acquisition unit 111 acquires the advice by accepting the advice input by the user U via the operation device 15. The advice acquired by the acquisition unit 111 is transmitted from the communication device 13 to the information processing device 20.

[0047] The presentation unit 112 presents various types of information to the user U. Specifically, the presentation unit 112 presents various types of information to the user U by displaying them on the display device 14. However, the presentation of information by the presentation unit 112 is not limited to presentation by display, and may be presentation by audio, for example.

[0048] First, the presenting unit 112 presents the corrected sentence generated by the generating unit 211 of the information processing device 20 to the user U. In the first embodiment, the corrected sentence generated by the generating unit 211 in accordance with the initial evaluation instructed by the user U is presented to the user U.

[0049] Secondly, the presentation unit 112 presents the evaluation (advice evaluation) generated by the generation unit 211 based on the evaluation criteria for the advice to the user U. The presentation unit 112 in the first embodiment also presents the corrected sentence generated by the generation unit 211 to the user U in addition to the advice evaluation.

[0050] The process by which the evaluation system 100 evaluates advice on a corrected text (hereinafter referred to as the "evaluation process") will be described in detail with reference to Figures 5 and 6. Figure 5 is a flowchart showing a specific procedure of the evaluation process executed by the evaluation system 100.

[0051] 6 is an example of an image G displayed on the display device 14 of the terminal device 10 in the evaluation process. In the first embodiment, since the generation unit 211 is configured with an interactive large-scale language model, the evaluation process progresses by interactively exchanging information generated by the information processing device 20 (the generation unit 211) with information input by the user U. The image G shows an area Ta (Ta1, Ta2, ...) indicating information transmitted from the information processing device 20 to the terminal device 10, and an area Tb (Tb1, Tb2, ...) indicating information input by the user U to the terminal device 10.

[0052] The evaluation process in Fig. 5 is started when the terminal device 10 (acquisition unit 111) receives an instruction to start from the user U (SA1). Upon receiving the instruction to start from the user U, the terminal device 10 (acquisition unit 111) transmits information (hereinafter referred to as "start information") for instructing the information processing device 20 to start the evaluation process via the communication device 13 (SA2). The start information will be described in detail later.

[0053] When the information processing device 20 receives the start information from the terminal device 10, the information processing device 20 transmits a plurality of options that are candidates for the score (initial evaluation) of the corrected text to be generated by the generation unit 211 to the terminal device 10 from the communication device 23 (SA3). Step SA3 also serves as a process of notifying the terminal device 10 of the start of the evaluation process.

[0054] The terminal device 10 (presentation unit 112) presents a plurality of options that are candidates for an initial evaluation to the user U (SA4). Specifically, the terminal device 10 receives the plurality of options transmitted from the information processing device 20, and causes the display device 14 to display the plurality of options (30 points, 50 points, 70 points, 90 points), as exemplified in the area Ta1 of FIG.

[0055] The user U selects one of a plurality of options by operating the operation device 15, thereby indicating an initial evaluation (30 points), as exemplified in the area Tb1 of Fig. 6. The acquisition unit 111 accepts the initial evaluation indicated by the user U, and transmits it to the information processing device 20 (SA5).

[0056] The information processing device 20 (the generation unit 211) generates a corrected text in accordance with the initial evaluation transmitted from the terminal device 10 (SA6). That is, the corrected text is generated in accordance with the initial evaluation selected by the user U from a plurality of options. Specifically, the generation unit 211 arbitrarily sets a theme and a title for the corrected text, and generates a corrected text that will score the initial evaluation (30 points) when evaluated based on the evaluation criteria. Then, the information processing device 20 transmits the corrected text generated by the generation unit 211 to the terminal device 10 (SA7).

[0057] The terminal device 10 (presentation unit 112) presents the corrected text transmitted from the information processing device 20 to the user U (SA8). Specifically, the terminal device 10 displays the corrected text on the display device 14, as exemplified in area Ta2 in Fig. 6. The corrected text is displayed together with an initial evaluation and information encouraging the user U to give advice that will improve the evaluation of the corrected text based on the evaluation criteria (for example, a sentence such as "Please give me some advice to improve this composition to 100 points").

[0058] 6, the user U inputs specific advice for the corrected text by operating the operation device 15. The terminal device 10 (acquisition unit 111) receives and acquires the advice input by the user U (SA9). Then, the acquisition unit 111 transmits the advice to the information processing device 20 (SA10).

[0059] The information processing device 20 (generation unit 211) generates amended sentence by correcting the corrected sentence in response to the advice transmitted from the terminal device 10, and evaluates the amended sentence based on the evaluation criteria (SA11). An evaluation of the amended sentence (i.e., a score for the amended sentence) is generated as an evaluation of the appropriateness of the advice for the user U (advice evaluation). The higher the score for the amended sentence, the higher the appropriateness of the advice. The information processing device 20 transmits the amended sentence and the advice evaluation to the terminal device 10 (SA12).

[0060] The terminal device 10 (presentation unit 112) presents the corrected sentence and the advice evaluation (score of the corrected sentence) transmitted by the information processing device 20 to the user U (SA13). Specifically, the terminal device 10 causes the display device 14 to display the corrected sentence and the advice evaluation (70 points), as exemplified in the area Ta3 in FIG. 6.

[0061] In addition, the user U may input further advice on the corrected sentence presented in step SA13, as exemplified in area Tb3 in Fig. 6. That is, the process may return to step SA9 after step SA13. Therefore, further advice is given to the corrected sentence presented in step SA13 as a corrected sentence. When the process returns to step SA9 and steps SA10-SA12 are executed, the corrected sentence further corrected in step SA13 and the score (advice evaluation) of the corrected sentence are displayed, as exemplified in area Ta4 in Fig. 6.

[0062] The evaluation process may be ended at any time, for example, by instruction from user U, or when the score of the revised sentence reaches 100 points. The evaluation process may also be ended when the processing of step SA8 is performed or when a pre-set time limit is reached. Furthermore, when the evaluation process is ended, the revised sentence with the highest score (i.e., full marks) based on the evaluation criteria may be presented.

[0063] An example of start information for instructing the start of the evaluation process of the first embodiment will be described below. As described above, in the first embodiment, since the generation unit 211 is an interactive language model, the start information is information corresponding to a prompt. The start information is information for instructing the generation unit 211 on the overall flow from the start to the end of the evaluation process (steps SA3-SA13) so that the generation unit 211 proceeds with the evaluation process.

[0064] The start information includes, for example, the following information P1 to P6. Information P1 is information instructing the generation unit 211 to perform role 1 (role of mainly performing processing related to corrected text). Information P2 is information instructing the generation unit 211 to perform role 2 (role of mainly performing processing related to corrected text). Information P3 is information indicating the role of user U (corrector). Information P4 is information indicating the rules of the evaluation process, and includes various types of optional information related to the evaluation process. Information P5 is information indicating how to start and end the evaluation process. Information P6 is information indicating the specific contents of the evaluation criteria.

[0065] <Information indicating Role 1 of the Generation Unit 211> Create a revised text composed of a theme, a title, and a passage. Create a revised text with the score selected by the user among 30 points, 50 points, 70 points, and 90 points in light of the evaluation criteria. Present the revised text and its score to the user via the terminal device.

[0066] <Information indicating Role 2 of the Generation Unit 211> Revise the revised text based on the advice. Evaluate the advice entered by the user based on the evaluation criteria. Determine what score the revised text will be under the evaluation criteria. Present the revised text and its score to the user via the terminal device.

[0067] <Information indicating the role of the user> The user gives advice on how to make the presented revised text worth 100 points.

[0068] <Information indicating the rules of the evaluation process> The scores of the revised text and the revised text after further revision are values obtained by converting 16 full marks under the following evaluation criteria to a 100 - point full - mark conversion. The user may further input advice on the revised text after further revision. Each time advice is input, the text is revised based on the advice and the advice is also evaluated.

[0069] <Information indicating how to start and end the evaluation process> Start by presenting the user with options for the scores of the revised text to be created. End when the score of the revised text after further revision reaches 100 points or when the user instructs to end.

[0070] <Information indicating the evaluation criteria> In the evaluation process, evaluation criteria including the above - mentioned multiple evaluation items (1) - (4) are used.

[0071] However, the information included in the start information is not limited to the above information P1-P6. For example, the start information may include information indicating conditions regarding the corrected text generated by the generation unit 211. Examples of the conditions regarding the corrected text include conditions regarding the attributes of the writer of the corrected text (for example, corrected text that is written by a junior high school student), conditions regarding the genre of the theme of the corrected text (for example, selected from themes suitable for junior high school students), or a condition that a different corrected text is generated each time the evaluation process is performed.

[0072] It is preferable that the start information transmitted to the information processing device 20 is not presented to the user (i.e., is not displayed on the display device 14). For example, it is possible to prevent the start information from being presented to the user by setting an API (Application Programming Interface).

[0073] As can be understood from the above explanation, in the first embodiment, the validity of advice given to corrected text is evaluated (i.e., feedback is obtained), so that the user U can improve his / her ability to correct text. By using the evaluation system 100, the user U can repeatedly practice correction without using the text of actual students as a practice model. In addition, the user U can input any advice that would be difficult to give to an actual student into the evaluation system 100 and check its validity.

[0074] In the first embodiment, in addition to evaluating the validity of the advice, the corrected sentence corrected by the user U based on the advice is presented to the user U, which has the advantage that the user U can confirm how the corrected sentence is corrected by the advice, as compared to a configuration in which the corrected sentence is not presented. However, the present invention also includes a configuration in which the corrected sentence is not presented.

[0075] Furthermore, because an evaluation of the revised sentence (i.e., a score for the revised sentence) is presented as an evaluation of the appropriateness of the advice, it is easier for user U to intuitively grasp the evaluation of the appropriateness of the advice, compared to a configuration in which a direct evaluation of the appropriateness of the advice (e.g., an evaluation with a score independent of the score for the revised sentence) is presented to user U. However, the present invention also encompasses a configuration in which a direct evaluation of the appropriateness of the advice is presented to user U. For example, a configuration in which the evaluation of the appropriateness of the advice is given with a score out of 10 points, or a configuration in which the evaluation of the appropriateness of the advice is shown in one of a number of stages (e.g., very good, good, average, poor) may be used.

[0076] In the first embodiment, a composition is used as an example of a text to be corrected, but the type of text to be corrected is not limited to a composition, and various types of text that can be the subject of correction (e.g., a diary, an impression, an essay) can be used as the text to be corrected.

[0077] <Second embodiment> A second embodiment of the present invention will be described. Note that, in each of the following exemplary embodiments, for elements whose functions are similar to those of the first embodiment, the reference numerals used in the description of the first embodiment will be used and detailed descriptions of each will be omitted as appropriate.

[0078] In the first embodiment, a configuration in which a corrected document generated by the information processing device 20 is used is exemplified, but in the second embodiment, a configuration in which any corrected text prepared in advance by the user U is used is exemplified. The user U uses, for example, a text written by the user himself or a text actually written by a student as the corrected text. The specific configurations of the terminal device 10 and the information processing device 20 are the same as those in the first embodiment.

[0079] Fig. 7 is a flowchart showing an example of an evaluation process according to the second embodiment. In Fig. 7, steps SB1 to SB5 are executed instead of steps SA4 to SA8 in Fig. 5.

[0080] First, as in the first embodiment, the process starts when the terminal device 10 (acquisition unit 111) receives a start instruction from the user U (SA1), and then starts by transmitting start information to the information processing device 20 to instruct the information processing device 20 to start the evaluation process (SA2).

[0081] When the information processing device 20 receives the start information from the terminal device 10, it notifies that the evaluation process has started (SA3). However, unlike the first embodiment, in the second embodiment, typically no option for the score of the corrected text is transmitted. The user U inputs a text (hereinafter referred to as "input text") by operating the operation device 15. The terminal device 10 (acquisition unit 111) acquires the input text entered by the user U by accepting it (SB1). Then, the acquisition unit 111 transmits the input text to the information processing device 20 (SB2).

[0082] The information processing device 20 (the generation unit 211) generates the input text transmitted from the terminal device 10 as a corrected text, and generates an initial evaluation of the corrected text based on the evaluation criteria (SB3). The generation unit 211 may use the input text itself as the corrected text, or may use a text generated by the generation unit 211 based on the input text as the corrected text. A score (any score between 0 and less than 100 points) assigned to the corrected text based on the same evaluation criteria as in the first embodiment is generated as the initial evaluation. However, any score close to the multiple options for the initial evaluation in the first embodiment (for example, 30 points, 50 points, 70 points, and 90 points) may be used as the score of the corrected text. The information processing device 20 transmits the corrected text and the initial evaluation of the corrected text to the terminal device 10 (SB4).

[0083] The presenting unit 112 presents the corrected text and the initial evaluation (i.e., the initial evaluation of the corrected text generated by the generating unit 211 based on the evaluation criteria) transmitted from the information processing device 20 to the user U (SB5). After the user U inputs advice on the corrected text, the processes of steps SA9-SA13 are the same as those in the first embodiment.

[0084] As can be understood from the above explanation, in the second embodiment, the corrected text is a text (input text) input by the user U to the evaluation system 100, so compared to the configuration of the first embodiment in which the generation unit 211 generates the corrected text, the corrected text can be a text with a theme or title desired by the user U.

[0085] <Third embodiment> The third embodiment illustrates a configuration for generating corrected text from an image input by a user U. In the third embodiment, the user U inputs an image representing a painting, and an appreciation text for the painting is used as the corrected text.

[0086] Fig. 8 is a flowchart of an example of the evaluation process according to the third embodiment. In Fig. 8, steps SC1-SC6 are executed instead of steps SA4-SA8 in Fig. 5. Fig. 9 is an example of an image G displayed on the display device 14 of the terminal device 10 when the evaluation process according to the third embodiment is executed. In the image G, similar to Fig. 6, an area Ta (Ta1, Ta2, ...) showing information transmitted from the information processing device 20 to the terminal device 10 and an area Tb (Tb1, Tb2, ...) showing information input by the user U to the terminal device 10 are shown.

[0087] First, as in the first embodiment, the process starts when the terminal device 10 (acquisition unit 111) receives a start instruction from the user U (SA1), and then starts by transmitting start information to the information processing device 20 to instruct the information processing device 20 to start the evaluation process (SA2).

[0088] When the information processing device 20 receives the start information from the terminal device 10, it notifies that the evaluation process has started (SA3). Then, as exemplified in area Ta1 of FIG. 9, information prompting the user U to input an image is displayed by the display device 14. As exemplified in area Tb1, the user U inputs an image (hereinafter referred to as an "input image") by operating the operation device 15. As described above, in the third embodiment, an image representing a painting is exemplified as the input image. The terminal device 10 (acquisition unit 111) acquires the input image input by the user U by accepting it (SC1). Then, the acquisition unit 111 transmits the input image to the information processing device 20 (SC2).

[0089] The information processing device 20 (generation unit 211) generates corrected sentences for each of a plurality of initial evaluation options (e.g., 30 points, 50 points, 70 points) from the input image transmitted from the terminal device 10 (SC3). That is, a plurality of corrected sentences (a corrected sentence for 30 points, a corrected sentence for 50 points, a corrected sentence for 70 points) are generated. The plurality of corrected sentences are transmitted to the terminal device 10 together with the initial evaluations of the corrected sentences (SC4). The evaluation criteria of the third embodiment will be described later.

[0090] The terminal device 10 (presentation unit 112) presents the multiple corrected sentences to the user U together with their initial evaluations (SC5). Specifically, as illustrated in area Ta2 of FIG. 9, each of the multiple corrected sentences is displayed on the display device 14 together with its initial evaluation (30 points, 50 points, 70 points). The user U selects one of the multiple corrected sentences as illustrated in area Tb2, and the selected corrected sentence is notified to the information processing device 20 (SC6). As illustrated in area Ta3, information is displayed prompting the user U to input advice, and the user U inputs advice on the corrected sentences as illustrated in area Tb3. After the user U has input his advice, the processing of steps SA9-SA13 is executed in the same manner as in the first embodiment.

[0091] The evaluation criteria in the third embodiment are, for example, composed of the following multiple evaluation items (1) to (10): The total score of each item (out of 100) is the evaluation (score) of the corrected and revised text.

[0092] (1) Composition (maximum 10 points): Is the composition and balance of the work explained in detail and does it clearly convey your impression of it? (2) Color (maximum 10 points): Have you accurately expressed your analysis and impressions of the use of color, color harmony, and color contrast? (3) Technique (maximum 10 points): Does the student provide specific observations or evaluations of the technique or use of materials? (4) Subject or Theme (maximum 10 points): Does the student demonstrate an understanding of the subject or theme of the painting and provide an evaluation or interpretation based on that understanding? (5) Historical Background (maximum 10 points): Does the analysis and commentary incorporate the historical background and cultural elements of the work? (6) Personal Impression (maximum 10 points): Does the viewer's own emotions, feelings, and intuitive impressions go into depth? (7) Light and Shadow (maximum 10 points): Has the student analyzed the use of light and shadow and evaluated their effects? (8) Space and Depth (maximum 10 points): Does the student state his / her evaluation and impression of the representation of three-dimensional space on a two-dimensional canvas? (9) Historical and Cultural Background (maximum 10 points): Does the work provide a detailed analysis and explanation of its historical value and cultural background? (10) Artist’s Background (maximum 10 points): Does the evaluation and interpretation incorporate knowledge of the artist’s background and art philosophy?

[0093] The evaluation criteria in the third embodiment (evaluation criteria when the image represents a painting) are, for example, composed of the above-mentioned multiple evaluation items (1) to (10). However, the evaluation criteria used in the third embodiment are not limited to the above examples, and may be changed as appropriate depending on the content of the image.

[0094] As can be understood from the above explanation, in the third embodiment, a corrected document generated from an input image entered by user U is presented to user U, so that user U can practice giving advice on sentences that may be created based on the information indicated in the input image.

[0095] It is not essential to generate a corrected document from an input image entered by user U, and for example, a corrected sentence may be generated using any image selected by generation unit 211. Also, although multiple corrected sentences generated for each of multiple options for initial evaluation are presented to user U, for example, when inputting an input image, a corrected sentence may be generated for any initial evaluation (score) specified by user U. That is, in the third embodiment, it is not essential to present multiple corrected sentences to user U.

[0096] In the above examples, an image showing a painting is input, but the information represented by the image is not limited to a painting. The information represented by the image may be, for example, a diagram, graph, table, or matrix that visually represents data, or a diagram, map, or format that shows a technical explanation. The corrected text is a text corresponding to the information represented by the image, such as a text showing the content grasped from the information, the characteristics understood, and an explanation, consideration, interpretation, etc. of the information.

[0097] <Modification> The above-mentioned exemplary embodiments can be modified in various ways. Specific modified embodiments are exemplified below. Two or more embodiments selected from the following examples can be appropriately combined.

[0098] (1) The generating unit 211 may be constructed with a generating model other than a language model. For example, a generating model that learns the relationship between the corrected text, the advice, and the evaluation of the validity of the advice by machine learning may be used as the generating unit 211. In addition, it is not essential that the generating unit 211 is constructed with a generating model. For example, a method in which the generating unit 211 generates a corrected text according to an initial evaluation specified by the user U using a data table in which an initial evaluation, a corrected text, advice, and the validity of the advice are associated and registered, or a method in which the generating unit 211 evaluates the validity of the advice of the user U (i.e., a rule-based method) may be adopted. However, according to a configuration in which the generating unit 211 is constructed with an interactive large-scale language model, there are advantages in that the specific process of the evaluation process can be easily changed by changing the content of the start information (prompt), and that the user U can give advice by assuming a dialogue with an actual correction target (typically a student).

[0099] (2) When presenting the user U with the scores for the corrected text and the revised text, the evaluation (score) for each item of the evaluation criteria may also be presented to the user U.

[0100] (3) A configuration may be adopted in which a sentence with a perfect score is presented based on the evaluation criteria in response to an instruction from the user U. Also, a configuration may be adopted in which a hint for advice is presented in response to an instruction from the user U. The hint for advice is generated by the generation unit 211, for example, based on the evaluation criteria.

[0101] (4) In the first embodiment, the user U sets the initial evaluation by selecting a score from multiple options, but the user U may set the initial evaluation by freely inputting any score less than 100 points. In this case, the information processing device 20 may be omitted from transmitting the multiple options to the terminal device 10.

[0102] The initial evaluation may be set by the generation unit 211. For example, the generation unit 211 may generate a corrected text in which one of a plurality of options is randomly selected as the initial evaluation upon receiving an instruction from the user U to start the evaluation process, or may generate a corrected text in which an arbitrary score less than 100 points is the initial evaluation. The configuration in which the generation unit 211 sets the initial evaluation has the advantage that the user U does not need to specify the initial evaluation. In other words, it is not essential that the generation unit 211 generates a corrected text upon receiving an instruction from the user U to specify the initial evaluation. However, the configuration in which the corrected text is generated according to the initial evaluation specified by the user U has the advantage that the user U can arbitrarily set the initial evaluation of the corrected text according to his or her own ability.

[0103] (5) The evaluation process is progressed by display on the display device 14, but the evaluation process may be progressed by voice, for example. When the evaluation process is progressed by voice, for example, the information shown in the area Ta (Ta1, Ta2, ...) and the information shown in the area Tb (Tb1, Tb2, ...) in the image G of Fig. 6 are presented to the user U by voice.

[0104] (6) If the evaluation of the validity of the advice does not meet the required evaluation criteria (for example, if there are no items that can be added), the score of the revised text may be lower than the score of the original corrected text. In other words, the revised text may be a worse version of the corrected text.

[0105] (7) In the first embodiment, the evaluation criteria created based on the 2017 junior high school curriculum guidelines were used, but the evaluation criteria used to evaluate the corrected and revised texts are not limited to the above examples. For example, evaluation criteria according to the National Assessment of Educational Progress (NAEP) may be used. The evaluation criteria are, for example, composed of the following items 1 to 4, and the total score for items 1 to 4 is 100 points.

[0106] <Item 1> Content and Idea (40 points) Originality and Depth (10 points): Demonstrates a unique perspective or deep insight into the topic. Clarity of topic (10 points): The topic is clear and consistent throughout. Support (10 points): Use specific evidence or examples to support your thesis or argument. Relevance (10 points): All information is directly relevant to the topic or purpose. <Item 2> Organization and structure (30 points) Introduction (10 points): There is an introduction that grabs the reader's attention and makes the topic clear. Paragraph Structure (10 points): Each paragraph is based on a clear theme or idea. Conclusion (10 points): There is an effective conclusion that brings the essay together. <Item 3> Grammar and style (20 points) Grammar and Spelling (10 points): Few or no grammar or spelling errors. Style and Vocabulary (10 points): Uses appropriate style and vocabulary. <Item 4> Format and presentation (10 points) Well-organized formatting (5 points): Proper paragraph indentations, margins, line spacing, etc. Legibility (5 points): If handwritten, the writing is neat; if typed, the font and size are appropriate.

[0107] The evaluation criteria are changed appropriately according to the type and content of the corrected text, and also according to the country. For example, in Germany, the evaluation criteria may be based on the German composition evaluation criteria table in the German language education standard Mittlerer Schulabschluss established on December 4, 2003. The evaluation criteria may be composed of, for example, the following items 1 to 3, with the total score of items 1 to 3 being 100 points.

[0108] <Item 1> Contents (30 points) Understanding and handling of the subject (10 points): Did you understand the subject accurately and state the main points clearly? Sentence structure (10 points): Is the sentence structured logically and is it consistent? Richness of expression (10 points): Is the vocabulary rich and varied? <Item 2> Language (40 points) Grammatical accuracy (20 points): Are there any grammatical or usage errors? Appropriateness of Vocabulary (10 points): Is age-appropriate vocabulary used? Spelling and punctuation (10 points): Are spelling and punctuation used correctly? <Item 3> Format (30 points) Size and shape of letters (10 points): Are the size and shape of letters consistent? Paragraphs and line breaks (10 points): Are the paragraph divisions and line breaks appropriate? Legibility of handwriting (10 points): Is the handwritten text easy to read?

[0109] (8) The process executed by the terminal device 10 will be specifically described with a focus on the case where the generation unit 211 is an interactive large-scale language model. First, the start information (i.e., the prompt) is information that enables the generation unit 211, to which the start information has been input, to start the evaluation process, and is information that includes at least, for example, information instructing the generation of a corrected sentence and an evaluation of the corrected sentence (the above-mentioned information P1), information instructing the generation of a revised sentence and an evaluation (advice evaluation) of the revised sentence (the above-mentioned information P2), information indicating the role of the user (the above-mentioned information P3), and information indicating the contents of the evaluation criteria (the above-mentioned information P6).

[0110] The terminal device 10 (acquisition unit 111) functions as an element that executes, for example, process 1 (SA2 in FIG. 5) of sending a prompt (start information) instructing the start of evaluation processing, process 2 (step SA5 in FIG. 5) of sending a prompt instructing the creation of corrected text, and process 3 (step SA10 in FIG. 5) of sending a prompt instructing the generation of corrected text and an evaluation of the corrected text (advice evaluation) in response to advice on the corrected text. However, the processes executed by the terminal device 10 are not limited to processes 1 to 3, and other processes may be executed, and some processes may be omitted.

[0111] (9) In each of the above-described embodiments, the information processing device 20 is a server device, but the functions of the information processing device 20 (e.g., the generation unit 211) may be installed in the terminal device 10 of the user U. Also, some of the functions of the information processing device 20 (e.g., some of the functions of the generation unit 211) may be installed in the terminal device 10. As can be understood from the above explanation, the evaluation system 100 may be a single device or a computer system composed of multiple devices.

[0112] (10) The evaluation system 100 according to the above embodiment is realized by the cooperation of a computer (specifically, a control device) and a program. The program according to the above embodiment can be provided in a form stored in a computer-readable recording medium and installed in the computer. The recording medium is, for example, a non-transitory recording medium, and is exemplified by an optical recording medium (optical disk) such as a CD-ROM, but may include any known type of recording medium such as a semiconductor recording medium or a magnetic recording medium. Note that a non-transitory recording medium includes any recording medium except for a transient, propagating signal, and does not exclude a volatile recording medium. It is also possible to provide the program to the computer in the form of distribution via a communication network.

[0113] (11) The present invention is also specified as an evaluation method realized by a computer that obtains advice input by a user U about a corrected text to be corrected, and presents an evaluation generated by a generation unit 211 based on a predetermined evaluation criterion regarding the validity of the advice to the user U. The present invention is also specified as a generation method realized by a computer that generates an evaluation based on a predetermined evaluation criterion regarding the validity of the advice input by the user U about a corrected text to be corrected. [Explanation of symbols]

[0114] 10: Terminal device 11: Control device 12:Storage device 13: Communication equipment 14:Display device 15: Operating device 20: Information processing device 21: Control device 22: Storage device 23: Communication equipment 100: Rating System 111: Acquisition Department 112: Presentation part 211 :Generation part 300: Communication network U:User

Claims

1. an acquisition unit for acquiring advice input by a user regarding a text to be corrected; a presentation unit that presents to the user an evaluation generated by the generation unit based on a predetermined evaluation criterion regarding the validity of the advice; A terminal device comprising:

2. The presentation unit presents to the user an evaluation generated by the generation unit regarding the validity of the advice and a corrected sentence obtained by correcting the corrected sentence by the generation unit based on the advice. The terminal device of claim 1.

3. The presentation unit presents the evaluation generated by the generation unit for the corrected sentence based on the evaluation criteria as an evaluation of the validity of the advice. The terminal device of claim 2.

4. the corrected text is a text generated based on the evaluation criteria in response to an initial evaluation designated by the user, The presentation unit presents the corrected sentence generated by the generation unit to the user. The terminal device of claim 1.

5. The presentation unit presents to the user a plurality of options that are candidates for the initial evaluation, and presents the corrected sentence generated by the generation unit according to an initial evaluation selected by the user from the plurality of options. The terminal device according to claim 4.

6. the corrected sentence is a sentence input by the user, The presentation unit presents to the user the initial evaluation of the corrected text generated by the generation unit based on the evaluation criteria. The terminal device of claim 1.

7. The acquisition unit acquires an image input by the user, The presentation unit presents the corrected document corresponding to information represented by the image generated by the generation unit and acquired by the acquisition unit to the user. The terminal device according to claim 1

8. The generator is an interactive large-scale language model. The terminal device of claim 1.

9. The system further includes a generating unit that generates an evaluation based on a predetermined evaluation criterion regarding the validity of advice input by a user regarding a text to be corrected. Information processing device.

10. an acquisition unit for acquiring advice input by a user regarding a text to be corrected; a generator for generating an evaluation of the validity of the advice based on a predetermined evaluation criterion; a presentation unit that presents the evaluation generated by the generation unit to the user; An evaluation system comprising:

11. Obtaining advice input by a user regarding the text to be corrected; Present to the user an evaluation of the validity of the advice based on a predetermined evaluation criterion. An evaluation method implemented by a computer system.

12. an acquisition unit that acquires advice input by a user regarding the text to be corrected; a presentation unit that presents to the user an evaluation generated by the generation unit on the validity of the advice based on a predetermined evaluation criterion; A program that causes a computer system to function as a

Images