How to Evaluate a Statement

Abstract

[Problem] To provide a computer-based text evaluation method that can also evaluate context. [Solution] This computer-based text evaluation method includes a text input step in which a text that is to be evaluated is input into a computer, an in-text word extracting step in which the computer extracts an in-text word, which is a word included in the text to be evaluated, a key sentence extracting step in which the computer extracts a key sentence included in the text to be evaluated, the key sentence including one or two or more in-text words and one or two or more particles, and a text evaluating step in which the computer evaluates the text on the basis of the key sentence, wherein: the text evaluating step includes a step for reading evaluation information relating to the key sentence from a storage unit; and the evaluation information is information relating to whether the key sentence is correct as the text to be evaluated, or information relating to a category to which the key sentence belongs.

Description

[Technical field]

[0001] The present invention relates to a method and system for evaluating a sentence by a computer. [Background technology]

[0002] Patent No. 7049010 describes a presentation evaluation system.

[0003] This evaluation system evaluates the content of the conversation or the people who conducted the conversation based on the number of keywords, the number of related words, the combination of keywords, or the combination of related words. In this case, there was a problem that even if the conversation was incorrect based on the context, if the conversation contained keywords, it would be evaluated as correct. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent No. 7049010 Summary of the Invention [Problem to be solved by the invention]

[0005] The present invention aims to provide a method for evaluating a sentence by a computer, which is also capable of evaluating the context.

[0006] An object of the present invention is to provide a learning support method using the above-mentioned evaluation method.

[0007] The present invention aims to provide a method by which the successor of a sentence can be obtained so that a role play can be performed after evaluating the context. [Means for solving the problem]

[0008] This method is basically concerned with evaluating sentences using key sentences that contain particles, thereby enabling the correct understanding of the context. This method allows a computer to evaluate sentences based on key sentences or based on key sentences.

[0009] This example method includes a sentence input step (S101), a sentence word extraction step (S102), a key sentence extraction step (S103), and a sentence evaluation step (S104). The sentence input step (S101) is a step in which a sentence to be evaluated is input to a computer. The in-sentence word extraction step (S102) is a step in which a computer extracts in-sentence words, which are words contained in the sentence to be evaluated. The key sentence extraction step (S103) is a step in which a computer extracts a key sentence contained in the sentence to be evaluated. The key sentence includes one or more words in the sentence and one or more particles. The sentence evaluation step (S104) is a step in which the computer evaluates the sentence based on the key sentence or based on the key sentence.

[0010] An example of the sentence input process (S101) includes a process in which speech is input to a computer, and a process in which the computer analyzes the speech input to the computer to obtain the sentence to be evaluated.

[0011] Examples of sentences to be evaluated are explanatory sentences or answer sentences.

[0012] Examples of in-sentence words are words stored in an in-sentence word memory section that stores in-sentence words related to the sentence to be evaluated.

[0013] The examples of key sentences are those stored in a key sentence storage section which stores key sentences related to the sentence to be evaluated.

[0014] An example of a key sentence may be a group of words that are in consecutive positions in the sentence to be evaluated, or may be a group of words that are in distant positions.

[0015] An example of the use of the above method is a computer-assisted learning method.

[0016] Another example of the use of the above method relates to a method for generating a follow-on sentence by a computer. This method further includes a subsequent sentence creation step (S105). The subsequent sentence creation step (S105) is a step of obtaining a subsequent sentence, which is a sentence following the sentence to be evaluated, based on the evaluation. The computer has a subsequent sentence storage unit that stores the subsequent sentence corresponding to the evaluation. In the subsequent sentence creation step (S105), the computer reads out the subsequent sentence corresponding to the evaluation using the evaluation of the sentence to be evaluated.

[0017] This specification also discloses a system for evaluating a computerized sentence. This system 1 is a system in which a computer evaluates a sentence. This system 1 includes a sentence input unit 3, a word-in-sentence extraction unit 5, a key sentence extraction unit 7, and a sentence evaluation unit 9. The sentence input section 3 is an element for inputting a sentence to be evaluated. The in-sentence word extraction unit 5 is an element for extracting in-sentence words, which are words contained in the sentence to be evaluated. The key sentence extraction unit 7 is an element for extracting a key sentence contained in a sentence to be evaluated. A key sentence includes one or more words in the sentence and one or more particles. The sentence evaluation unit 9 is an element for evaluating a sentence based on a key sentence.

[0018] This specification also discloses a learning support system that utilizes the above-mentioned system.

[0019] This specification also discloses a system for creating a subsequent sentence using the above-mentioned system. The system for creating a subsequent sentence has a subsequent sentence creation unit.

[0020] This specification also discloses a program for causing a computer to execute the above-mentioned method or for causing a computer to function as the above-mentioned system, as well as a non-transitory information recording medium that can be read by a computer and has such a program recorded thereon. Effect of the Invention

[0021] The present invention can provide a method for evaluating a sentence by a computer that can also evaluate the context.

[0022] The present invention can provide a learning support method using the above-mentioned evaluation method.

[0023] The present invention can provide a method for obtaining the successor of a sentence so that a role play can be performed after evaluating the context. [Brief description of the drawings]

[0024] [Figure 1] FIG. 1 is a flow chart for explaining a method for evaluating sentences using a computer. [Diagram 2] FIG. 2 is a block diagram showing an example of the configuration of a system for evaluating sentences by a computer. [Diagram 3] FIG. 3 is a diagram showing an outline of the experiment in Example 1. [Figure 4] FIG. 4 is a diagram showing an example of implementation of iRolePlay (registered trademark). [Diagram 5] FIG. 5 is a graph, instead of a drawing, showing the verification of the memory retention rate of word memorization learning. [Figure 6] FIG. 6 is a graph, instead of a drawing, showing the retention rate of word explanation sentences using speech learning. [Figure 7] Figure 7 is a conceptual diagram explaining the concept of role-playing. [Figure 8] Figure 8 is a conceptual diagram showing an example of a role-play in which the AI ​​doctor's response varies depending on the MR's answer. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0025] Hereinafter, the embodiments for carrying out the present invention will be described with reference to the drawings. The present invention is not limited to the embodiments described below, and includes appropriate modifications of the embodiments described below within the scope obvious to those skilled in the art.

[0026] This method is basically a method of evaluating sentences that can correctly grasp the context by evaluating the sentences using key sentences that include particles. In this method, a computer evaluates sentences based on key sentences or based on key sentences.

[0027] "Evaluating a sentence" can mean interpreting whether the sentence is the correct answer, interpreting which category the sentence belongs to, or determining how well the sentence is written.

[0028] Figure 1 is a flow chart for explaining a method for evaluating a sentence using a computer. As shown in Figure 1, an example of this method includes a sentence input step (S101), a word in the sentence extraction step (S102), a key sentence extraction step (S103), and a sentence evaluation step (S104). This method may further include a subsequent sentence creation step (S105). S stands for step.

[0029] FIG. 2 is a block diagram showing an example of the configuration of a system for evaluating a sentence by a computer. As shown in FIG. 2, the system 1 includes a sentence input unit 3, a word-in-sentence extraction unit 5, a key sentence extraction unit 7, and a sentence evaluation unit 9. The system may further include any one or more of a word-in-sentence storage unit 6, a key sentence storage unit 8, and a sentence evaluation information storage unit 10. The system 1 may further include a subsequent sentence creation unit 11. When the system 1 includes the subsequent sentence creation unit 11, the system 1 may further include a subsequent sentence storage unit 13. The sentence input unit 3 is an element for inputting a sentence to be evaluated. The word-in-sentence extraction unit 5 is an element for extracting a word-in-sentence that is a word included in the sentence to be evaluated. The key sentence extraction unit 7 is an element for extracting a key sentence included in the sentence to be evaluated. The sentence evaluation unit 9 is an element for evaluating a sentence based on a key sentence or based on a key sentence. The subsequent sentence generating unit 11 is an element for obtaining a subsequent sentence which is a sentence following the sentence to be evaluated. This system 1 is a computer-based system for executing the above-mentioned method.

[0030] The computer has an input unit, an output unit, a control unit, a calculation unit, and a storage unit, and each element is connected by a bus or the like so as to be able to exchange information. For example, the storage unit may store a control program or various information. When predetermined information is input from the input unit, the control unit reads out the control program stored in the storage unit. Then, the control unit appropriately reads out the information stored in the storage unit and transmits it to the calculation unit. The control unit also appropriately transmits the input information to the calculation unit. The calculation unit performs calculation processing using the various information received and stores it in the storage unit. The control unit reads out the calculation results stored in the storage unit and outputs them from the output unit. In this way, various processes and each step are executed. The various parts and means execute these various processes. The computer may have a processor, and the processor may realize various functions and various steps. The computer may be standalone. A part of the computer's functions may be distributed to a server and a terminal. In that case, it is preferable that the server and the terminal are able to exchange information through a network such as the Internet or an intranet.

[0031] The computer may receive information wirelessly transmitted from various external devices or information output as an optical signal, convert it into an electrical signal, and use it. The optical signal or the electrical signal may include on-off keying and various modulation signals. The storage unit may store various input information based on, for example, the presence or absence of an electric charge, or may store various input information based on multiple physical states such as a quantum state. When extracting words or key sentences, the computer may compare the words or key sentences stored in the storage unit with the input sentence, and if a word or sentence contained in the input sentence matches a word or key sentence stored in the storage unit, extract the matching word or key sentence. Alternatively, a learning model may be constructed by performing machine learning, and a specific word or key sentence may be output by inputting the input sentence and the words or key sentences stored in the storage unit into the learning model.

[0032] The computer may be equipped with, for example, a machine learning engine, and may construct a trained model using teacher data by machine learning, and obtain various information by inputting various information to the trained model. The obtained various information may be stored in a storage unit. The trained model may improve its accuracy by feeding back the obtained various information. Furthermore, the accuracy of the trained model may be improved by repeatedly inputting teacher data, correct answer data, teacher data, and incorrect answer data. For example, in this invention, machine learning may be performed using various sentences and evaluation values ​​as teacher data to construct a trained model and obtain an evaluation of the sentence. Furthermore, machine learning may be performed using one or more key sentences and evaluation values ​​(evaluation values ​​indicating how good the sentence is) as teacher data to construct a trained model and obtain an evaluation of the sentence. Furthermore, for a sentence for which an evaluation value has already been obtained, one or more key sentences may be obtained, and the one or more key sentences may be input into the trained model to obtain an evaluation value. The evaluation value thus obtained may be compared with the evaluation value of the sentence itself, and the comparison result may be fed back as teacher data to improve the accuracy of the trained model.

[0033] The sentence input step (S101) is a step in which a sentence to be evaluated is input to the computer. The sentence input unit 3 in the system 1 may input the sentence to be evaluated to the system. For example, the sentence may be input to the computer using an input device (for example, a keyboard or a mouse). Also, the voice that is the source of the sentence may be input to the computer using a voice input device such as a microphone. The voice input to the computer may be recognized by a known method to obtain the sentence to be evaluated. In this case, the computer performs a calculation process to digitize the voice and appropriately stores the digitized voice in a storage unit. The computer may have a voice analysis unit that analyzes words and terms contained in the voice. The voice analysis unit reads out a voice analysis program and the digitized voice from the storage unit, analyzes the voice, and obtains the sentence to be evaluated. The obtained sentence to be evaluated may be appropriately stored in the storage unit.

[0034] When a sentence to be evaluated is input to a computer, if the sentence is related to something, information about the thing to which the sentence is related may also be input to the computer. Then, this system can perform various calculations using various information about the thing to which the sentence is related. Examples of calculations are speech analysis, word extraction, sentence extraction, and various calculations in subsequent sentence analysis. Examples of things to which a sentence is related are presentation materials, presentation material pages, reports, meeting materials, medicine, questions, problems, questionnaires, Q&A collections, telephone response records, and chatbot manuals. For example, when a presentation material is launched on a terminal, information related to the presentation material may be input to the system. In this case, as will be described later, dictionaries and memory units related to the presentation material can be used. The sentence to be evaluated may be any sentence. Examples of sentences to be evaluated are explanatory sentences or answer sentences. In other words, this system can be used to evaluate sentences when a person explains something to another person, or to automatically evaluate answer sentences to a question. In addition, this system can automatically interpret an input question and effectively create an answer to it.

[0035] For example, an MR (Medical Representative) inputs information about a certain medicine A into the system, or starts an explanatory document about a certain medicine A and displays the explanatory document on the display unit. The system then stores in the memory unit information that subsequent utterances and conversations are about medicine A. Then, a voice from the MR is input into the microphone saying, "If you are pregnant or have a medical condition that may be causing pregnancy, please contact us if the medical condition exceeds the threshold of pregnancy."

[0036] The system performs terminology analysis on the above speech, taking into consideration the terminology conversion dictionary for medicine A (conversion terminology storage section for medicine A). In the terminology conversion dictionary for medicine A, words including "pregnant woman," "pregnant," "possibly," "woman," "treatment," "therapeutic," "beneficial," "risk," "exceed," "judgment," "case," and "administration" are stored as high-priority words. For example, this dictionary stores "woman" as a word with a higher priority than "aid," so for the word "woman" in the above input speech, "woman" is read out first.

[0037] In this way, the following sentence is entered into the system regarding Drug A: "Administer to pregnant women or women who may be pregnant only if it is determined that the therapeutic benefits outweigh the risks."

[0038] The in-sentence word extraction step (S102) is a step in which the computer extracts in-sentence words, which are words included in the sentence to be evaluated. The in-sentence word extraction unit 5 extracts in-sentence words, which are words included in the sentence to be evaluated. The system may have a dictionary related to in-sentence words, and extract in-sentence words from the sentence to be evaluated by reading the sentence to be evaluated from the storage unit and comparing it with the words in the dictionary. Furthermore, when information on things to which the sentence is related is input to the computer, it becomes possible to easily extract in-sentence words by using a dictionary related to things to which the sentence is related and for extracting in-sentence words (in-sentence word storage unit 6). In this case, examples of in-sentence words are words stored in the in-sentence word storage unit that stores in-sentence words related to the sentence to be evaluated. The system (in-sentence word extraction unit 5) can extract in-sentence words included in the sentence to be evaluated by reading the in-sentence words stored in the in-sentence word storage unit 6 and comparing it with the sentence to be evaluated. The in-sentence words extracted by the system may be appropriately stored in the storage unit. The system can obtain in-sentence words in this manner.

[0039] The system has a sentence word storage unit 6 related to medicine A. In the sentence word storage unit 6 related to medicine A, words including "pregnant woman," "pregnancy," "woman," "beneficial," "risk," "exceeds," and "administration" are stored as sentence words with high priority.

[0040] For this reason, from the sentence "Administer to pregnant women or women who may be pregnant when it is determined that the therapeutic benefits outweigh the risks" regarding Medicine A input to the system, the words in the sentence, such as "pregnant woman," "pregnancy," "woman," "benefit," "risk," "outweighs," and "administer," are extracted. When a voice is input to the system, the term conversion dictionary and the in-sentence word storage unit 6 may be the same, but it is preferable that they are different. In other words, the terms used for term conversion and the words extracted as in-sentence words may be the same, but may be different since the purposes are different.

[0041] The key sentence extraction step (S103) is a step in which a computer extracts key sentences contained in a sentence to be evaluated. For example, the key sentence extraction unit 7 extracts key sentences contained in a sentence to be evaluated. A key sentence includes one or more sentence words and one or more particles. A key sentence may include one or more sentence words. When one key sentence includes two or more sentence words, those sentence words may be sentence words that exist consecutively in the sentence to be evaluated, or one or more other sentence words or nouns may exist before those sentence words.

[0042] In the above-mentioned in-sentence word extraction step (S102), words included in the sentence to be evaluated are extracted. However, even if words alone are used, it may not be possible to determine whether the sentence is correct (explanation or answer sentence). For this reason, this system 1 extracts a key sentence by a key sentence extraction step (S103). A key sentence means a clause used to evaluate whether a sentence is a correct explanation sentence or a correct answer sentence. A key sentence usually includes two or more words. The key sentence may be stored in a key sentence storage unit 8 that stores key sentences related to the sentence to be evaluated, or may be extracted from the sentence to be evaluated as including a certain in-sentence word A and a particle related to the in-sentence word A included in the sentence to be evaluated. Note that the in-sentence word extraction step (S102) and the key sentence extraction step (S103) are for convenience, and may be performed simultaneously. In this case, the in-sentence word is also extracted when the key sentence extraction step (S103) is performed.

[0043] In the former case, the system (key sentence extraction unit 7) reads out one or more key sentences stored in association with the sentence to be evaluated in the key sentence storage unit 8, performs a calculation to compare it with the sentence to be evaluated, and extracts the key sentences included in the sentence to be evaluated. The system may store the extracted key sentences included in the sentence to be evaluated in the storage unit as appropriate.

[0044] In the latter case, the system (key sentence extraction unit 7) stores, for example, in-sentence words for key sentences stored in association with the sentence to be evaluated in key sentence storage unit 8. The system may then read out the in-sentence words for key sentences from key sentence storage unit 8, and read out the in-sentence words for key sentences and the particles following them from the sentence to be evaluated, thereby extracting key sentences from the sentence to be evaluated. The system may store the extracted key sentences contained in the sentence to be evaluated in the storage unit as appropriate.

[0045] For example, medicine A is administered to pregnant women or women who may be pregnant when the benefits outweigh the risks. However, simply extracting words in a sentence may result in the sentence being evaluated being one that means the risks outweigh the benefits. Such a sentence is not correct as an explanatory sentence regarding medicine A. Therefore, the key sentence storage unit 8 stores the key sentence ("therapeutic benefits"..."outweigh the risks"). The key sentence storage unit 8 may also store key sentences ("benefits outweigh risks"), ("risks are lower"), and ("risk is low"). The key sentence storage unit 8 may store key sentences including "pregnant women" and the words in the sentence "pregnancy" for the subjects of administration, and any of the above for the efficacy.

[0046] For this reason, for example, from the sentence "Administer to pregnant women or women who may be pregnant only if it is determined that the therapeutic benefits outweigh the risks," key sentences are extracted, such as the subject (either or both of "pregnant women" and "pregnancy"), and regarding the medicinal effects ("therapeutic benefits outweigh the risks").

[0047] The sentence evaluation step (S104) is a step in which a computer evaluates a sentence based on a key sentence. For example, the sentence evaluation unit 9 evaluates the sentence to be evaluated based on the key sentence. The system 1 may further include a sentence evaluation information storage unit 10. The sentence evaluation information storage unit 10 stores, for example, a key sentence and evaluation information related to the key sentence. Examples of the evaluation information include information on whether the key sentence is correct as the sentence to be evaluated, information on which category the key sentence belongs to, and an evaluation value associated with the key sentence. The system (sentence evaluation unit 9) reads out an evaluation related to the extracted key sentence from the sentence evaluation information storage unit 10 using the key sentence extracted in the key sentence extraction step (S103). In this way, the system 1 can evaluate the sentence to be evaluated.

[0048] The evaluation value associated with the key sentence may be an evaluation value (score) stored in a storage unit in association with each key sentence. For example, a higher evaluation value may indicate that an appropriate key sentence has been used. As a result, a sentence using an appropriate key sentence may have a higher evaluation value and be highly evaluated. When multiple evaluation values ​​are obtained, the evaluation values ​​may be added together to obtain an evaluation value (an index of how good a sentence is) for the sentence.

[0049] For example, the sentence evaluation information storage unit 10 stores "correct" as an evaluation in association with a key sentence including (either or both of "pregnant woman" and "pregnancy"), ("therapeutic benefits") and ("outweigh the risks") in association with medicine A. Then, the sentence evaluation unit 9 reads out the evaluation "correct" from the sentence evaluation information storage unit 10 using the above key sentence. The sentence evaluation unit 9 may store the read evaluation "correct" in the storage unit as an evaluation of the sentence to be evaluated. Then, the system 1 may output the evaluation "correct". Also, suppose that the category "administration prescription" is stored in the sentence evaluation information storage unit 10 in association with a key sentence including (either or both of "pregnant woman" and "pregnancy"), ("therapeutic benefits") and ("outweigh the risks"). Then, the system 1 reads out the category "administration prescription" as one of the evaluations in association with medicine A using the extracted key sentence. In this case, system 1 may obtain a rating of "correct" for the category "dosage prescription" in relation to medicine A.

[0050] The in-sentence word storage unit 6 may store a classification of words in association with the in-sentence words. Examples of classifications are positive, negative, and ambiguous. Table 1 shows examples of classifications and in-sentence words related to each classification. Furthermore, when a key sentence includes an in-sentence word, the key sentence storage unit 8 may store a classification of the in-sentence word. The sentence evaluation information storage unit 10 may store a classification of the in-sentence word included in the key sentence. The system (sentence evaluation unit 9) reads out an evaluation of the extracted key sentence from the sentence evaluation information storage unit 10 using the key sentence extracted in the key sentence extraction step (S103). For example, the system 1 reads out information on the classification of the in-sentence word included in the key sentence (e.g., positive, negative, and ambiguous) from the sentence evaluation information storage unit 10 (the key sentence storage unit 8 or the in-sentence word storage unit 6) using the extracted key sentence. If the classification of the words in the key sentence that is read out is negative (or ambiguous), the evaluation of the sentence to be evaluated may be evaluated as incorrect (inappropriate). In this way, it is possible to prevent ambiguous sentence expressions and improve answering ability.

[0051] In addition, this specification also describes an invention in which words in a sentence are extracted from the sentence to be evaluated, instead of the key sentence, and the sentence to be evaluated is evaluated based on the classification of the extracted words in the sentence. In this invention, the key sentence extraction step (S103) and the key sentence extraction unit 7 are not required.

[0052] [Table 1]

[0053] The subsequent sentence creation step (S105) is a step of obtaining a subsequent sentence that is a sentence that follows the sentence to be evaluated based on the evaluation. The subsequent sentence creation unit 11 may obtain the subsequent sentence that is a sentence that follows the sentence to be evaluated. The computer has, for example, a subsequent sentence storage unit 13 that stores a subsequent sentence corresponding to the evaluation. In the subsequent sentence creation step (S105), the computer reads out the subsequent sentence that corresponds to the evaluation using the evaluation of the sentence to be evaluated. Information about what the sentence is related to is input to the computer, and an appropriate subsequent sentence may be read out from the subsequent sentence storage unit 13 that corresponds to the information about what the sentence is related to. For example, if the information about what the sentence is related to is "a certain question" and the evaluation is "correct" (correct answer), the subsequent sentence storage unit 13 stores the evaluation "well done" and an explanation about the question. The system (subsequent sentence creation unit 11) reads the evaluation from the memory unit, reads the evaluation "Well done," which is the subsequent sentence corresponding to the evaluation, and an explanation for the question from the subsequent sentence memory unit 13, and stores them in the memory unit as appropriate. The read subsequent sentence may be output as appropriate. If the sentence is a chatbot about a product with information about related items, and the evaluation is "issue identification" (category), then the answer to the sentence to be evaluated can be read from the subsequent sentence memory unit 13 based on the evaluation to obtain an appropriate answer. In this way, the system can obtain the subsequent sentence. For example, if the evaluation of the sentence is a "question" or "question," an answer to the sentence can be obtained.

[0054] Assume that the following is input to the system 1 with regard to medicine A: "For pregnant women or women who may be pregnant, administer the medicine only if it is judged that the therapeutic benefits outweigh the risks." The system 1 extracts words and key sentences from the sentence and obtains an evaluation of "correct" for the category of "administration prescription" in relation to medicine A. For example, the subsequent sentence storage unit 13 stores a subsequent sentence, "The administration frequency of medicine A is one tablet per day," in relation to the evaluation of "correct" for the category of "administration prescription" in relation to medicine A. The system (subsequent sentence creation unit 11) may read the evaluation from the storage unit, read the subsequent sentence from the subsequent sentence storage unit 13, and store it in the storage unit. The system may also output the subsequent sentence. Then, "The administration frequency of medicine A is one tablet per day" is output as the subsequent sentence of the sentence.

[0055] Computer-assisted learning An example of the use of the above method and system is a learning support method using a computer. This specification also discloses a learning support system using the above system. This aspect of the invention may be provided in a downloadable form as a learning support application. In this case, various dictionaries related to each question are stored in a server or installed in a mobile terminal. In this system, for example, a certain question A is stored from a storage unit and displayed on a display unit of the terminal. A user answers the question by voice. The user's voice is input to the terminal via an input unit of the terminal. The input voice (user's answer) is stored in the terminal as a sentence to be evaluated. A system including either or both of a terminal and a server has a term conversion dictionary related to the question A, a word in sentence storage unit 6, a key sentence storage unit 8, a sentence evaluation information storage unit 10, and / or a subsequent sentence storage unit 13. This allows the system to evaluate the user's answer. The system can also read out the subsequent sentence corresponding to the user's answer and display it on the display unit of the terminal. As will be described later, memorizing words using sentences rather than simply memorizing words not only helps to solidify memories but also improves communication skills. For this reason, the learning support system and learning support method using this invention can provide users with a high learning effect.

[0056] Chatbots and role-playing systems Examples of the use of the above system are chatbots and role-playing systems. The chatbot or role-play system has one or more of a term conversion dictionary related to a certain product or service, a sentence word storage unit 6, a key sentence storage unit 8, a sentence evaluation information storage unit 10, and a subsequent sentence storage unit 13. Then, when a question or the like is input to the system by voice input via telephone or automatically input via the Internet, the system can search for the key sentence, obtain an evaluation related to the product or service, and obtain an appropriate subsequent sentence. Then, an appropriate answer to the question can be automatically output. In this case, for example, in response to an input of "The wheels of product AA have stopped working," the system may use the storage unit related to product AA to obtain a subsequent sentence of "Is the power on?" and output it to the questioner. Also, if the user inputs "Yes" in response to this, the system may obtain a subsequent sentence such as "Try cleaning around the wheels with a brush. If the wheels still do not work, please contact the service center and we will come to collect it." and output it to the questioner.

[0057] Conversation or presentation support system An example of the use of the above system is a conversation or presentation support system. This system has, for example, any one or more of explanatory materials, presentation materials, a term conversion dictionary for each page of the presentation materials, a word in sentence storage unit 6, a key sentence storage unit 8, a sentence evaluation information storage unit 10, and a subsequent sentence storage unit 13. Then, when a conversation or presentation is conducted based on the materials or presentation materials, a key sentence can be obtained, and using the key sentence, an appropriate evaluation can be obtained from the sentence evaluation information storage unit 10, and an appropriate subsequent sentence can be obtained from the subsequent sentence storage unit 13. In this case, the evaluation may be "very good", "excellent", or "good". An example of the subsequent sentence may be an explanation that leads to an improvement in the conversation or presentation, such as "You will get a higher evaluation if you explain using BBB instead of AAA."

[0058] This specification also discloses a program having instructions for causing a computer to execute the above-mentioned method and for causing a computer to function as the above-mentioned system, as well as a non-transitory information recording medium (e.g., a CD-ROM, a DVD, an SD card, and a USB memory) that can be read by a computer and has such a program recorded thereon. EXAMPLES

[0059] Consideration of learning models using spoken word learning It is believed that one can give an easy-to-understand explanation through three steps: "words and explanations are fixed in memory," "fixed memories are replaced with one's own words," and "being able to give an easy-to-understand explanation." In this example, we consider the results of verification of "words and explanations are fixed in memory" and "fixed memories are replaced with one's own words." In this study, the following three verification items were verified. In the following experiments, an application that implements the computer-based learning support system described in this specification was used as a role-playing application. An overview is shown in Figure 3.

[0060] (i) Verification of memory retention of word memorization learning (ii) Measurement of retention rate of vocabulary explanations using spoken word learning (Pharmaceutical-related questions were asked, and the subjects were 10 men and women in their 20s to 50s. They were divided into two groups and administered the same content.) (iii) Measuring retention rate and learning time using iPad® role-playing apps (Network-related questions were asked, and the subjects were six men and women in their 30s to 50s.)

[0061] Verification of memory retention of rote word learning First, to verify how much vocabulary can be memorized through silent reading, 21 technical terms related to the pharmaceutical industry that are not used on a daily basis were selected and used to create questions. An overview of the relevant words and explanations is shown in Table 2.

[0062] [Table 2]

[0063] On the first day of the demonstration experiment, the subjects were given A4 sheets of paper with words and descriptions written on them, and were asked to memorize the words on the spot for 10 minutes using a method other than memorizing them out loud (for example, by looking at them visually or writing them down on paper). After 10 minutes, the descriptions of the words were turned into questions and read aloud to the subjects, who were then asked a question. If the subject answered incorrectly, the correct answer was given orally on the spot as feedback. On the second and third days, only the test was administered, and the changes in memory consolidation were observed, and the number of correct answers given by the subjects was tallied.

[0064] Measuring retention rate of word explanations using spoken word learning After the three-day vocabulary test was completed, the subjects were given a test to see if they could speak the word descriptions in order to measure how well they could recall the descriptions for the words. The test consisted of verbally asking the subjects 21 questions, such as "Please explain about XX," and recording the subjects' answers (the tests were recorded using the iPad (registered trademark) voice recognition function to make the process more efficient). After that, as shown in Table 3, the same tests were repeated periodically over a total of 19 days (please note that due to work circumstances, the tests were not conducted on an even schedule), and the results were recorded to measure the memory retention rate of the word descriptions.

[0065] [Table 3]

[0066] Additionally, in order to measure the effect of memory consolidation through oral learning, subjects underwent oral learning three times, with an interval of approximately 3-4 days from the first day of measurement. As with the above verification, the content of the oral learning was the same as that of the above verification, in which subjects were given A4 sheets of paper with the words and descriptions shown in Table 1 written on them, and they were asked to memorize the word descriptions out loud for 10 minutes on the spot. The "memory test" in Table 2 refers to a test to see how well the subjects could speak the word descriptions, and the "confirmation test" refers to a test to see how well the subjects could speak the word descriptions immediately after oral learning.

[0067] Measuring retention rate and learning time using iPad(R) role-playing apps Using the iPhone (registered trademark) and iPad (registered trademark) app "iRolePlay (registered trademark)" developed by Interactive Solutions (registered trademark), we measured the memory rate of words and explanations, as well as the study time. "iRolePlay(R)" is a new-generation reskilling tool that "helps users acquire explanation and proposal skills" through objective analysis, evaluation, and advice functions using AI that utilizes voice recognition of the user's speech and the Neural Engine installed in the iPhone(R) and iPad(R). It consists of two modes (learning mode / challenge mode). In the former, answers to questions are read out loud and displayed, allowing users to practice speaking (Figure 4). In the latter, users can acquire practical explanation skills by practicing speaking answers to questions read out loud.

[0068] The test questions consisted of 21 unfamiliar network-related vocabulary words, and two courses were prepared, one for learning vocabulary words and one for learning word explanations. The subjects were given iPads (registered trademark). They were encouraged to study independently during their free time between work for three days. As shown in Figure 4, the subjects studied in a role-play format, speaking in the learning mode, that is, in the course where the questions and answers were displayed on the iPad (registered trademark). This app has a system where you can move on to the next question by correctly speaking the correct word or explanation. The test method used a challenge mode in which questions asking about words or word explanations in the form of questions are spoken, and voice recognition was used to determine whether the questions were answered correctly.

[0069] Experimental results and discussion Verification of memory retention of rote word learning As mentioned above, it is effective to memorize words by speaking them out loud, but even when studying by sight or writing them down after 10 minutes of memorization, the correct answer rate was 78.6%. In addition, the correct answer rate on the second day, when there was no time for memorization, was 81.9%, and on the third day, the correct answer rate was 87.6%, showing an upward trend (Figure 5).

[0070] Consideration As is well known from the "Ebbinghaus forgetting curve," it is generally said that memories are forgotten over time. However, in this test, the fact that the answers to incorrect answers were given to the subjects on the spot made them easier to remember, which is thought to be a factor in the resulting increase in test scores. In fact, when interviewing the subjects, three of them commented that they had remembered the correct answers that they had been given to questions they had gotten wrong on the previous test. This shows that in word memory, immediate feedback of the correct answer to an incorrect answer is effective in solidifying the memory.

[0071] Measuring retention rate of word explanations using spoken word learning The correct answer rate for the vocabulary test was 87.6%, but the correct answer rate for the word explanation test was 43.3%, less than half. This result proves that even if a student can answer a word in a question-and-answer format, it does not necessarily mean that he or she can explain that word.

[0072] Now, we will present the results regarding memory of word explanations. When comparing the results of visual learning and oral learning in the word explanation test, the latter resulted in a 64% increase in the correct answer rate compared to the former. Also, the progress of the correct answer rate in the word explanation test, that is, the progress of the memory consolidation rate, is shown in Figure 6. By regularly conducting oral learning, the correct answer rate throughout the entire schedule has progressed. Even 19 days after the first explanation test, the answer rate remains above 70%.

[0073] Consideration By repeating speech training, the subjects were able to utter answers similar to the sample answers (example explanations shown in Figure 3). Analysis of the answers recorded by the subjects using speech recognition showed that the proportion of technical terms in the answers increased with repeated speech training. In interviews with the subjects, some said that speaking out even technical terms that they do not normally use and taking in sound with their ears was an advantage when answering out loud. As the test was repeated, the subjects became able to speak answers fluently, including technical terms, and at the same time, the rate of correct answers gradually increased. In order to develop explanation skills, it is necessary to speak to others using appropriate expressions, and sometimes technical terms. In that respect, it is thought that in this test, technical terms were solidified in memory by speaking out loud, and the subjects became accustomed to speaking the words, which allowed them to smoothly explain things in their own words. This experiment also revealed that explanatory texts that have been memorized once tend to remain in long-term memory. Looking at the results of the 21 questions, the rate of correct answers to questions that were once answered did not decrease even after a number of days had passed. In interviews with the subjects, two of them commented that by memorizing the key words in the explanatory texts, they were able to give correct explanations during the word explanation test.

[0074] Measuring retention rate and learning time using iPad(R) role-playing apps Although the verification using the iPad (registered trademark) role-playing app is still in progress, the progress of the correct answer rate for the word explanation text shows a similar trend. The biggest advantage is that the time it took for educators to measure was only the distribution of the app. In the above verification, it took 2,480 minutes to improve the scores of 10 people, including the setting of test questions, feedback on correct answers to incorrect answers, and grading of the test results. Another benefit is that subjects can study whenever they have free time. Because it is possible to obtain logs from the app, it is possible to measure study time accurately, which is expected to lead to advantageous learning guidance for educators. It remains to be verified how effectively subjects can use their free time to improve their accuracy rate in answering vocabulary explanations and shorten the period.

[0075] In this study, explanatory ability was defined as "the ability to explain things to others using appropriate expressions," and three steps were considered to solidify explanatory ability. In examining the degree of memory retention in word memorization learning, it was shown that immediate feedback of the correct answer to an incorrect answer was effective in solidifying memory. On the other hand, it was proven that being able to answer a word in a question-and-answer format does not necessarily mean that one can explain that word. In measuring the retention rate of word explanation sentences using oral learning, it was revealed that the retention rate of word explanation sentences was improved by repeating oral learning. In addition, the content of the word explanation test by the subjects showed that the rate of correct answers increased through three steps. EXAMPLES

[0076] role play Figure 7 is a conceptual diagram explaining the concept of role-playing. The "brackets" in the diagram are categories that are evaluations.

[0077] Figure 8 is a conceptual diagram showing an example of a role-play in which the AI ​​doctor's response varies depending on the MR's answer.

[0078] The developed application was installed on a computer, and instructions based on the application were executed by the computer. Q below is the sentence to be evaluated, and A is the sentence that follows it. Q: What is the patient's medical history? A Myocardial infarction 3 years ago Q: Are you experiencing any other symptoms? A: It seems that you have a headache or dizziness. Q: What were the findings before administration? A: I thought it was stressful. QWhat is the current solution? A: I am on a diet and taking antihypertensive drugs. Q: What is the relationship with AiPro tablets? A: Aipro tablets are easy to prescribe because they have few respiratory side effects.

[0079] In this example, when a doctor inputs a question, first, the term conversion dictionary related to the medical field and the target patient, the in-sentence word memory unit 6, the key sentence memory unit 8, the sentence evaluation information memory unit 10, and the subsequent sentence memory unit 13 are referenced, and the appropriate subsequent sentence is displayed on the display unit. In the middle of the question, "EyePro tablets" appears as a word in the sentence. Therefore, this system references the key sentence memory unit 8, the sentence evaluation information memory unit 10, and the subsequent sentence memory unit 13 related to EyePro tablets, and obtains the appropriate subsequent sentence. Since this system performs key sentence analysis, chatbots and role-playing become conversational, and natural conversations can be conducted even using a computer. Also, unlike previous conversation systems using deep learning, this system can be constructed to enable conversations in a narrowed area very easily and without spending a lot of time. Role-playing using this system was effective in cultivating advanced conversation skills. [Industrial Applicability]

[0080] This invention can be used in the information industry, education industry, and the like. [Explanation of symbols]

[0081] 1 System 3. Sentence input section 5. Extraction of words from a sentence 6. Sentence word memory section 7 Key sentence extraction section 8 Key sentence memory section 9 Sentence Evaluation Section 10 Sentence evaluation information storage unit 11 Subsequent sentence creation section 13 Subsequent sentence memory section

Claims

1. 1. A method of evaluating a computer-implemented statement, comprising: a sentence input step of inputting a sentence to be evaluated into the computer; a sentence word extraction step in which the computer extracts sentence words that are words included in the sentence to be evaluated; a key sentence extraction step in which the computer extracts a key sentence included in the sentence to be evaluated, the key sentence including one or more words in the sentence and one or more particles; and a sentence evaluation step in which the computer evaluates the sentence based on the key sentence, the sentence evaluation step includes a step of reading evaluation information regarding the key sentence from a storage unit; The evaluation information is information on whether the key sentence is correct as a sentence to be evaluated, or information on which category the key sentence belongs to.

2. 2. The method of claim 1 , The sentence input step includes: inputting speech into the computer; and wherein the computer analyzes the speech to obtain the sentence to be evaluated.

3. 2. The method of claim 1 , The method, wherein the sentence to be evaluated is an explanatory sentence or an answer sentence.

4. 2. The method of claim 1 , A method, wherein the evaluation information is information regarding to which category the key sentence belongs.

5. 2. The method of claim 1 , The key sentence may be a group of words located in consecutive positions in the sentence to be evaluated, or a group of words located at separate positions.

6. evaluating the sentence to be evaluated according to the method of claim 1, The computer-assisted learning method.

7. A computer-based system for evaluating sentences (1), comprising: a sentence input unit (3) for inputting a sentence to be evaluated; a sentence word extraction unit (5) for extracting sentence words which are words included in the sentence to be evaluated; a key sentence extraction unit (7) for extracting a key sentence contained in the sentence to be evaluated, the key sentence including one or more words in the sentence and one or more particles; A sentence evaluation unit (9) that reads evaluation information related to the key sentence from a storage unit based on the key sentence or based on the key sentence, and evaluates the sentence to be evaluated, The evaluation information is information regarding whether the key sentence is correct as a sentence to be evaluated, or information regarding which category the key sentence belongs to.

8. A program for causing a computer to execute a method for evaluating a statement, comprising: The program includes: a sentence input step of inputting a sentence to be evaluated into the computer; a sentence word extraction step in which the computer extracts sentence words that are words included in the sentence to be evaluated; a key sentence extraction step in which the computer extracts a key sentence included in the sentence to be evaluated, the key sentence including one or more words in the sentence and one or more particles; a sentence evaluation step of evaluating the sentence to be evaluated based on the key sentence, the sentence evaluation step includes a step of reading evaluation information regarding the key sentence from a storage unit; The evaluation information is information regarding whether the key sentence is correct as a sentence to be evaluated, or information regarding which category the key sentence belongs to.

9. 9. A non-transitory information recording medium that can be read by a computer and stores the program according to claim 8.

Images