System for evaluating language read aloud and displaying results

Abstract

To make it possible to provide an information processing device capable of evaluating reading aloud and allowing users to experience enjoyment while displaying it appropriately.SOLUTION: The inventions described in claims 1 to 17 of the present invention aim to make learning fun by incorporating a game-like element, and solve a problem by creating a system that automatically and comprehensively evaluates a language read aloud and displays its results. In addition, the inventions described in claims 9 to 17 of the present invention solve the problem by capturing and recognizing a phenomenon that reveals inner thoughts of a reader in relation to the enjoyment of learning to read aloud, and further by creating a system with an essential, broad and specific configuration and procedure.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to language learning and to a system for examining spoken language and displaying the results. [Background technology]

[0002] With regard to language learning, some devices evaluate language read aloud and display the results, with the aim of effectively using a variety of learning functions to learn a language. Some devices output and display a list of learning methods for words judged to be incorrectly pronounced among the words contained in the audio when read aloud, or assign scores to multiple assessment items of the audio and suggest the next learning content. However, these devices are not intended to make learning fun.

[0003] As a result, students were not fully convinced when they were simply instructed on something that had been determined to be incorrect, or when they were told what to do next, and they tended to gradually become bored with learning by repeating the same thing over and over again.

[0004] This drawback in reading aloud during language learning has been a major obstacle to continued good learning, resulting in a decline in the motivation to study language enthusiastically and positively, especially among children and adolescents, and a loss of opportunities to effectively acquire language while still young.

[0005] As a solution to this problem, there are systems that convert the audio of the reading aloud into text and display it in a highlighted or emphasized text display, etc., to encourage students to read aloud, but these systems do not evaluate the content of the learning and present it appropriately, nor do they make learning enjoyable. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] JP 2019-174525 A

[0007] [Patent Document 2] Patent No. 7166696 Summary of the Invention [Problem to be solved by the invention]

[0008] It is not possible to provide an information processing device that can evaluate reading aloud, display the results appropriately, and provide a fun experience. [Means for solving the problem]

[0009] The present invention provides an information processing method having a system configuration according to one embodiment of the present invention. This information processing method relates to language learning, and by learning in a fun and natural way through a game, fostering interest, curiosity and a competitive spirit, and by providing a system that comprehensively evaluates the language read aloud and appropriately displays the results, the learning contents can be naturally visualized and turned into episodes without being conscious of it, and the language learning contents can be stored in a long-term memory in a short period of time through experience. Effect of the Invention

[0010] The present invention introduces a game-like element into the reading of a language, and by providing a system that comprehensively evaluates the language that has been read aloud and displays the results, it enables a person to memorize and experience a language in a short period of time while reading aloud in a fun way that develops interest, curiosity and a competitive spirit, and while learning naturally. In educational psychology, the Ebbinghaus forgetting curve, the results of an experiment on the memory of meaningless syllables, is well known (Figure 3). Half of what you learn is forgotten within 30 minutes, and only 20% is remembered after one month. If you approach learning with interest and curiosity, the retention rate will be higher. There are short-term and long-term memories. Short-term memories become long-term memories through 1. repetition, 2. visualization, 3. giving meaning, and 4. episodicization. In games, visualization and episodicization progress in a short period of time through experience. Games have also been introduced in elderly care facilities, and their effectiveness in delaying dementia has been confirmed. By incorporating game elements, students can learn in a fun and natural way, stimulating their interest, concern, and competitive spirit. [Brief description of the drawings]

[0011] [Figure 1] FIG. 1 is a diagram showing an example of an embodiment of the present invention. [Diagram 2] FIG. 2 is a diagram showing the implementation of the present invention in each phase. (Example) [Diagram 3] Ebbinghaus's forgetting curve (supplementary diagram to make the contents of this patent easier to understand) [Figure 4] An example of the development software MATLAB (registered trademark)_simulink (registered trademark) ((supplementary figure to make the contents of this patent easier to understand) [Diagram 5] A diagram showing the scope of the claims of the present invention in accordance with the claims (supplementary diagram to make the contents of this patent easier to understand) DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] The system is composed of a convolutional neural network that takes action at a specific level in response to read-aloud input, a multi-layer neural network that performs deep learning, and a generative AI that displays the results.This allows the system to realize an information processing device that can review and evaluate the results of read-aloud input as well as allow users to experience the joy of reading aloud. EXAMPLES

[0013] Fig. 1 is a diagram showing an embodiment of the device of the present invention. From the left, there is a convolutional neural network that takes action at a specific level in response to an input, a multi-layer neural network that performs deep learning, and a generation AI that displays the results. Figure 2 shows the implementation of the device of the present invention in each phase. From the left, it shows the reading aloud actions of the learner and the model reader, the evaluation method and system, and the displayed results. Figures 3 to 5 are supplementary and previously known information to make the contents of this patent easier to understand. Figure 3 shows the Ebbinghaus forgetting curve, a theory in educational psychology about short-term and long-term memory. Figure 4 shows an example of high-level language software MATLAB (registered trademark)_simulink (registered trademark), which is often used in software development. FIG. 5 is a diagram showing the scope of the claims of the present invention in an easy-to-understand manner. Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings. Figure 1 is an example diagram showing an embodiment of the present invention, and Figure 2 is a diagram showing the implementation content of the present invention by each phase. The explanation will be given along the flow shown in Figure 2.

[0014] A "reading behavior" is a network action1 at a certain level that is performed on speech that is read aloud. The speech of the learner and the model reader is filtered to remove background sounds and noise, and the language read aloud by one or more people is processed by a convolutional neural network (used in the field of image recognition, which processes information simultaneously in parallel). In the convolutional neural network 4, various filters (audio data 5, audio analysis 6, frequency characteristics 7) are applied to each learning image in multiple convolution layers, and the output of each convolutional image is used as the input for the next layer. In other words, the output of all feature maps is the result of applying filters to the image, and the new feature map becomes the input for the next stage. In the process, the speech is divided into linguistic units (vowels, consonants, syllables, words, phrases, onomatopoeia, voicings, stress, pauses, rhythm, etc.), extracted, analyzed, and digitized. Language is also extracted as sound by image recognition of waveforms, patterns, frequencies, distributions, etc., and analyzed and digitized in the same way. As mentioned above, the voice of the sample reader is also analyzed and digitized. The results are output as outputs X1, X2, X3,... to the "Evaluation Method / System" Deep Learning 2.

[0015] In the "evaluation method and system", deep learning 2 is performed using a multi-layer neural network 8 with X1, X2, X3, ... as input. The multi-layer neural network 8 is composed of an input layer, a hidden layer, and an output layer. Multi-layer neural networks have been widely used in AI (artificial intelligence) learning in recent years. The overall evaluation is performed by reviewing against the objectives. The neural network is multi-layered, and a deep learning algorithm is used to automatically extract and analyze features from the data, and automatically update and learn the weight parameters.

[0016] Using the neural network, evaluation results data such as noticing misreadings of characters, being able to read by dividing them into phrases or words, being able to read vocabulary and kanji, the time it takes to read aloud, onomatopoeia, pronunciation, stress, pauses, rhythm, etc. are extracted as evaluation items such as natural pronunciation, fluent pronunciation, easy to listen to, being able to read material appropriate for the age, and good reading skills, and the achieved level is assessed using the evaluation method and system described below.

[0017] The system evaluates not only the results but also the process and takes into account the reasons and justification for the selection. For example, it considers the reader's enthusiasm and perseverance, such as the number of times reading aloud per unit time and the volume of the reading voice. DLA evaluation is performed in accordance with the DLA (Dialogue Language Assessment) of the Ministry of Education, Culture, Sports, Science and Technology. AI is used to evaluate the learning process and daily learning results for each achievement stage evaluation and overall evaluation of "vocabulary, kanji, reading aloud behavior, reading comprehension, reading behavior, reading habits, interests, and attitude" in accordance with the implementation standards of DLA <Reading>. In addition to the DLA evaluation, the evaluation is also performed based on the sample audio or the audio selected by the learner as a sample, and a differential evaluation is performed on these. The evaluation is also performed in the same way for audio in which multiple learners participate competitively.

[0018] "Result display" is performed by the generation AI 3. A generation AI is an artificial intelligence capable of generating various contents such as images, videos, text, audio, program code, and structured data. In this embodiment, the basic generation AI 9 is composed of an encoder, a decoder, a vector (a matrix sequence representing the input contents), and a generator. The images, videos, text, and audio (for example, "Amazing," "Almost there," etc.) generated by the basic generation AI 9 are displayed in an appropriate form, and the related program code, structured data, etc. are generated 10.

[0019] The evaluation is based on the DLA (Interactive Language Assessment) of the Ministry of Education, Culture, Sports, Science and Technology, and on the sample voice or voice selected by the learner as a sample. When there is a sample, the generative AI network 11 is composed of a generator, a real sample, a generated sample, and a discriminator. The real sample is a group of data extracted by a circuit not shown in the figure after passing through the multi-layer neural network 8 as a sample sound source and recorded and stored, and the generated sample is that of a learner who has passed through the basic generative AI 9. Both are fed back between the generator and the discriminator, and a comparison evaluation is made between real and fake. The results are incorporated into the basic generative AI 9 and displayed as images, videos, sentences, voices, program codes related to them, structured data, etc. The above is also the case when evaluation is based on voices in which multiple learners compete.

[0020] Although the embodiment of the present invention has been described in detail above with reference to the drawings, the specific configuration is not limited to the embodiment of the present invention, and even if there are design changes within the scope of the present invention, the present invention is included. For example, instead of processing the read-aloud language using a convolutional neural network, it may be processed using a recurrent neural network (used for natural language processing that handles words such as translation and dialogue, and is good at analyzing sequence data), or a logistic regression model analysis process may be performed without making it a network program. The screening phase may also be performed using a logistic regression model. The logistic regression model is also excellent as a method for increasing the success rate of results and improving cost performance. When processing with a logistic regression model, natural pronunciation, fluent pronunciation, easy to listen to, reading appropriate material for age, reading skill, etc. are set as objective variables, and reading sentences and meanings separately, noticing misreadings, reading phrases and words separately, vocabulary and kanji reading, reading time, katakana English, consonant strength, sharpness, smoothness, rhythm, etc. are set as explanatory variables, factors are predicted, results are displayed by factor, and examination is performed in light of the objective. At first glance, the patent described in this case seems to be broad-ranging and complex, and unlikely to be realized, but it goes without saying that the above-mentioned network program and logistic regression program can be easily created using high-level language software such as MATLAB (registered trademark) and simulink (registered trademark) (Figure 4). [Industrial Applicability]

[0021] It can be widely used in the field of audiovisual education, which is fundamental in multilingual language learning. It can be used as an independent audiovisual educational device as well as on network devices (mobile phones, PCs, TVs, game consoles, etc.). [Explanation of symbols]

[0022] 1. Network actions at a particular level 2. Deep Learning System 3. Generative AI System 4. Convolutional Neural Networks 5. Input voice data processing system 6. Voice Analysis System 7 Frequency response analysis system 8 Multilayer Neural Networks 9 Generation AI 10 Various content generated, such as images, videos, text, audio, program code, and structured data 11 Generative AI network when examples are available

Claims

1. A system that examines the spoken language and displays the results

2. The system according to claim 1, which uses artificial intelligence (AI) to examine the spoken language and display the results.

3. A system for examining the spoken language using a logistic regression model and displaying the results as claimed in claim 2.

4. A system for examining the language read aloud as claimed in claim 2 by deep learning using a neural network and displaying the results.

5. A system according to claim 4, which examines the read language using a convolutional neural network and displays the results.

6. A system according to claim 4, which examines the spoken language using a recurrent neural network and displays the results.

7. A system according to claim 4, which examines the spoken language using a multi-layer neural network and displays the results.

8. A system for examining the language read aloud as described in claim 2 using a generation AI and displaying the results.

9. A system for generating voices, images, comments, and program codes and structured data related thereto, such as "amazing," "close," and "good job," as described in claim 8.

10. In claim 1, the review is a system that evaluates not only the results but also the process.

11. The system according to claim 1, in which the evaluation is performed in accordance with the DLA (Interactive Language Assessment) of the Ministry of Education, Culture, Sports, Science and Technology.

12. A system as claimed in claim 3, which predicts factors using natural pronunciation, fluent pronunciation, easy to listen to, being able to read age-appropriate material, reading skill, etc. as objective variables, and can read sentences or chunks of meaning separately, notice misreading of characters, can read phrases or words separately, can read vocabulary or kanji, time required for reading aloud, onomatopoeia, pronunciation, stress, pauses, rhythm, etc. as explanatory variables, displays the results by factor, and performs an overall evaluation by reviewing them against the objectives.

13. A system according to claim 5, which performs image recognition of speech data of a spoken language, such as waveforms, patterns, frequencies, distributions, etc., and displays the results after examining the data.

14. In claim 9, the voice is generated by a generation AI, and the system uses images, videos, text, voice, program codes related thereto, structured data, etc.

15. In claim 10, the screening is a system that considers the reasons and grounds for selection, and also considers the reader's enthusiasm and perseverance, such as the number of times the reader reads aloud in a unit time and the volume of the reading voice.

16. A system according to claim 1, in which the assessment is based on a sample speech or a speech selected by the learner as a sample.

17. The system according to claim 1, in which the assessment is based on the speech of a plurality of learners who compete with each other.

Images