System

Abstract

To provide a system according to an embodiment that improves pronunciation and provides cultural background information through an interaction in a language selected by a user.SOLUTION: A system according to an embodiment comprises an interaction unit, a pronunciation analysis unit, a quiz unit, and a background providing unit. The interaction unit performs an interaction with a user in a language selected by the user. The pronunciation analysis unit analyzes in real time the pronunciation in the interaction with the user performed by the interaction unit, and provides a specific improvement point. The quiz unit provides, through the interaction unit, an interactive quiz related to the grammar and vocabularies according to the progress of the user's language learning. The background providing unit provides, through the interaction unit, information on the culture and customs in a country where the language selected by the user is used.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including a description and related instruction sentence regarding the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2022-180282 A Summary of the Invention [Problem to be solved by the invention]

[0004] Conventional techniques do not adequately provide real-time pronunciation analysis or cultural background information for language learning, leaving room for improvement.

[0005] The system according to the embodiment aims to improve pronunciation and provide cultural background information through dialogue in a language selected by the user. [Means for solving the problem]

[0006] The system according to the embodiment includes a dialogue unit, a pronunciation analysis unit, a quiz unit, and a background provision unit. The dialogue unit dialogues with a user in a language selected by the user. The pronunciation analysis unit analyzes the pronunciation of the user's dialogue with the dialogue unit in real time and provides specific improvements. The quiz unit provides interactive quizzes on grammar and vocabulary according to the user's language learning progress through the dialogue unit. The background provision unit provides information on the culture and customs of the language selected by the user through the dialogue unit. Effect of the Invention

[0007] The system according to the embodiment can improve pronunciation and provide cultural background information through dialogue in a language selected by the user. [Brief description of the drawings]

[0008] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Diagram 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. FIG. [Diagram 3] FIG. 11 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Diagram 5] FIG. 13 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 13 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 13 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0010] First, the terms used in the following description will be explained.

[0011] In the following embodiments, a signed processor (hereinafter simply referred to as a "processor") may be one arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be one type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), or a TPU (Tensor Processing Unit).

[0012] In the following embodiments, a signed RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by the processor.

[0013] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0014] In the following embodiments, a communication I / F (Interface) with a code is an interface including a communication processor and an antenna. The communication I / F controls communication between multiple computers. An example of a communication standard applied to the communication I / F is a wireless communication standard including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0015] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. In addition, in this specification, the same idea as "A and / or B" is also applied when three or more things are expressed by connecting them with "and / or."

[0016] [First embodiment] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0017] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0018] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0019] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0020] The reception device 38 includes a touch panel 38A and a microphone 38B, and receives user input. The touch panel 38A detects contact with a pointer (e.g., a pen or a finger) to receive user input by the touch of the pointer. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 (see FIG. 2) acquires the data indicating the user input.

[0021] Output device 40 includes a display 40A and a speaker 40B, and presents data to a user by outputting the data in a form of expression that the user can perceive (e.g., voice and / or text). Display 40A displays visible information such as text and images according to instructions from processor 46. Speaker 40B outputs voice according to instructions from processor 46. Camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a Complementary Metal-Oxide-Semiconductor (CMOS) image sensor or a Charge Coupled Device (CCD) image sensor.

[0022] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for transmitting and receiving various types of information between the processor 46 and the processor 28 via the network 54.

[0023] FIG. 2 shows an example of main functions of the data processing device 12 and the smart device 14.

[0024] As shown in Fig. 2, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32. The specific process program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific process program 56 from the storage 32, and executes the read specific process program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific process program 56 executed on the RAM 30.

[0025] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the emotion of the user using the emotion identification model 59 and perform identification processing using the user's emotion. In the emotion estimation function (emotion identification function) using the emotion identification model 59, various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, are performed, but are not limited to such examples. In addition, the estimation and prediction of the emotion also include, for example, analysis of the emotion.

[0026] In the smart device 14, the specific processing is performed by the processor 46. The storage 50 stores a specific processing program 60. The specific processing program 60 is used together with the specific processing program 56 by the data processing system 10. The processor 46 reads the specific processing program 60 from the storage 50 and executes the specific processing program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as the control unit 46A in accordance with the specific processing program 60 executed on the RAM 48. Note that the smart device 14 has a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can also perform processing similar to that of the specific processing unit 290 using these models.

[0027] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device (e.g., a generation server) may have the data generation model 58. In this case, the data processing device 12 obtains a processing result (prediction result, etc.) using the data generation model 58 by communicating with the server device having the data generation model 58. Also, the data processing device 12 may be a server device, or may be a terminal device (e.g., a mobile phone, a robot, a home appliance, etc.) owned by a user. Next, an example of processing by the data processing system 10 according to the first embodiment will be described.

[0028] (Example 1) A conversational application AI assistant for language learning according to an embodiment of the present invention is a system that provides interactive practice, allowing a user to converse in a new language in real time and receive instant feedback. The system includes a dialogue unit that dialogues in a language selected by the user, and supports language learning through dialogue with the user. The system further includes a pronunciation analysis unit that analyzes the user's pronunciation in real time and provides improvements and advice. For example, when the user pronounces "hello", the system evaluates the accuracy of the pronunciation and presents necessary improvements. The system also includes a quiz unit that provides interactive quizzes on grammar and vocabulary according to the user's language learning progress. For example, if the user is at a beginner's level, quizzes on basic grammar and vocabulary are presented to advance the user's learning. The system also includes a background provision unit that provides information on the culture and customs of the language being learned, and naturally introduces cultural background information during the dialogue. There is also a situation-specific learning unit that provides practice for different situations, allowing the user to practice conversations tailored to specific situations, such as travel, business, and daily conversation. For example, practice based on actual situations, such as reserving a hotel at a travel destination and ordering at a restaurant, is performed. The system also includes a daily task learning unit that practices daily language tasks, providing practice in completing daily tasks in language, such as "explaining today's weather" and "ordering at a restaurant." In addition, the system also includes a story providing unit that provides interactive content in which the user is the protagonist and progresses through a story using language. The system also includes a progress tracking unit that tracks the user's learning progress and ability and recommends optimal lessons and practice, efficiently supporting the user's learning. For example, if the user is not good at a particular grammar item, the system recommends lessons related to that grammar item. The system also integrates multimedia that combines different media such as audio, text, and video to provide a rich learning experience. For example, in pronunciation practice, the system plays the audio of a native speaker, allowing the user to practice pronunciation along with it. In this way, the present invention provides a conversational application AI assistant with a variety of functions to enable users to efficiently learn new languages.This allows conversational application AI assistants for language learning to help users learn new languages ​​efficiently.

[0029] A conversational application AI assistant for language learning according to an embodiment includes a dialogue unit, a pronunciation analysis unit, a quiz unit, and a background provision unit. The dialogue unit dialogues with a user in a language selected by the user. The language selected by the user includes, but is not limited to, English, French, Chinese, and the like. The dialogue unit can dialogue in the form of, for example, voice dialogue, text dialogue, and the like. The pronunciation analysis unit analyzes the pronunciation of the user's dialogue with the dialogue unit in real time and provides specific improvements. The real-time analysis is performed within a delay tolerance range, for example, within 1 second or within 5 seconds. For example, when a user pronounces "hello," the pronunciation analysis unit evaluates the accuracy of the pronunciation and presents improvements to the intonation. The quiz unit provides an interactive quiz on grammar and vocabulary according to the progress of the user's language learning. Quiz formats include, for example, multiple choice questions and fill-in-the-blank questions. For example, when a user is at a beginner's level, the quiz unit asks quizzes on basic grammar and vocabulary to advance learning. The background providing unit provides information about the culture and customs of the language selected by the user. The provided information includes, for example, food culture, holidays, and social customs. The background providing unit, for example, naturally introduces cultural background information during a conversation. In this way, the conversational application AI assistant for language learning according to the embodiment can support the user to efficiently learn a new language.

[0030] The dialogue unit dialogues with the user in a language selected by the user. Examples of the language selected by the user include, but are not limited to, English, French, and Chinese. The dialogue unit can dialogue in the form of, for example, voice dialogue, text dialogue, and the like. Specifically, in the case of voice dialogue, when the user speaks into a microphone, the dialogue unit recognizes the voice and generates an appropriate response. For voice recognition, a voice recognition engine is used to convert the user's utterance into text. In the case of text dialogue, the dialogue unit generates a response to the text entered by the user using a keyboard or touch screen. The dialogue unit uses natural language processing technology to analyze the user's intention and generate an appropriate response. For example, when the user asks, "What's the weather like today?", the dialogue unit acquires weather information and responds, "It's sunny today." The dialogue unit can not only generate grammatically correct responses to the user's utterances and inputs, but also conduct natural dialogue according to the context. This allows the user to learn a language in a form close to an actual conversation. Furthermore, the dialogue unit can customize the content of the dialogue according to the user's learning progress and interests. For example, if the user is interested in traveling, a dialogue containing many phrases and words related to traveling is provided. This allows users to learn a language efficiently while keeping their interest.

[0031] The pronunciation analysis unit analyzes the pronunciation of the user's dialogue with the dialogue unit in real time and provides specific points for improvement. The real-time analysis is performed within a delay tolerance range, such as within 1 second or 5 seconds. For example, when a user pronounces "hello", the pronunciation analysis unit evaluates the accuracy of the pronunciation and presents points for improvement in intonation. Specifically, the pronunciation analysis unit uses voice recognition technology to analyze the user's pronunciation and compares it with standard pronunciation. As a result of the comparison, it evaluates how accurate the user's pronunciation is and provides specific feedback. For example, if the "ko" part is too low in the pronunciation of "hello", it instructs the user to pronounce the "ko" sound a little higher. In addition, the pronunciation analysis unit can learn the user's pronunciation pattern and provide optimal feedback to each individual user. For example, if a particular user has difficulty pronouncing a particular sound, it provides feedback focusing on that sound. In addition, the pronunciation analysis unit can track the user's improvement in pronunciation and record progress. This allows the user to check how much their pronunciation has improved and maintain their motivation. The pronunciation analyzer also acts as a guide for the user when practicing pronunciation, helping the user to master correct pronunciation.

[0032] The quiz section provides interactive quizzes on grammar and vocabulary according to the progress of the user's language learning. Quiz formats include, for example, multiple choice questions and fill-in-the-blank questions. For example, if the user is at a beginner's level, the quiz section poses quizzes on basic grammar and vocabulary to advance the user's learning. Specifically, the quiz section generates quizzes of appropriate difficulty based on the user's learning history and progress. For example, the quiz section poses quizzes including words and phrases the user has recently learned to encourage the user to review. In the case of multiple choice questions, the user selects the correct answer from multiple options, and can proceed to the next question if the user answers correctly. In the case of fill-in-the-blank questions, the user enters the appropriate word or phrase in the blank in the sentence, and can proceed to the next question if the user answers correctly. The quiz section provides the user with the correct answer and an explanation for questions that the user gets wrong, thereby supporting the user in deepening their understanding. Furthermore, the quiz section can analyze the user's answer data and identify the user's weak points and areas of expertise. This allows the user to be provided with individually customized quizzes to support efficient learning. For example, if a user feels that he or she is weak at a particular grammar point, the quiz section can provide many quizzes related to that grammar point, allowing the user to focus on that point in their learning. This allows the quiz section to maximize the effectiveness of the user's learning and efficiently support language acquisition.

[0033] The background providing unit provides information on the culture and customs of the language selected by the user. The provided information includes, for example, food culture, holidays, social customs, and the like. The background providing unit, for example, naturally introduces cultural background information during a dialogue. Specifically, when the dialogue unit dialogues with the user, it inserts related cultural information. For example, if the user asks, "Tell me about French food," the background providing unit provides information on France's representative dishes and eating customs. In addition, the background providing unit can customize the information provided according to the user's learning progress and interests. For example, if the user is interested in traveling, a lot of cultural information related to travel is provided. In addition, the background providing unit can provide information on the history, literature, music, and the like of the language the user is learning. This allows the user to deepen their understanding not only of the language but also of the culture and society in which the language is used. The background providing unit is also useful when simulating a dialogue with a native speaker of the language the user is learning. For example, the user can learn how to behave in a particular cultural situation. This allows the user to dialogue with confidence in real communication. The context provider serves to enrich the user's learning experience and increase motivation for language learning.

[0034] The situation-specific learning unit can perform language practice based on a specific situation through the dialogue unit. The situation-specific learning unit provides language practice based on a specific situation, such as ordering at a restaurant or checking in at an airport. The situation-specific learning unit can perform practice through the dialogue unit based on a situation selected by the user. For example, if a user wants to practice making a hotel reservation at a travel destination, the situation-specific learning unit provides a dialogue related to hotel reservations. Also, if a user wants to practice conversation in a business setting, the situation-specific learning unit provides a dialogue related to business meetings and presentations. This makes it possible to practice conversation tailored to a specific situation. Some or all of the above-mentioned processing in the situation-specific learning unit may be performed using, for example, AI, or may be performed without using AI. For example, the situation-specific learning unit can generate a dialogue scenario using an AI model based on a situation selected by the user and provide it through the dialogue unit.

[0035] The daily task learning unit can practice specific daily tasks in a language selected by the user through the dialogue unit. The daily task learning unit provides specific daily tasks such as shopping and how to make a phone call. The daily task learning unit can practice through the dialogue unit based on the daily task selected by the user. For example, if the user wants to practice "explaining today's weather", the daily task learning unit provides a dialogue related to the weather. Also, if the user wants to practice "ordering at a restaurant", the daily task learning unit provides a dialogue related to ordering at a restaurant. This allows practice in accomplishing daily tasks in a language. Some or all of the above-mentioned processing in the daily task learning unit may be performed, for example, using AI, or may be performed without using AI. For example, the daily task learning unit can generate a dialogue scenario using an AI model based on the daily task selected by the user and provide it through the dialogue unit.

[0036] The story providing unit can provide specific interactive content in which the user becomes the main character and progresses the story using language. The story providing unit provides specific interactive content such as a story in which an option is selected, an interactive adventure game, etc. The story providing unit can provide interactive content through the dialogue unit based on the story selected by the user. For example, if the user selects an adventure story, the story providing unit provides adventure options and progresses the story based on the option selected by the user. Also, if the user selects a mystery story, the story providing unit provides mystery options and progresses the story based on the option selected by the user. This enables practical language learning by the user progressing through the story using language. Part or all of the above-mentioned processing in the story providing unit may be performed, for example, using AI, or may be performed without using AI. For example, the story providing unit can generate a story scenario using an AI model based on the story selected by the user, and provide it through the dialogue unit.

[0037] The dialogue unit can perform a specific dialogue with the user by combining voice, text, and video. The dialogue unit can perform a dialogue with the user in the form of, for example, voice dialogue, text dialogue, video dialogue, and the like. The dialogue unit can perform a dialogue by combining voice, text, and video based on the dialogue format selected by the user. For example, if the user selects voice dialogue, the dialogue unit performs a dialogue using voice. Also, if the user selects text dialogue, the dialogue unit performs a dialogue using text. Furthermore, if the user selects video dialogue, the dialogue unit performs a dialogue using video. This provides a rich learning experience by combining voice, text, and video. Part or all of the above-mentioned processing in the dialogue unit may be performed, for example, using AI, or may be performed without using AI. For example, the dialogue unit can generate a dialogue scenario using an AI model based on the dialogue format selected by the user, and provide a combination of voice, text, and video.

[0038] The progress tracking unit can track the user's learning progress and ability and recommend specific lessons and exercises. The progress tracking unit can track the user's learning progress and ability and recommend optimal lessons and exercises. The progress tracking unit performs tracking based on evaluation criteria such as the user's learning time and correct answer rate. The progress tracking unit recommends specific lessons and exercises such as grammar lessons and pronunciation exercises according to the user's learning progress. For example, if the user is not good at a specific grammar item, the progress tracking unit recommends lessons related to that grammar item. Also, if the user feels that pronunciation is a problem, the progress tracking unit recommends pronunciation exercises. This provides optimal lessons and exercises according to the user's learning progress. Part or all of the above-mentioned processing in the progress tracking unit may be performed using, for example, AI, or may be performed without using AI. For example, the progress tracking unit can input the user's learning data into an AI model and recommend optimal lessons and exercises.

[0039] The feedback providing unit can provide specific feedback based on the user's learning progress and ability. The feedback providing unit provides individual feedback based on the user's learning progress and ability. The feedback providing unit provides feedback based on evaluation criteria such as the user's learning time and correct answer rate. The feedback providing unit provides specific feedback such as text feedback and voice feedback according to the user's learning progress. For example, if the user is not good at a specific grammar item, the feedback providing unit provides feedback on specific improvements regarding the grammar item. Also, if the user feels that pronunciation is a problem, the feedback providing unit provides feedback on specific advice regarding pronunciation. This provides individual feedback according to the user's learning progress. A part or all of the above-mentioned processing in the feedback providing unit may be performed using, for example, AI, or may be performed without using AI. For example, the feedback providing unit can input the user's learning data into an AI model to generate individual feedback.

[0040] The pronunciation analysis unit can visually display the analysis result of the user's pronunciation. The display format includes, for example, a graph display, a text display, and the like. The pronunciation analysis unit visually displays the accuracy of the user's pronunciation and the improvement points of the intonation. For example, the pronunciation analysis unit displays the accuracy of the user's pronunciation in a graph to indicate which part needs improvement. In addition, the pronunciation analysis unit displays the improvement points of the user's intonation in text and provides specific advice. As a result, the analysis result of the user's pronunciation is visually displayed, making it easier to understand the improvement points. A part or all of the above-mentioned processing in the pronunciation analysis unit may be performed, for example, using AI, or may be performed without using AI. For example, the pronunciation analysis unit can input the user's pronunciation data into an AI model and generate a visual display.

[0041] The story providing unit can provide an interactive story with different endings based on the user's selection. The types of endings include, for example, a success ending and a failure ending. The story providing unit can change the development of the story based on the choice selected by the user. For example, if the user selects a specific choice in an adventure story, the story progresses based on that choice and reaches a different ending. Also, if the user selects a different choice in a mystery story, the story progresses based on that choice and reaches a different ending. This increases the fun of learning by providing a story with a different ending based on the user's selection. Some or all of the above-mentioned processing in the story providing unit may be performed, for example, using AI or may be performed without using AI. For example, the story providing unit can input the user's selection data into an AI model to generate different endings.

[0042] The pronunciation analysis unit can present specific improvements based on the results of comparing the user's pronunciation with a native speaker. The standards for native speakers include, for example, a native speaker of American English, a native speaker of British English, and the like. The pronunciation analysis unit compares the user's pronunciation with the pronunciation of a native speaker and indicates which part needs improvement. For example, the pronunciation analysis unit compares the accuracy of the user's pronunciation with that of a native speaker and presents specific improvements. In addition, the pronunciation analysis unit compares the user's intonation with that of a native speaker and provides specific advice. In this way, the user's pronunciation is compared with that of a native speaker and specific improvements are presented. A part or all of the above-mentioned processing in the pronunciation analysis unit may be performed, for example, using AI or may be performed without using AI. For example, the pronunciation analysis unit can input the user's pronunciation data into an AI model and generate a comparison result with a native speaker.

[0043] The dialogue unit can analyze the user's past dialogue history and select an optimal dialogue scenario. The types of dialogue scenarios include, for example, a scenario for beginners and a scenario for intermediate users. The dialogue unit analyzes the user's past dialogue history and selects an optimal dialogue scenario based on the user's interests and learning progress. For example, the dialogue scenario is selected based on topics in which the user has shown interest in the past. In addition, the dialogue unit avoids topics that the user has had difficulty with in the past and focuses on topics that the user is good at. Furthermore, a scenario according to the progress of learning is selected from the user's past dialogue history. As a result, an optimal dialogue scenario is selected based on the user's past dialogue history. A part or all of the above-mentioned processing in the dialogue unit may be performed using, for example, AI, or may be performed without using AI. For example, the dialogue unit can input the user's dialogue history data into an AI model to generate an optimal dialogue scenario.

[0044] The dialogue unit can provide additional information based on the user's interests and concerns according to the content of the dialogue. The form of the additional information includes, for example, related articles, video links, and the like. The dialogue unit provides the additional information based on the interests and concerns expressed by the user during the dialogue. For example, when the user is having a dialogue about travel, the dialogue unit provides tourist information about the travel destination. Also, when the user is having a dialogue about business, the dialogue unit provides related business news. Furthermore, when the user is having a dialogue about culture, the dialogue unit provides detailed information about the culture. As a result, the dialogue is enriched by providing additional information based on the user's interests and concerns. Some or all of the above-mentioned processing in the dialogue unit may be performed, for example, using AI, or may be performed without using AI. For example, the dialogue unit can input the user's dialogue data into an AI model to generate related additional information.

[0045] The dialogue unit can provide highly relevant topics during dialogue, taking into account the geographical location information of the user. Highly relevant topics include, for example, local news, local events, and the like. The dialogue unit adjusts the content of the dialogue, taking into account the geographical location information of the user. For example, when the user is in a specific city, the dialogue unit provides topics related to the city. When the user is traveling, the dialogue unit provides tourist information about the travel destination. Furthermore, when the user is in a specific region, the dialogue unit provides topics related to the culture and customs of the region. This provides highly relevant topics based on the geographical location information of the user. Some or all of the above-mentioned processing in the dialogue unit may be performed, for example, using AI, or may be performed without using AI. For example, the dialogue unit can input the geographical location data of the user into an AI model to generate highly relevant topics.

[0046] The dialogue unit can analyze the user's social media activity during the dialogue and provide related topics. The related topics include, for example, topics based on the user's interests and topics related to recent posts. The dialogue unit analyzes the user's social media activity and adjusts the content of the dialogue. For example, the dialogue unit provides topics based on the content shared by the user on social media. Also, the dialogue unit provides topics based on the content posted by the accounts the user follows. Furthermore, the dialogue unit provides topics based on groups and events in which the user participates. In this way, related topics are provided based on the user's social media activity. Some or all of the above-mentioned processing in the dialogue unit may be performed, for example, using AI, or may be performed without using AI. For example, the dialogue unit can input the user's social media data into an AI model to generate related topics.

[0047] The pronunciation analysis unit can present the improvement points by referring to the user's past pronunciation data during the pronunciation analysis. The past pronunciation data includes, for example, recording data, pronunciation scores, and the like. The pronunciation analysis unit refers to the user's past pronunciation data and indicates which parts need improvement. For example, the pronunciation analysis unit presents the improvement points based on data of words that the user has pronounced in the past. Also, the pronunciation analysis unit presents the improvement points for specific sounds from the user's past pronunciation data. Furthermore, the pronunciation analysis unit analyzes the user's past pronunciation data and presents the overall improvement points. As a result, specific improvement points are presented based on the user's past pronunciation data. A part or all of the above-mentioned processing in the pronunciation analysis unit may be performed using, for example, AI, or may be performed without using AI. For example, the pronunciation analysis unit can input the user's past pronunciation data into an AI model to generate improvement points.

[0048] The pronunciation analysis unit can perform an analysis corresponding to different accents and dialects during the pronunciation analysis. The types of accents and dialects include, for example, American English accents and British English dialects. When a user pronounces with a specific accent, the pronunciation analysis unit provides feedback based on the accent. For example, when a user pronounces with an American English accent, the pronunciation analysis unit provides feedback based on the accent. Also, when a user pronounces with a dialect, the pronunciation analysis unit provides feedback based on the dialect. Furthermore, when a user wants to learn a different accent or dialect, the pronunciation analysis unit performs an analysis corresponding to the accent. This enables pronunciation analysis corresponding to different accents and dialects. A part or all of the above-mentioned processing in the pronunciation analysis unit may be performed using, for example, AI, or may be performed without using AI. For example, the pronunciation analysis unit can input the user's pronunciation data into an AI model and generate feedback corresponding to different accents and dialects.

[0049] The pronunciation analysis unit can provide related pronunciation feedback in consideration of the geographical location information of the user during the pronunciation analysis. The related pronunciation feedback includes, for example, regional pronunciation, local accent, and the like. The pronunciation analysis unit provides the pronunciation feedback in consideration of the geographical location information of the user. For example, when the user is in a specific region, the pronunciation analysis unit provides feedback based on the accent of the region. Also, when the user is traveling, the pronunciation analysis unit provides feedback on the pronunciation of the travel destination. Furthermore, when the user is in a specific country, the pronunciation analysis unit provides feedback on the pronunciation of the country. This provides related pronunciation feedback based on the geographical location information of the user. Some or all of the above-mentioned processing in the pronunciation analysis unit may be performed, for example, using AI, or may be performed without using AI. For example, the pronunciation analysis unit can input the geographical location data of the user into an AI model to generate related pronunciation feedback.

[0050] The pronunciation analysis unit can analyze the user's social media activities during the pronunciation analysis and provide related pronunciation feedback. The related pronunciation feedback includes, for example, pronunciation related to recent posts, pronunciation based on interests, and the like. The pronunciation analysis unit analyzes the user's social media activities and provides pronunciation feedback. For example, the pronunciation analysis unit provides pronunciation feedback based on the content shared by the user on social media. Also, the pronunciation analysis unit provides pronunciation feedback based on the content posted by the accounts the user follows. Furthermore, the pronunciation analysis unit provides pronunciation feedback based on groups and events in which the user participates. In this way, related pronunciation feedback is provided based on the user's social media activities. Some or all of the above-mentioned processing in the pronunciation analysis unit may be performed using, for example, AI, or may be performed without using AI. For example, the pronunciation analysis unit can input the user's social media data into an AI model to generate related pronunciation feedback.

[0051] When providing a quiz, the quiz section can select the most suitable question by referring to the user's past quiz results. The most suitable question includes, for example, a question based on the user's past mistakes and a question according to the learning progress. The quiz section selects the most suitable question by referring to the user's past quiz results. For example, the quiz section re-asks the question that the user got wrong in the past. Also, the quiz section asks a new question based on the question that the user was good at in the past. Furthermore, the quiz section selects a question according to the learning progress from the user's past quiz results. In this way, the most suitable question is selected based on the user's past quiz results. A part or all of the above-mentioned processing in the quiz section may be performed, for example, using AI, or may be performed without using AI. For example, the quiz section can input the user's quiz result data into an AI model to generate the most suitable question.

[0052] When providing a quiz, the quiz unit can provide a quiz format corresponding to different learning styles. The learning styles include, for example, visual learning, auditory learning, and the like. The quiz unit adjusts the format of the quiz according to the learning style of the user. For example, a visual learner is provided with a quiz using images and videos. Also, an auditory learner is provided with a quiz using audio. Furthermore, a tactile learner is provided with an interactive quiz. This provides a quiz format corresponding to different learning styles. A part or all of the above-mentioned processing in the quiz unit may be performed, for example, using AI, or may be performed without using AI. For example, the quiz unit can input the learning style data of the user into an AI model to generate an optimal quiz format.

[0053] When providing a quiz, the quiz section can provide relevant questions taking into account the geographical location information of the user. The relevant questions include, for example, questions specific to a region and questions related to a local culture. The quiz section adjusts the content of the quiz taking into account the geographical location information of the user. For example, if the user is in a specific region, the quiz section asks questions related to the region. Also, if the user is traveling, the quiz section asks questions related to the travel destination. Furthermore, if the user is in a specific country, the quiz section asks questions related to the country. This provides relevant questions based on the geographical location information of the user. Some or all of the above-mentioned processing in the quiz section may be performed, for example, using AI, or may be performed without using AI. For example, the quiz section can input the geographical location data of the user into an AI model to generate relevant questions.

[0054] When providing a quiz, the quiz section can analyze the social media activity of the user and provide relevant questions. The relevant questions include, for example, questions related to recent posts and questions based on interests. The quiz section analyzes the social media activity of the user and adjusts the content of the quiz. For example, questions are provided based on the content shared by the user on social media. Also, questions are provided based on the content posted by the accounts the user follows. Furthermore, questions are provided based on groups and events in which the user participates. In this way, relevant questions are provided based on the social media activity of the user. Some or all of the above-mentioned processing in the quiz section may be performed, for example, using AI, or may be performed without using AI. For example, the quiz section can input the social media data of the user into an AI model to generate relevant questions.

[0055] The background providing unit can select optimal information by referring to the user's past learning history when providing background information. The optimal information includes, for example, information based on past learning contents, information based on interests, and the like. The background providing unit selects optimal information by referring to the user's past learning history. For example, new information is provided based on cultural information that the user has learned in the past. In addition, cultural information that is likely to interest the user is selected from the user's past learning history. Furthermore, the user's past learning history is analyzed, and cultural information according to the progress of learning is provided. As a result, optimal information is selected based on the user's past learning history. A part or all of the above-mentioned processing in the background providing unit may be performed, for example, using AI, or may be performed without using AI. For example, the background providing unit can input the user's learning history data into an AI model to generate optimal information.

[0056] The background providing unit can provide information corresponding to different cultural backgrounds when providing background information. Different cultural backgrounds include, for example, Asian culture, European culture, and the like. The background providing unit adjusts the content of the information based on the cultural background of the user. For example, if the user is interested in a particular culture, information about that culture is provided. Also, if the user has a different cultural background, information based on that background is provided. Furthermore, information based on the cultural background of the language the user is learning is provided. This provides information corresponding to different cultural backgrounds. A part or all of the above-mentioned processing in the background providing unit may be performed, for example, using AI, or may be performed without using AI. For example, the background providing unit can input the user's cultural background data into an AI model to generate related information.

[0057] The background providing unit can provide related information in consideration of the geographical location information of the user when providing background information. The related information includes, for example, information specific to a region, information related to a local culture, and the like. The background providing unit adjusts the content of the information in consideration of the geographical location information of the user. For example, when the user is in a specific region, the background providing unit provides cultural information of the region. Also, when the user is traveling, the background providing unit provides cultural information of the travel destination. Furthermore, when the user is in a specific country, the background providing unit provides cultural information of the country. As a result, related information is provided based on the geographical location information of the user. A part or all of the above-mentioned processing in the background providing unit may be performed, for example, using AI, or may be performed without using AI. For example, the background providing unit can input the geographical location data of the user into an AI model to generate related information.

[0058] The background providing unit can analyze the user's social media activity and provide related information when providing background information. The related information includes, for example, information related to recent posts and information based on interests. The background providing unit analyzes the user's social media activity and adjusts the content of the information. For example, the background providing unit provides cultural information based on the content shared by the user on social media. Also, the background providing unit provides cultural information based on the content posted by the accounts the user follows. Furthermore, the background providing unit provides cultural information based on groups and events in which the user participates. As a result, related information is provided based on the user's social media activity. A part or all of the above-mentioned processing in the background providing unit may be performed, for example, using AI, or may be performed without using AI. For example, the background providing unit can input the user's social media data into an AI model to generate related information.

[0059] The situation-specific learning unit can select the optimal situation by referring to the user's past learning history during situation-specific learning. The optimal situation includes, for example, a situation based on past learning content, a situation based on interest, and the like. The situation-specific learning unit selects the optimal situation by referring to the user's past learning history. For example, a new situation is selected based on a situation that the user has learned in the past. In addition, a situation that is likely to interest the user is selected from the user's past learning history. Furthermore, the user's past learning history is analyzed, and a situation according to the progress of learning is selected. As a result, the optimal situation is selected based on the user's past learning history. A part or all of the above-mentioned processing in the situation-specific learning unit may be performed, for example, using AI, or may be performed without using AI. For example, the situation-specific learning unit can input the user's learning history data into an AI model to generate an optimal situation.

[0060] The situation-specific learning unit can provide relevant situations in consideration of the user's geographical location information during situation-specific learning. The relevant situations include, for example, regional situations and situations related to local culture. The situation-specific learning unit adjusts the content of the situation in consideration of the user's geographical location information. For example, when the user is in a specific region, the situation-specific learning unit provides a situation related to the region. Also, when the user is traveling, the situation-specific learning unit provides a situation related to the travel destination. Furthermore, when the user is in a specific country, the situation related to the country is provided. As a result, relevant situations are provided based on the user's geographical location information. Part or all of the above-mentioned processing in the situation-specific learning unit may be performed, for example, using AI, or may be performed without using AI. For example, the situation-specific learning unit can input the user's geographical location data into an AI model to generate relevant situations.

[0061] The daily task learning unit can select an optimal task by referring to the user's past learning history during daily task learning. The optimal task includes, for example, a task based on past learning content, a task based on interests, and the like. The daily task learning unit selects an optimal task by referring to the user's past learning history. For example, a new task is selected based on a task the user has learned in the past. In addition, a task that is likely to interest the user is selected from the user's past learning history. Furthermore, the user's past learning history is analyzed, and a task according to the progress of learning is selected. In this way, an optimal task is selected based on the user's past learning history. A part or all of the above-mentioned processing in the daily task learning unit may be performed, for example, using AI, or may be performed without using AI. For example, the daily task learning unit can input the user's learning history data into an AI model to generate an optimal task.

[0062] The daily task learning unit can provide a related task in consideration of the geographical location information of the user during daily task learning. The related tasks include, for example, region-specific tasks and tasks related to local culture. The daily task learning unit adjusts the content of the task in consideration of the geographical location information of the user. For example, when the user is in a specific region, the daily task learning unit provides a task related to the region. Also, when the user is traveling, the daily task learning unit provides a task related to the travel destination. Furthermore, when the user is in a specific country, the daily task learning unit provides a task related to the country. As a result, a related task is provided based on the geographical location information of the user. Part or all of the above-mentioned processing in the daily task learning unit may be performed, for example, using AI, or may be performed without using AI. For example, the daily task learning unit can input the geographical location data of the user into an AI model to generate a related task.

[0063] The story providing unit can select an optimal story development by referring to the user's past selection history when providing a story. The optimal story development includes, for example, a development based on past selections and a development based on interests. The story providing unit selects an optimal story development by referring to the user's past selection history. For example, a new development is selected based on a story development selected by the user in the past. Also, a story development that is likely to interest the user is selected from the user's past selection history. Furthermore, the user's past selection history is analyzed, and a story development according to the progress of learning is selected. As a result, an optimal story development is selected based on the user's past selection history. A part or all of the above-mentioned processing in the story providing unit may be performed, for example, using AI, or may be performed without using AI. For example, the story providing unit can input the user's selection history data into an AI model to generate an optimal story development.

[0064] The story providing unit can provide related stories in consideration of the geographical location information of the user when providing a story. The related stories include, for example, stories specific to a region and stories related to a local culture. The story providing unit adjusts the content of the story in consideration of the geographical location information of the user. For example, when the user is in a specific region, the story providing unit provides a story related to the region. Also, when the user is traveling, the story providing unit provides a story related to the travel destination. Furthermore, when the user is in a specific country, the story providing unit provides a story related to the country. As a result, related stories are provided based on the geographical location information of the user. A part or all of the above-mentioned processing in the story providing unit may be performed, for example, using AI, or may be performed without using AI. For example, the story providing unit can input the geographical location data of the user into an AI model to generate related stories.

[0065] The progress tracking unit can select an optimal tracking method by referring to the user's past learning history when tracking the progress. The optimal tracking method includes, for example, a tracking method based on past learning contents, a tracking method based on interests, and the like. The progress tracking unit selects an optimal tracking method by referring to the user's past learning history. For example, a new tracking method is selected based on the tracking method used by the user in the past. Also, a tracking method that is likely to interest the user is selected from the user's past learning history. Furthermore, the user's past learning history is analyzed, and a tracking method according to the progress of learning is selected. As a result, the optimal tracking method is selected based on the user's past learning history. A part or all of the above-mentioned processing in the progress tracking unit may be performed, for example, using AI, or may be performed without using AI. For example, the progress tracking unit can input the user's learning history data into an AI model to generate an optimal tracking method.

[0066] The progress tracking unit can provide relevant tracking information in consideration of the geographical location information of the user during progress tracking. The relevant tracking information includes, for example, information specific to a region, information related to a local culture, and the like. The progress tracking unit adjusts the content of the tracking information in consideration of the geographical location information of the user. For example, when the user is in a specific region, the progress tracking unit provides tracking information related to the region. Also, when the user is traveling, the progress tracking unit provides tracking information related to the travel destination. Furthermore, when the user is in a specific country, the progress tracking unit provides tracking information related to the country. This provides relevant tracking information based on the geographical location information of the user. Some or all of the above-mentioned processing in the progress tracking unit may be performed, for example, using AI, or may be performed without using AI. For example, the progress tracking unit can input the geographical location data of the user into an AI model to generate relevant tracking information.

[0067] The feedback providing unit can select optimal feedback by referring to the user's past learning history when providing feedback. The optimal feedback includes, for example, feedback based on past learning contents, feedback based on interests, and the like. The feedback providing unit selects optimal feedback by referring to the user's past learning history. For example, new feedback is provided based on feedback the user has received in the past. Also, feedback that is likely to interest the user is selected from the user's past learning history. Furthermore, the user's past learning history is analyzed, and feedback according to the progress of learning is provided. As a result, optimal feedback is selected based on the user's past learning history. A part or all of the above-mentioned processing in the feedback providing unit may be performed, for example, using AI, or may be performed without using AI. For example, the feedback providing unit can input the user's learning history data into an AI model to generate optimal feedback.

[0068] The feedback providing unit can provide relevant feedback in consideration of the geographical location information of the user when providing feedback. The relevant feedback includes, for example, region-specific feedback, feedback related to local culture, and the like. The feedback providing unit adjusts the content of the feedback in consideration of the geographical location information of the user. For example, when the user is in a specific region, the feedback providing unit provides feedback related to the region. Also, when the user is traveling, the feedback providing unit provides feedback related to the travel destination. Furthermore, when the user is in a specific country, the feedback providing unit provides feedback related to the country. This provides relevant feedback based on the geographical location information of the user. Some or all of the above-mentioned processing in the feedback providing unit may be performed, for example, using AI, or may be performed without using AI. For example, the feedback providing unit can input the geographical location data of the user into an AI model to generate relevant feedback.

[0069] The system according to the embodiment is not limited to the above-described example, and various modifications are possible, for example, as follows.

[0070] The dialogue unit can analyze the dialogue history of the user, identify pronunciations and grammar mistakes made by the user in the past, and generate a dialogue scenario that focuses on practicing those parts. For example, if the user has made mistakes in pronouncing "r" and "l" in the past, the dialogue unit provides a dialogue scenario including that pronunciation. Also, if the user has made mistakes in a specific grammatical structure in the past, the dialogue unit provides a dialogue scenario including that grammatical structure. Furthermore, if the user has made mistakes in a specific vocabulary in the past, the dialogue unit provides a dialogue scenario including that vocabulary. This enables effective practice based on the user's past mistakes.

[0071] The situation-specific learning unit can customize the method of practicing situations according to the user's learning style. For example, visual learners can be provided with situation practice using images and videos. Also, auditory learners can be provided with situation practice centered on audio. Furthermore, tactile learners can be provided with interactive situation practice. In this way, the most suitable situation practice according to the user's learning style can be provided.

[0072] The daily task learning unit can provide region-specific daily tasks by taking into account the geographic location information of the user. For example, if the user is in Japan, daily tasks based on Japanese culture and customs are provided. If the user is in France, daily tasks based on French culture and customs are provided. If the user is in America, daily tasks based on American culture and customs are provided. This provides related daily tasks based on the geographic location information of the user.

[0073] The story providing unit can analyze the user's social media activity and provide stories that the user may be interested in. For example, if the user posts a lot about travel on social media, stories related to travel are provided. If the user posts a lot about cooking, stories related to cooking are provided. If the user posts a lot about sports, stories related to sports are provided. In this way, relevant stories are provided based on the user's social media activity.

[0074] The dialogue unit can adjust the difficulty of the dialogue according to the user's learning progress. For example, if the user is at a beginner's level, an easy dialogue scenario is provided. If the user is at an intermediate level, a somewhat difficult dialogue scenario is provided. If the user is at an advanced level, a more difficult dialogue scenario is provided. In this way, an optimal dialogue according to the user's learning progress is provided.

[0075] The process flow of the first embodiment will be briefly described below.

[0076] Step 1: The dialogue unit dialogues with the user in a language selected by the user. The language selected by the user includes, but is not limited to, English, French, Chinese, etc. The dialogue unit can dialogue in the form of, for example, voice dialogue, text dialogue, etc. Step 2: The pronunciation analysis unit analyzes the pronunciation of the user's dialogue with the dialogue unit in real time and provides specific improvements. The real-time analysis is performed within a delay tolerance of, for example, 1 second or 5 seconds. For example, when the user pronounces "hello," the pronunciation analysis unit evaluates the accuracy of the pronunciation and provides improvements on intonation. Step 3: The quiz section provides interactive quizzes on grammar and vocabulary according to the user's language learning progress. Quiz formats include, for example, multiple choice questions, fill-in-the-blank questions, etc. For example, if the user is at a beginner's level, the quiz section provides quizzes on basic grammar and vocabulary to advance the user's learning. Step 4: The background provider provides information about the culture and customs of the language selected by the user. The provided information includes, for example, food culture, holidays, social customs, etc. The background provider introduces cultural background information naturally during the dialogue, for example.

[0077] (Example 2) A conversational application AI assistant for language learning according to an embodiment of the present invention is a system that provides interactive practice, allowing a user to converse in a new language in real time and receive instant feedback. The system includes a dialogue unit that dialogues in a language selected by the user, and supports language learning through dialogue with the user. The system further includes a pronunciation analysis unit that analyzes the user's pronunciation in real time and provides improvements and advice. For example, when the user pronounces "hello", the system evaluates the accuracy of the pronunciation and presents necessary improvements. The system also includes a quiz unit that provides interactive quizzes on grammar and vocabulary according to the user's language learning progress. For example, if the user is at a beginner's level, quizzes on basic grammar and vocabulary are presented to advance the user's learning. The system also includes a background provision unit that provides information on the culture and customs of the language being learned, and naturally introduces cultural background information during the dialogue. There is also a situation-specific learning unit that provides practice for different situations, allowing the user to practice conversations tailored to specific situations, such as travel, business, and daily conversation. For example, practice based on actual situations, such as reserving a hotel at a travel destination and ordering at a restaurant, is performed. The system also includes a daily task learning unit that practices daily language tasks, providing practice in completing daily tasks in language, such as "explaining today's weather" and "ordering at a restaurant." In addition, the system also includes a story providing unit that provides interactive content in which the user is the protagonist and progresses through a story using language. The system also includes a progress tracking unit that tracks the user's learning progress and ability and recommends optimal lessons and practice, efficiently supporting the user's learning. For example, if the user is not good at a particular grammar item, the system recommends lessons related to that grammar item. The system also integrates multimedia that combines different media such as audio, text, and video to provide a rich learning experience. For example, in pronunciation practice, the system plays the audio of a native speaker, allowing the user to practice pronunciation along with it. In this way, the present invention provides a conversational application AI assistant with a variety of functions to enable users to efficiently learn new languages.This allows conversational application AI assistants for language learning to help users learn new languages ​​efficiently.

[0078] A conversational application AI assistant for language learning according to an embodiment includes a dialogue unit, a pronunciation analysis unit, a quiz unit, and a background provision unit. The dialogue unit dialogues with a user in a language selected by the user. The language selected by the user includes, but is not limited to, English, French, Chinese, and the like. The dialogue unit can dialogue in the form of, for example, voice dialogue, text dialogue, and the like. The pronunciation analysis unit analyzes the pronunciation of the user's dialogue with the dialogue unit in real time and provides specific improvements. The real-time analysis is performed within a delay tolerance range, for example, within 1 second or within 5 seconds. For example, when a user pronounces "hello," the pronunciation analysis unit evaluates the accuracy of the pronunciation and presents improvements to the intonation. The quiz unit provides an interactive quiz on grammar and vocabulary according to the progress of the user's language learning. Quiz formats include, for example, multiple choice questions and fill-in-the-blank questions. For example, when a user is at a beginner's level, the quiz unit asks quizzes on basic grammar and vocabulary to advance learning. The background providing unit provides information about the culture and customs of the language selected by the user. The provided information includes, for example, food culture, holidays, and social customs. The background providing unit, for example, naturally introduces cultural background information during a conversation. In this way, the conversational application AI assistant for language learning according to the embodiment can support the user to efficiently learn a new language.

[0079] The dialogue unit dialogues with the user in a language selected by the user. Examples of the language selected by the user include, but are not limited to, English, French, and Chinese. The dialogue unit can dialogue in the form of, for example, voice dialogue, text dialogue, and the like. Specifically, in the case of voice dialogue, when the user speaks into a microphone, the dialogue unit recognizes the voice and generates an appropriate response. For voice recognition, a voice recognition engine is used to convert the user's utterance into text. In the case of text dialogue, the dialogue unit generates a response to the text entered by the user using a keyboard or touch screen. The dialogue unit uses natural language processing technology to analyze the user's intention and generate an appropriate response. For example, when the user asks, "What's the weather like today?", the dialogue unit acquires weather information and responds, "It's sunny today." The dialogue unit can not only generate grammatically correct responses to the user's utterances and inputs, but also conduct natural dialogue according to the context. This allows the user to learn a language in a form close to an actual conversation. Furthermore, the dialogue unit can customize the content of the dialogue according to the user's learning progress and interests. For example, if the user is interested in traveling, a dialogue containing many phrases and words related to traveling is provided. This allows users to learn a language efficiently while keeping their interest.

[0080] The pronunciation analysis unit analyzes the pronunciation of the user's dialogue with the dialogue unit in real time and provides specific points for improvement. The real-time analysis is performed within a delay tolerance range, such as within 1 second or 5 seconds. For example, when a user pronounces "hello", the pronunciation analysis unit evaluates the accuracy of the pronunciation and presents points for improvement in intonation. Specifically, the pronunciation analysis unit uses voice recognition technology to analyze the user's pronunciation and compares it with standard pronunciation. As a result of the comparison, it evaluates how accurate the user's pronunciation is and provides specific feedback. For example, if the "ko" part is too low in the pronunciation of "hello", it instructs the user to pronounce the "ko" sound a little higher. In addition, the pronunciation analysis unit can learn the user's pronunciation pattern and provide optimal feedback to each individual user. For example, if a particular user has difficulty pronouncing a particular sound, it provides feedback focusing on that sound. In addition, the pronunciation analysis unit can track the user's improvement in pronunciation and record progress. This allows the user to check how much their pronunciation has improved and maintain their motivation. The pronunciation analyzer also acts as a guide for the user when practicing pronunciation, helping the user to master correct pronunciation.

[0081] The quiz section provides interactive quizzes on grammar and vocabulary according to the progress of the user's language learning. Quiz formats include, for example, multiple choice questions and fill-in-the-blank questions. For example, if the user is at a beginner's level, the quiz section poses quizzes on basic grammar and vocabulary to advance the user's learning. Specifically, the quiz section generates quizzes of appropriate difficulty based on the user's learning history and progress. For example, the quiz section poses quizzes including words and phrases the user has recently learned to encourage the user to review. In the case of multiple choice questions, the user selects the correct answer from multiple options, and can proceed to the next question if the user answers correctly. In the case of fill-in-the-blank questions, the user enters the appropriate word or phrase in the blank in the sentence, and can proceed to the next question if the user answers correctly. The quiz section provides the user with the correct answer and an explanation for questions that the user gets wrong, thereby supporting the user in deepening their understanding. Furthermore, the quiz section can analyze the user's answer data and identify the user's weak points and areas of expertise. This allows the user to be provided with individually customized quizzes to support efficient learning. For example, if a user feels that he or she is weak at a particular grammar point, the quiz section can provide many quizzes related to that grammar point, allowing the user to focus on that point in their learning. This allows the quiz section to maximize the effectiveness of the user's learning and efficiently support language acquisition.

[0082] The background providing unit provides information on the culture and customs of the language selected by the user. The provided information includes, for example, food culture, holidays, social customs, and the like. The background providing unit, for example, naturally introduces cultural background information during a dialogue. Specifically, when the dialogue unit dialogues with the user, it inserts related cultural information. For example, if the user asks, "Tell me about French food," the background providing unit provides information on France's representative dishes and eating customs. In addition, the background providing unit can customize the information provided according to the user's learning progress and interests. For example, if the user is interested in traveling, a lot of cultural information related to travel is provided. In addition, the background providing unit can provide information on the history, literature, music, and the like of the language the user is learning. This allows the user to deepen their understanding not only of the language but also of the culture and society in which the language is used. The background providing unit is also useful when simulating a dialogue with a native speaker of the language the user is learning. For example, the user can learn how to behave in a particular cultural situation. This allows the user to dialogue with confidence in real communication. The context provider serves to enrich the user's learning experience and increase motivation for language learning.

[0083] The situation-specific learning unit can perform language practice based on a specific situation through the dialogue unit. The situation-specific learning unit provides language practice based on a specific situation, such as ordering at a restaurant or checking in at an airport. The situation-specific learning unit can perform practice through the dialogue unit based on a situation selected by the user. For example, if a user wants to practice making a hotel reservation at a travel destination, the situation-specific learning unit provides a dialogue related to hotel reservations. Also, if a user wants to practice conversation in a business setting, the situation-specific learning unit provides a dialogue related to business meetings and presentations. This makes it possible to practice conversation tailored to a specific situation. Some or all of the above-mentioned processing in the situation-specific learning unit may be performed using, for example, AI, or may be performed without using AI. For example, the situation-specific learning unit can generate a dialogue scenario using an AI model based on a situation selected by the user and provide it through the dialogue unit.

[0084] The daily task learning unit can practice specific daily tasks in a language selected by the user through the dialogue unit. The daily task learning unit provides specific daily tasks such as shopping and how to make a phone call. The daily task learning unit can practice through the dialogue unit based on the daily task selected by the user. For example, if the user wants to practice "explaining today's weather", the daily task learning unit provides a dialogue related to the weather. Also, if the user wants to practice "ordering at a restaurant", the daily task learning unit provides a dialogue related to ordering at a restaurant. This allows practice in accomplishing daily tasks in a language. Some or all of the above-mentioned processing in the daily task learning unit may be performed, for example, using AI, or may be performed without using AI. For example, the daily task learning unit can generate a dialogue scenario using an AI model based on the daily task selected by the user and provide it through the dialogue unit.

[0085] The story providing unit can provide specific interactive content in which the user becomes the main character and progresses the story using language. The story providing unit provides specific interactive content such as a story in which an option is selected, an interactive adventure game, etc. The story providing unit can provide interactive content through the dialogue unit based on the story selected by the user. For example, if the user selects an adventure story, the story providing unit provides adventure options and progresses the story based on the option selected by the user. Also, if the user selects a mystery story, the story providing unit provides mystery options and progresses the story based on the option selected by the user. This enables practical language learning by the user progressing through the story using language. Part or all of the above-mentioned processing in the story providing unit may be performed, for example, using AI, or may be performed without using AI. For example, the story providing unit can generate a story scenario using an AI model based on the story selected by the user, and provide it through the dialogue unit.

[0086] The dialogue unit can perform a specific dialogue with the user by combining voice, text, and video. The dialogue unit can perform a dialogue with the user in the form of, for example, voice dialogue, text dialogue, video dialogue, and the like. The dialogue unit can perform a dialogue by combining voice, text, and video based on the dialogue format selected by the user. For example, if the user selects voice dialogue, the dialogue unit performs a dialogue using voice. Also, if the user selects text dialogue, the dialogue unit performs a dialogue using text. Furthermore, if the user selects video dialogue, the dialogue unit performs a dialogue using video. This provides a rich learning experience by combining voice, text, and video. Part or all of the above-mentioned processing in the dialogue unit may be performed, for example, using AI, or may be performed without using AI. For example, the dialogue unit can generate a dialogue scenario using an AI model based on the dialogue format selected by the user, and provide a combination of voice, text, and video.

[0087] The progress tracking unit can track the user's learning progress and ability and recommend specific lessons and exercises. The progress tracking unit can track the user's learning progress and ability and recommend optimal lessons and exercises. The progress tracking unit performs tracking based on evaluation criteria such as the user's learning time and correct answer rate. The progress tracking unit recommends specific lessons and exercises such as grammar lessons and pronunciation exercises according to the user's learning progress. For example, if the user is not good at a specific grammar item, the progress tracking unit recommends a lesson related to that grammar item. Also, if the user feels that pronunciation is a problem, the progress tracking unit recommends pronunciation exercises. This provides optimal lessons and exercises according to the user's learning progress. Part or all of the above-mentioned processing in the progress tracking unit may be performed using, for example, AI, or may be performed without using AI. For example, the progress tracking unit can input the user's learning data into an AI model and recommend optimal lessons and exercises.

[0088] The feedback providing unit can provide specific feedback based on the user's learning progress and ability. The feedback providing unit provides individual feedback based on the user's learning progress and ability. The feedback providing unit provides feedback based on evaluation criteria such as the user's learning time and correct answer rate. The feedback providing unit provides specific feedback such as text feedback and voice feedback according to the user's learning progress. For example, if the user is not good at a specific grammar item, the feedback providing unit provides feedback on specific improvements regarding the grammar item. Also, if the user feels that pronunciation is a problem, the feedback providing unit provides feedback on specific advice regarding pronunciation. This provides individual feedback according to the user's learning progress. A part or all of the above-mentioned processing in the feedback providing unit may be performed using, for example, AI, or may be performed without using AI. For example, the feedback providing unit can input the user's learning data into an AI model to generate individual feedback.

[0089] The pronunciation analysis unit can visually display the analysis result of the user's pronunciation. The display format includes, for example, a graph display, a text display, and the like. The pronunciation analysis unit visually displays the accuracy of the user's pronunciation and the improvement points of the intonation. For example, the pronunciation analysis unit displays the accuracy of the user's pronunciation in a graph to indicate which part needs improvement. In addition, the pronunciation analysis unit displays the improvement points of the user's intonation in text and provides specific advice. As a result, the analysis result of the user's pronunciation is visually displayed, making it easier to understand the improvement points. A part or all of the above-mentioned processing in the pronunciation analysis unit may be performed, for example, using AI, or may be performed without using AI. For example, the pronunciation analysis unit can input the user's pronunciation data into an AI model and generate a visual display.

[0090] The story providing unit can provide an interactive story with different endings based on the user's selection. The types of endings include, for example, a success ending and a failure ending. The story providing unit can change the development of the story based on the choice selected by the user. For example, if the user selects a specific choice in an adventure story, the story progresses based on that choice and reaches a different ending. Also, if the user selects a different choice in a mystery story, the story progresses based on that choice and reaches a different ending. This increases the fun of learning by providing a story with a different ending based on the user's selection. Some or all of the above-mentioned processing in the story providing unit may be performed, for example, using AI or may be performed without using AI. For example, the story providing unit can input the user's selection data into an AI model to generate different endings.

[0091] The pronunciation analysis unit can present specific improvements based on the results of comparing the user's pronunciation with a native speaker. The standards for native speakers include, for example, a native speaker of American English, a native speaker of British English, and the like. The pronunciation analysis unit compares the user's pronunciation with the pronunciation of a native speaker and indicates which part needs improvement. For example, the pronunciation analysis unit compares the accuracy of the user's pronunciation with that of a native speaker and presents specific improvements. In addition, the pronunciation analysis unit compares the user's intonation with that of a native speaker and provides specific advice. In this way, the user's pronunciation is compared with that of a native speaker and specific improvements are presented. A part or all of the above-mentioned processing in the pronunciation analysis unit may be performed, for example, using AI or may be performed without using AI. For example, the pronunciation analysis unit can input the user's pronunciation data into an AI model and generate a comparison result with a native speaker.

[0092] The dialogue unit can estimate the user's emotion and adjust the tone and speed of the dialogue based on the estimated user's emotion. The types of emotions include, for example, joy, sadness, and anger. The dialogue unit estimates the user's emotion and adjusts the tone and speed of the dialogue according to the emotion. For example, if the user is nervous, the dialogue unit makes the tone gentle and the speed slow. Also, if the user is relaxed, the dialogue unit makes the tone bright and the speed normal. Furthermore, if the user is in a hurry, the dialogue unit speeds up and engages in dialogue that focuses on the main points. As a result, the tone and speed of the dialogue are adjusted according to the user's emotion, making it possible to have a more appropriate dialogue. The emotion estimation is realized using an emotion estimation function, for example, using an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the dialogue unit may be performed, for example, using an AI, or may be performed without using an AI. For example, the dialogue unit can input user emotional data into the AI ​​model to adjust the tone and pace of the dialogue.

[0093] The dialogue unit can analyze the user's past dialogue history and select an optimal dialogue scenario. The types of dialogue scenarios include, for example, a scenario for beginners and a scenario for intermediate users. The dialogue unit analyzes the user's past dialogue history and selects an optimal dialogue scenario based on the user's interests and learning progress. For example, the dialogue scenario is selected based on topics in which the user has shown interest in the past. In addition, the dialogue unit avoids topics that the user has had difficulty with in the past and focuses on topics that the user is good at. Furthermore, a scenario according to the progress of learning is selected from the user's past dialogue history. As a result, an optimal dialogue scenario is selected based on the user's past dialogue history. A part or all of the above-mentioned processing in the dialogue unit may be performed using, for example, AI, or may be performed without using AI. For example, the dialogue unit can input the user's dialogue history data into an AI model to generate an optimal dialogue scenario.

[0094] The dialogue unit can provide additional information based on the user's interests and concerns according to the content of the dialogue. The form of the additional information includes, for example, related articles, video links, and the like. The dialogue unit provides the additional information based on the interests and concerns expressed by the user during the dialogue. For example, when the user is having a dialogue about travel, the dialogue unit provides tourist information about the travel destination. Also, when the user is having a dialogue about business, the dialogue unit provides related business news. Furthermore, when the user is having a dialogue about culture, the dialogue unit provides detailed information about the culture. As a result, the dialogue is enriched by providing additional information based on the user's interests and concerns. Some or all of the above-mentioned processing in the dialogue unit may be performed, for example, using AI, or may be performed without using AI. For example, the dialogue unit can input the user's dialogue data into an AI model to generate related additional information.

[0095] The dialogue unit can estimate the user's emotion and select a dialogue theme based on the estimated user's emotion. The dialogue theme includes, for example, travel, work, and hobbies. The dialogue unit estimates the user's emotion and selects a dialogue theme according to the emotion. For example, if the user is sad, the dialogue unit selects a fun theme to ease the mood. Also, if the user is excited, the dialogue unit selects an interesting theme. Furthermore, if the user is tired, the dialogue unit selects a relaxing theme. As a result, a more appropriate dialogue is possible by selecting a dialogue theme according to the user's emotion. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the dialogue unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the dialogue unit can input the user's emotion data into an AI model and select a dialogue theme.

[0096] The dialogue unit can provide highly relevant topics during dialogue, taking into account the geographical location information of the user. Highly relevant topics include, for example, local news, local events, and the like. The dialogue unit adjusts the content of the dialogue, taking into account the geographical location information of the user. For example, when the user is in a specific city, the dialogue unit provides topics related to the city. When the user is traveling, the dialogue unit provides tourist information about the travel destination. Furthermore, when the user is in a specific region, the dialogue unit provides topics related to the culture and customs of the region. This provides highly relevant topics based on the geographical location information of the user. Some or all of the above-mentioned processing in the dialogue unit may be performed, for example, using AI, or may be performed without using AI. For example, the dialogue unit can input the geographical location data of the user into an AI model to generate highly relevant topics.

[0097] The dialogue unit can analyze the user's social media activity during the dialogue and provide related topics. The related topics include, for example, topics based on the user's interests and topics related to recent posts. The dialogue unit analyzes the user's social media activity and adjusts the content of the dialogue. For example, the dialogue unit provides topics based on the content shared by the user on social media. Also, the dialogue unit provides topics based on the content posted by the accounts the user follows. Furthermore, the dialogue unit provides topics based on groups and events in which the user participates. In this way, related topics are provided based on the user's social media activity. Some or all of the above-mentioned processing in the dialogue unit may be performed, for example, using AI, or may be performed without using AI. For example, the dialogue unit can input the user's social media data into an AI model to generate related topics.

[0098] The pronunciation analysis unit can estimate the user's emotion and adjust the expression method of the pronunciation feedback based on the estimated user's emotion. The feedback expression method includes, for example, a gentle expression, a strict expression, and the like. The pronunciation analysis unit estimates the user's emotion and adjusts the expression method of the feedback according to the emotion. For example, if the user is nervous, the pronunciation analysis unit provides feedback in gentle words. Furthermore, if the user is relaxed, the pronunciation analysis unit provides detailed feedback. Furthermore, if the user is in a hurry, the pronunciation analysis unit provides concise feedback. This adjusts the expression method of the pronunciation feedback according to the user's emotion. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the pronunciation analysis unit may be performed, for example, using an AI, or may be performed without using an AI. For example, the pronunciation analysis unit can input the user's emotion data into an AI model and adjust the expression method of the feedback.

[0099] The pronunciation analysis unit can present the improvement points by referring to the user's past pronunciation data during the pronunciation analysis. The past pronunciation data includes, for example, recording data, pronunciation scores, and the like. The pronunciation analysis unit refers to the user's past pronunciation data and indicates which parts need improvement. For example, the pronunciation analysis unit presents the improvement points based on data of words that the user has pronounced in the past. Also, the pronunciation analysis unit presents the improvement points for specific sounds from the user's past pronunciation data. Furthermore, the pronunciation analysis unit analyzes the user's past pronunciation data and presents the overall improvement points. As a result, specific improvement points are presented based on the user's past pronunciation data. A part or all of the above-mentioned processing in the pronunciation analysis unit may be performed using, for example, AI, or may be performed without using AI. For example, the pronunciation analysis unit can input the user's past pronunciation data into an AI model to generate improvement points.

[0100] The pronunciation analysis unit can perform an analysis corresponding to different accents and dialects during the pronunciation analysis. The types of accents and dialects include, for example, American English accents and British English dialects. When a user pronounces with a specific accent, the pronunciation analysis unit provides feedback based on the accent. For example, when a user pronounces with an American English accent, the pronunciation analysis unit provides feedback based on the accent. Also, when a user pronounces with a dialect, the pronunciation analysis unit provides feedback based on the dialect. Furthermore, when a user wants to learn a different accent or dialect, the pronunciation analysis unit performs an analysis corresponding to the accent. This enables pronunciation analysis corresponding to different accents and dialects. A part or all of the above-mentioned processing in the pronunciation analysis unit may be performed using, for example, AI, or may be performed without using AI. For example, the pronunciation analysis unit can input the user's pronunciation data into an AI model and generate feedback corresponding to different accents and dialects.

[0101] The pronunciation analysis unit can estimate the user's emotion and determine the priority of the pronunciation feedback based on the estimated user's emotion. The priority of the feedback includes, for example, importance and urgency. The pronunciation analysis unit estimates the user's emotion and determines the priority of the feedback according to the emotion. For example, when the user is nervous, the pronunciation analysis unit gives priority to important improvement points and feedback. Also, when the user is relaxed, the pronunciation analysis unit gives detailed improvement points and feedback. Furthermore, when the user is in a hurry, the pronunciation analysis unit gives priority to the most important improvement points and feedback. In this way, the priority of the pronunciation feedback is determined according to the user's emotion. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the pronunciation analysis unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the pronunciation analysis unit can input the user's emotion data into an AI model and determine the priority of the feedback.

[0102] The pronunciation analysis unit can provide related pronunciation feedback in consideration of the geographical location information of the user during the pronunciation analysis. The related pronunciation feedback includes, for example, regional pronunciation, local accent, and the like. The pronunciation analysis unit provides the pronunciation feedback in consideration of the geographical location information of the user. For example, when the user is in a specific region, the pronunciation analysis unit provides feedback based on the accent of the region. Also, when the user is traveling, the pronunciation analysis unit provides feedback on the pronunciation of the travel destination. Furthermore, when the user is in a specific country, the pronunciation analysis unit provides feedback on the pronunciation of the country. This provides related pronunciation feedback based on the geographical location information of the user. Some or all of the above-mentioned processing in the pronunciation analysis unit may be performed, for example, using AI, or may be performed without using AI. For example, the pronunciation analysis unit can input the geographical location data of the user into an AI model to generate related pronunciation feedback.

[0103] The pronunciation analysis unit can analyze the user's social media activities during the pronunciation analysis and provide related pronunciation feedback. The related pronunciation feedback includes, for example, pronunciation related to recent posts, pronunciation based on interests, and the like. The pronunciation analysis unit analyzes the user's social media activities and provides pronunciation feedback. For example, the pronunciation analysis unit provides pronunciation feedback based on the content shared by the user on social media. Also, the pronunciation analysis unit provides pronunciation feedback based on the content posted by the accounts the user follows. Furthermore, the pronunciation analysis unit provides pronunciation feedback based on groups and events in which the user participates. In this way, related pronunciation feedback is provided based on the user's social media activities. Some or all of the above-mentioned processing in the pronunciation analysis unit may be performed using, for example, AI, or may be performed without using AI. For example, the pronunciation analysis unit can input the user's social media data into an AI model to generate related pronunciation feedback.

[0104] The quiz section can estimate the user's emotions and adjust the difficulty of the quiz based on the estimated user's emotions. The difficulty of the quiz includes, for example, the difficulty of the questions and the frequency of questions. The quiz section estimates the user's emotions and adjusts the difficulty of the quiz according to the emotions. For example, if the user is nervous, the quiz section asks easy questions. Furthermore, if the user is relaxed, the quiz section asks questions of normal difficulty. Furthermore, if the user is excited, the quiz section asks questions of high difficulty. This adjusts the difficulty of the quiz according to the user's emotions. The estimation of emotions is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the quiz section may be performed using, for example, an AI, or may be performed without using an AI. For example, the quiz section can input the user's emotion data into an AI model and adjust the difficulty of the quiz.

[0105] When providing a quiz, the quiz section can select the most suitable question by referring to the user's past quiz results. The most suitable question includes, for example, a question based on the user's past mistakes and a question according to the learning progress. The quiz section selects the most suitable question by referring to the user's past quiz results. For example, the quiz section re-asks the question that the user got wrong in the past. Also, the quiz section asks a new question based on the question that the user was good at in the past. Furthermore, the quiz section selects a question according to the learning progress from the user's past quiz results. In this way, the most suitable question is selected based on the user's past quiz results. A part or all of the above-mentioned processing in the quiz section may be performed, for example, using AI, or may be performed without using AI. For example, the quiz section can input the user's quiz result data into an AI model to generate the most suitable question.

[0106] When providing a quiz, the quiz unit can provide a quiz format corresponding to different learning styles. The learning styles include, for example, visual learning, auditory learning, and the like. The quiz unit adjusts the format of the quiz according to the learning style of the user. For example, a visual learner is provided with a quiz using images and videos. Also, an auditory learner is provided with a quiz using audio. Furthermore, a tactile learner is provided with an interactive quiz. This provides a quiz format corresponding to different learning styles. A part or all of the above-mentioned processing in the quiz unit may be performed, for example, using AI, or may be performed without using AI. For example, the quiz unit can input the learning style data of the user into an AI model to generate an optimal quiz format.

[0107] The quiz section can estimate the user's emotion and adjust the order of quiz questions based on the estimated user's emotion. The order of quiz questions includes, for example, order of difficulty and random order. The quiz section estimates the user's emotion and adjusts the order of quiz questions according to the emotion. For example, when the user is nervous, the quiz section asks questions starting from easy questions. Furthermore, when the user is relaxed, the quiz section asks questions in the normal order. Furthermore, when the user is excited, the quiz section asks questions starting from difficult questions. In this way, the order of quiz questions is adjusted according to the user's emotion. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the quiz section may be performed using, for example, an AI, or may be performed without using an AI. For example, the quiz section can input the user's emotion data into an AI model and adjust the order of quiz questions.

[0108] When providing a quiz, the quiz section can provide relevant questions taking into account the geographical location information of the user. The relevant questions include, for example, questions specific to a region and questions related to a local culture. The quiz section adjusts the content of the quiz taking into account the geographical location information of the user. For example, if the user is in a specific region, the quiz section asks questions related to the region. Also, if the user is traveling, the quiz section asks questions related to the travel destination. Furthermore, if the user is in a specific country, the quiz section asks questions related to the country. This provides relevant questions based on the geographical location information of the user. Some or all of the above-mentioned processing in the quiz section may be performed, for example, using AI, or may be performed without using AI. For example, the quiz section can input the geographical location data of the user into an AI model to generate relevant questions.

[0109] When providing a quiz, the quiz section can analyze the social media activity of the user and provide relevant questions. The relevant questions include, for example, questions related to recent posts and questions based on interests. The quiz section analyzes the social media activity of the user and adjusts the content of the quiz. For example, questions are provided based on the content shared by the user on social media. Also, questions are provided based on the content posted by the accounts the user follows. Furthermore, questions are provided based on groups and events in which the user participates. In this way, relevant questions are provided based on the social media activity of the user. Some or all of the above-mentioned processing in the quiz section may be performed, for example, using AI, or may be performed without using AI. For example, the quiz section can input the social media data of the user into an AI model to generate relevant questions.

[0110] The background providing unit can estimate the user's emotion and adjust the method of providing the cultural information based on the estimated user's emotion. The method of providing the cultural information includes, for example, text information, video information, and the like. The background providing unit estimates the user's emotion and adjusts the method of providing the cultural information according to the emotion. For example, when the user is nervous, the background providing unit provides concise and easy-to-understand cultural information. Furthermore, when the user is relaxed, the background providing unit provides detailed cultural information. Furthermore, when the user is excited, the background providing unit provides interesting cultural information. This adjusts the method of providing the cultural information according to the user's emotion. The estimation of the emotion is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the background providing unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the background providing unit can input the user's emotion data into an AI model and adjust the method of providing the cultural information.

[0111] The background providing unit can select optimal information by referring to the user's past learning history when providing background information. The optimal information includes, for example, information based on past learning contents, information based on interests, and the like. The background providing unit selects optimal information by referring to the user's past learning history. For example, new information is provided based on cultural information that the user has learned in the past. In addition, cultural information that is likely to interest the user is selected from the user's past learning history. Furthermore, the user's past learning history is analyzed, and cultural information according to the progress of learning is provided. As a result, optimal information is selected based on the user's past learning history. A part or all of the above-mentioned processing in the background providing unit may be performed, for example, using AI, or may be performed without using AI. For example, the background providing unit can input the user's learning history data into an AI model to generate optimal information.

[0112] The background providing unit can provide information corresponding to different cultural backgrounds when providing background information. Different cultural backgrounds include, for example, Asian culture, European culture, and the like. The background providing unit adjusts the content of the information based on the cultural background of the user. For example, if the user is interested in a particular culture, information about that culture is provided. Also, if the user has a different cultural background, information based on that background is provided. Furthermore, information based on the cultural background of the language the user is learning is provided. This provides information corresponding to different cultural backgrounds. A part or all of the above-mentioned processing in the background providing unit may be performed, for example, using AI, or may be performed without using AI. For example, the background providing unit can input the user's cultural background data into an AI model to generate related information.

[0113] The background providing unit can estimate the user's emotion and determine the priority of the cultural information based on the estimated user's emotion. The priority of the cultural information includes, for example, importance and interest. The background providing unit estimates the user's emotion and determines the priority of the cultural information according to the emotion. For example, when the user is nervous, the background providing unit provides important cultural information with priority. Furthermore, when the user is relaxed, the background providing unit provides detailed cultural information with priority. Furthermore, when the user is excited, the background providing unit provides interesting cultural information with priority. In this way, the priority of the cultural information is determined according to the user's emotion. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the background providing unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the background providing unit can input the user's emotion data into an AI model to determine the priority of the cultural information.

[0114] The background providing unit can provide related information in consideration of the geographical location information of the user when providing background information. The related information includes, for example, information specific to a region, information related to a local culture, and the like. The background providing unit adjusts the content of the information in consideration of the geographical location information of the user. For example, when the user is in a specific region, the background providing unit provides cultural information of the region. Also, when the user is traveling, the background providing unit provides cultural information of the travel destination. Furthermore, when the user is in a specific country, the background providing unit provides cultural information of the country. As a result, related information is provided based on the geographical location information of the user. A part or all of the above-mentioned processing in the background providing unit may be performed, for example, using AI, or may be performed without using AI. For example, the background providing unit can input the geographical location data of the user into an AI model to generate related information.

[0115] The background providing unit can analyze the user's social media activity and provide related information when providing background information. The related information includes, for example, information related to recent posts and information based on interests. The background providing unit analyzes the user's social media activity and adjusts the content of the information. For example, the background providing unit provides cultural information based on the content shared by the user on social media. Also, the background providing unit provides cultural information based on the content posted by the accounts the user follows. Furthermore, the background providing unit provides cultural information based on groups and events in which the user participates. As a result, related information is provided based on the user's social media activity. A part or all of the above-mentioned processing in the background providing unit may be performed, for example, using AI, or may be performed without using AI. For example, the background providing unit can input the user's social media data into an AI model to generate related information.

[0116] The situation-specific learning unit can estimate the user's emotion and select a situation based on the estimated user's emotion. The selection of the situation includes, for example, selection based on emotion and selection based on learning progress. The situation-specific learning unit estimates the user's emotion and selects a situation according to the emotion. For example, when the user is nervous, the situation-specific learning unit selects a situation that allows the user to relax. When the user is relaxed, the situation-specific learning unit selects a normal situation. Furthermore, when the user is excited, the situation-specific learning unit selects an interesting situation. In this way, the situation is selected according to the user's emotion. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. Some or all of the above-mentioned processing in the situation-specific learning unit may be performed using, for example, AI, or may be performed without using AI. For example, the situation-specific learning unit can input the user's emotion data into an AI model and select a situation.

[0117] The situation-specific learning unit can select the optimal situation by referring to the user's past learning history during situation-specific learning. The optimal situation includes, for example, a situation based on past learning content, a situation based on interest, and the like. The situation-specific learning unit selects the optimal situation by referring to the user's past learning history. For example, a new situation is selected based on a situation that the user has learned in the past. In addition, a situation that is likely to interest the user is selected from the user's past learning history. Furthermore, the user's past learning history is analyzed, and a situation according to the progress of learning is selected. As a result, the optimal situation is selected based on the user's past learning history. A part or all of the above-mentioned processing in the situation-specific learning unit may be performed, for example, using AI, or may be performed without using AI. For example, the situation-specific learning unit can input the user's learning history data into an AI model to generate an optimal situation.

[0118] The situation-specific learning unit can estimate the user's emotion and adjust the practice order of the situations based on the estimated user's emotion. The practice order of the situations includes, for example, difficulty order, random order, and the like. The situation-specific learning unit estimates the user's emotion and adjusts the practice order of the situations according to the emotion. For example, when the user is nervous, the situation-specific learning unit starts practice from an easy situation. Also, when the user is relaxed, the situation-specific learning unit practices in the normal order. Furthermore, when the user is excited, the situation-specific learning unit starts practice from a difficult situation. In this way, the practice order of the situations is adjusted according to the user's emotion. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the situation-specific learning unit may be performed using, for example, AI, or may be performed without using AI. For example, the situation-specific learning unit can input the user's emotion data into an AI model and adjust the practice order of the situations.

[0119] The situation-specific learning unit can provide relevant situations in consideration of the user's geographical location information during situation-specific learning. The relevant situations include, for example, regional situations and situations related to local culture. The situation-specific learning unit adjusts the content of the situation in consideration of the user's geographical location information. For example, when the user is in a specific region, the situation-specific learning unit provides a situation related to the region. Also, when the user is traveling, the situation-specific learning unit provides a situation related to the travel destination. Furthermore, when the user is in a specific country, the situation related to the country is provided. As a result, relevant situations are provided based on the user's geographical location information. Part or all of the above-mentioned processing in the situation-specific learning unit may be performed, for example, using AI, or may be performed without using AI. For example, the situation-specific learning unit can input the user's geographical location data into an AI model to generate relevant situations.

[0120] The daily task learning unit can estimate the user's emotions and select a daily task based on the estimated user's emotions. The selection of daily tasks includes, for example, selection based on emotions and selection based on learning progress. The daily task learning unit estimates the user's emotions and selects a daily task according to the emotions. For example, if the user is nervous, the daily task learning unit selects an easy daily task. Furthermore, if the user is relaxed, the daily task learning unit selects a normal daily task. Furthermore, if the user is excited, the daily task learning unit selects a difficult daily task. In this way, a daily task is selected according to the user's emotions. The estimation of emotions is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the daily task learning unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the daily task learning unit can input the user's emotion data into an AI model and select a daily task.

[0121] The daily task learning unit can select an optimal task by referring to the user's past learning history during daily task learning. The optimal task includes, for example, a task based on past learning content, a task based on interests, and the like. The daily task learning unit selects an optimal task by referring to the user's past learning history. For example, a new task is selected based on a task the user has learned in the past. In addition, a task that is likely to interest the user is selected from the user's past learning history. Furthermore, the user's past learning history is analyzed, and a task according to the progress of learning is selected. In this way, an optimal task is selected based on the user's past learning history. A part or all of the above-mentioned processing in the daily task learning unit may be performed, for example, using AI, or may be performed without using AI. For example, the daily task learning unit can input the user's learning history data into an AI model to generate an optimal task.

[0122] The daily task learning unit can estimate the user's emotion and adjust the practice order of the daily tasks based on the estimated user's emotion. The practice order of the daily tasks includes, for example, order of difficulty and random order. The daily task learning unit estimates the user's emotion and adjusts the practice order of the daily tasks according to the emotion. For example, when the user is nervous, the daily task learning unit starts the practice from an easy task. Also, when the user is relaxed, the daily task learning unit practices in a normal order. Furthermore, when the user is excited, the daily task learning unit starts the practice from a task with a high level of difficulty. In this way, the practice order of the daily tasks is adjusted according to the user's emotion. The estimation of the emotion is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the daily task learning unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the daily task learning unit can input the user's emotion data into an AI model and adjust the practice order of the daily tasks.

[0123] The daily task learning unit can provide a related task in consideration of the geographical location information of the user during daily task learning. The related tasks include, for example, region-specific tasks and tasks related to local culture. The daily task learning unit adjusts the content of the task in consideration of the geographical location information of the user. For example, when the user is in a specific region, the daily task learning unit provides a task related to the region. Also, when the user is traveling, the daily task learning unit provides a task related to the travel destination. Furthermore, when the user is in a specific country, the daily task learning unit provides a task related to the country. As a result, a related task is provided based on the geographical location information of the user. Part or all of the above-mentioned processing in the daily task learning unit may be performed, for example, using AI, or may be performed without using AI. For example, the daily task learning unit can input the geographical location data of the user into an AI model to generate a related task.

[0124] The story providing unit can estimate the user's emotion and adjust the progress of the story based on the estimated emotion of the user. The progress of the story includes, for example, progress based on emotion and progress based on learning progress. The story providing unit estimates the user's emotion and adjusts the progress of the story according to the emotion. For example, when the user is nervous, the story providing unit selects a gentle progress method. Furthermore, when the user is relaxed, the story providing unit selects a normal progress method. Furthermore, when the user is excited, the story providing unit selects a stimulating progress method. In this way, the progress of the story is adjusted according to the emotion of the user. The estimation of the emotion is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the story providing unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the story providing unit can input the user's emotion data into an AI model and adjust the progress of the story.

[0125] The story providing unit can select an optimal story development by referring to the user's past selection history when providing a story. The optimal story development includes, for example, a development based on past selections and a development based on interests. The story providing unit selects an optimal story development by referring to the user's past selection history. For example, a new development is selected based on a story development selected by the user in the past. Also, a story development that is likely to interest the user is selected from the user's past selection history. Furthermore, the user's past selection history is analyzed, and a story development according to the progress of learning is selected. As a result, an optimal story development is selected based on the user's past selection history. A part or all of the above-mentioned processing in the story providing unit may be performed, for example, using AI, or may be performed without using AI. For example, the story providing unit can input the user's selection history data into an AI model to generate an optimal story development.

[0126] The story providing unit can estimate the user's emotion and adjust the story options based on the estimated user's emotion. The story options include, for example, options based on emotion and options based on learning progress. The story providing unit estimates the user's emotion and adjusts the story options according to the emotion. For example, if the user is nervous, the story providing unit provides an easy option. Furthermore, if the user is relaxed, the story providing unit provides a normal option. Furthermore, if the user is excited, the story providing unit provides an option with a high level of difficulty. In this way, the story options are adjusted according to the user's emotion. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the story providing unit may be performed, for example, using an AI, or may be performed without using an AI. For example, the story providing unit can input the user's emotion data into an AI model and adjust the story options.

[0127] The story providing unit can provide related stories in consideration of the geographical location information of the user when providing a story. The related stories include, for example, stories specific to a region and stories related to a local culture. The story providing unit adjusts the content of the story in consideration of the geographical location information of the user. For example, when the user is in a specific region, the story providing unit provides a story related to the region. Also, when the user is traveling, the story providing unit provides a story related to the travel destination. Furthermore, when the user is in a specific country, the story providing unit provides a story related to the country. As a result, related stories are provided based on the geographical location information of the user. A part or all of the above-mentioned processing in the story providing unit may be performed, for example, using AI, or may be performed without using AI. For example, the story providing unit can input the geographical location data of the user into an AI model to generate related stories.

[0128] The progress tracking unit can estimate the user's emotion and adjust the progress tracking method based on the estimated user's emotion. The progress tracking method includes, for example, a concise tracking method, a detailed tracking method, a visually stimulating tracking method, and the like. The progress tracking unit estimates the user's emotion and adjusts the progress tracking method according to the emotion. For example, when the user is nervous, the progress tracking unit provides a concise tracking method. Furthermore, when the user is relaxed, the progress tracking unit provides a detailed tracking method. Furthermore, when the user is excited, the progress tracking unit provides a visually stimulating tracking method. This adjusts the progress tracking method according to the user's emotion. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the progress tracking unit may be performed, for example, using an AI, or may be performed without using an AI. For example, the progress tracking unit can input user emotional data into an AI model to adjust how progress is tracked.

[0129] The progress tracking unit can select an optimal tracking method by referring to the user's past learning history when tracking the progress. The optimal tracking method includes, for example, a tracking method based on past learning contents, a tracking method based on interests, and the like. The progress tracking unit selects an optimal tracking method by referring to the user's past learning history. For example, a new tracking method is selected based on the tracking method used by the user in the past. Also, a tracking method that is likely to interest the user is selected from the user's past learning history. Furthermore, the user's past learning history is analyzed, and a tracking method according to the progress of learning is selected. As a result, the optimal tracking method is selected based on the user's past learning history. A part or all of the above-mentioned processing in the progress tracking unit may be performed, for example, using AI, or may be performed without using AI. For example, the progress tracking unit can input the user's learning history data into an AI model to generate an optimal tracking method.

[0130] The progress tracking unit can estimate the user's emotion and adjust the display method of the progress based on the estimated user's emotion. The display method of the progress includes, for example, a simple and highly visible display method, a display method including detailed information, and a visually stimulating display method. The progress tracking unit estimates the user's emotion and adjusts the display method of the progress according to the emotion. For example, when the user is nervous, the progress tracking unit provides a simple and highly visible display method. Furthermore, when the user is relaxed, the progress tracking unit provides a display method including detailed information. Furthermore, when the user is excited, the progress tracking unit provides a visually stimulating display method. This adjusts the display method of the progress according to the user's emotion. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the progress tracking unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the progress tracker can input user emotional data into the AI ​​model to adjust how progress is displayed.

[0131] The progress tracking unit can provide relevant tracking information in consideration of the geographical location information of the user during progress tracking. The relevant tracking information includes, for example, information specific to a region, information related to a local culture, and the like. The progress tracking unit adjusts the content of the tracking information in consideration of the geographical location information of the user. For example, when the user is in a specific region, the progress tracking unit provides tracking information related to the region. Also, when the user is traveling, the progress tracking unit provides tracking information related to the travel destination. Furthermore, when the user is in a specific country, the progress tracking unit provides tracking information related to the country. This provides relevant tracking information based on the geographical location information of the user. Some or all of the above-mentioned processing in the progress tracking unit may be performed, for example, using AI, or may be performed without using AI. For example, the progress tracking unit can input the geographical location data of the user into an AI model to generate relevant tracking information.

[0132] The feedback providing unit can estimate the user's emotion and adjust the feedback expression method based on the estimated user's emotion. The feedback expression method includes, for example, a gentle expression, a strict expression, and the like. The feedback providing unit estimates the user's emotion and adjusts the feedback expression method according to the emotion. For example, if the user is nervous, the feedback providing unit provides feedback in gentle words. Furthermore, if the user is relaxed, the feedback providing unit provides detailed feedback. Furthermore, if the user is in a hurry, the feedback providing unit provides concise feedback. This adjusts the feedback expression method according to the user's emotion. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the feedback providing unit may be performed, for example, using an AI, or may be performed without using an AI. For example, the feedback providing unit can input the user's emotion data into an AI model and adjust the feedback expression method.

[0133] The feedback providing unit can select optimal feedback by referring to the user's past learning history when providing feedback. The optimal feedback includes, for example, feedback based on past learning contents, feedback based on interests, and the like. The feedback providing unit selects optimal feedback by referring to the user's past learning history. For example, new feedback is provided based on feedback the user has received in the past. Also, feedback that is likely to interest the user is selected from the user's past learning history. Furthermore, the user's past learning history is analyzed, and feedback according to the progress of learning is provided. As a result, optimal feedback is selected based on the user's past learning history. A part or all of the above-mentioned processing in the feedback providing unit may be performed, for example, using AI, or may be performed without using AI. For example, the feedback providing unit can input the user's learning history data into an AI model to generate optimal feedback.

[0134] The feedback providing unit can estimate the user's emotion and determine the priority of the feedback based on the estimated user's emotion. The priority of the feedback includes, for example, important improvements, detailed improvements, and the most important improvements. The feedback providing unit estimates the user's emotion and determines the priority of the feedback according to the emotion. For example, when the user is nervous, the feedback providing unit gives priority to important improvements and provides feedback. Also, when the user is relaxed, the feedback providing unit gives priority to detailed improvements. Furthermore, when the user is in a hurry, the feedback providing unit gives priority to the most important improvements and provides feedback. In this way, the priority of the feedback is determined according to the user's emotion. The estimation of the emotion is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the feedback providing unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the feedback providing unit can input the user's emotion data into an AI model and determine the priority of the feedback.

[0135] The feedback providing unit can provide relevant feedback in consideration of the geographical location information of the user when providing feedback. The relevant feedback includes, for example, region-specific feedback, feedback related to local culture, and the like. The feedback providing unit adjusts the content of the feedback in consideration of the geographical location information of the user. For example, when the user is in a specific region, the feedback providing unit provides feedback related to the region. Also, when the user is traveling, the feedback providing unit provides feedback related to the travel destination. Furthermore, when the user is in a specific country, the feedback providing unit provides feedback related to the country. This provides relevant feedback based on the geographical location information of the user. Some or all of the above-mentioned processing in the feedback providing unit may be performed, for example, using AI, or may be performed without using AI. For example, the feedback providing unit can input the geographical location data of the user into an AI model to generate relevant feedback.

[0136] The system according to the embodiment is not limited to the above-described example, and various modifications are possible, for example, as follows.

[0137] The dialogue unit can analyze the dialogue history of the user, identify pronunciations and grammar mistakes made by the user in the past, and generate a dialogue scenario that focuses on practicing those parts. For example, if the user has made mistakes in pronouncing "r" and "l" in the past, the dialogue unit provides a dialogue scenario including that pronunciation. Also, if the user has made mistakes in a specific grammatical structure in the past, the dialogue unit provides a dialogue scenario including that grammatical structure. Furthermore, if the user has made mistakes in a specific vocabulary in the past, the dialogue unit provides a dialogue scenario including that vocabulary. This enables effective practice based on the user's past mistakes.

[0138] The situation-specific learning unit can customize the method of practicing situations according to the user's learning style. For example, visual learners can be provided with situation practice using images and videos. Also, auditory learners can be provided with situation practice centered on audio. Furthermore, tactile learners can be provided with interactive situation practice. In this way, the most suitable situation practice according to the user's learning style can be provided.

[0139] The daily task learning unit can provide region-specific daily tasks by taking into account the geographic location information of the user. For example, if the user is in Japan, daily tasks based on Japanese culture and customs are provided. If the user is in France, daily tasks based on French culture and customs are provided. If the user is in America, daily tasks based on American culture and customs are provided. This provides related daily tasks based on the geographic location information of the user.

[0140] The story providing unit can analyze the user's social media activity and provide stories that the user may be interested in. For example, if the user posts a lot about travel on social media, stories related to travel are provided. If the user posts a lot about cooking, stories related to cooking are provided. If the user posts a lot about sports, stories related to sports are provided. In this way, relevant stories are provided based on the user's social media activity.

[0141] The dialogue unit can adjust the difficulty of the dialogue according to the user's learning progress. For example, if the user is at a beginner's level, an easy dialogue scenario is provided. If the user is at an intermediate level, a somewhat difficult dialogue scenario is provided. If the user is at an advanced level, a more difficult dialogue scenario is provided. In this way, an optimal dialogue according to the user's learning progress is provided.

[0142] The progress tracking unit can estimate the user's emotion and adjust the progress tracking method based on the estimated user's emotion. For example, if the user is nervous, the progress tracking unit provides a simple and highly visible tracking method. If the user is relaxed, the progress tracking unit provides a tracking method including detailed information. If the user is excited, the progress tracking unit provides a visually stimulating tracking method. In this way, the progress tracking method is adjusted according to the user's emotion.

[0143] The feedback providing unit can estimate the user's emotion and adjust the manner of expressing the feedback based on the estimated user's emotion. For example, if the user is nervous, the feedback providing unit provides feedback in gentle words. If the user is relaxed, the feedback providing unit provides detailed feedback. If the user is in a hurry, the feedback providing unit provides concise feedback. In this way, the manner of expressing the feedback is adjusted according to the user's emotion.

[0144] The quiz section can estimate the user's emotions and adjust the difficulty of the quiz based on the estimated user's emotions. For example, if the user is nervous, the quiz section poses easy questions. If the user is relaxed, the quiz section poses questions of normal difficulty. If the user is excited, the quiz section poses difficult questions. In this way, the difficulty of the quiz is adjusted according to the user's emotions.

[0145] The situation-specific learning unit can estimate the user's emotions and select a situation based on the estimated user's emotions. For example, if the user is nervous, the situation-specific learning unit selects a situation that allows the user to relax. If the user is relaxed, the situation-specific learning unit selects a normal situation. If the user is excited, the situation-specific learning unit selects an interesting situation. In this way, a situation is selected according to the user's emotions.

[0146] The story providing unit can estimate the user's emotions and adjust the way the story progresses based on the estimated user's emotions. For example, if the user is nervous, the story providing unit selects a gentle way of progressing. If the user is relaxed, the story providing unit selects a normal way of progressing. If the user is excited, the story providing unit selects a stimulating way of progressing. In this way, the way the story progresses is adjusted according to the user's emotions.

[0147] The process flow of the second embodiment will be briefly described below.

[0148] Step 1: The dialogue unit dialogues with the user in a language selected by the user. The language selected by the user includes, but is not limited to, English, French, Chinese, etc. The dialogue unit can dialogue in the form of, for example, voice dialogue, text dialogue, etc. Step 2: The pronunciation analysis unit analyzes the pronunciation of the user's dialogue with the dialogue unit in real time and provides specific improvements. The real-time analysis is performed within a delay tolerance of, for example, 1 second or 5 seconds. For example, when the user pronounces "hello," the pronunciation analysis unit evaluates the accuracy of the pronunciation and provides improvements on intonation. Step 3: The quiz section provides interactive quizzes on grammar and vocabulary according to the user's language learning progress. Quiz formats include, for example, multiple choice questions, fill-in-the-blank questions, etc. For example, if the user is at a beginner's level, the quiz section provides quizzes on basic grammar and vocabulary to advance the user's learning. Step 4: The background provider provides information about the culture and customs of the language selected by the user. The provided information includes, for example, food culture, holidays, social customs, etc. The background provider introduces cultural background information naturally during the dialogue, for example.

[0149] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires a voice indicating a user input for the result of the specific processing. The control unit 46A transmits the voice data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0150] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image (e.g., still image data or video data) are input. The data generation model 58 infers the input inference data according to the instruction indicated by the prompt, and outputs the inference result in one or more data formats of voice data, text data, and image data. The data generation model 58 includes, for example, a text generation AI, an image generation AI, and a multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specification processing unit 290 performs the above-mentioned specification processing while using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction, and in this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. In the data processing device 12 and the like, the data generation model 58 includes a plurality of types, and the data generation model 58 includes an AI other than the generation AI. The AI ​​other than the generation AI is, for example, a linear regression, a logistic regression, a decision tree, a random forest, a support vector machine (SVM), a k-means clustering, a convolutional neural network (CNN), a recurrent neural network (RNN), a generative adversarial network (GAN), or a naive Bayes, and can perform various processes, but is not limited to such examples. The AI ​​may also be an AI agent. In addition, when the processes of each of the above-mentioned parts are performed by an AI, the processes are performed in part or in whole by the AI, but are not limited to such examples.In addition, processing performed by AI including the generation AI may be replaced with rule-based processing, and rule-based processing may be replaced with processing performed by AI including the generation AI.

[0151] Furthermore, the above-described processing by the data processing system 10 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart device 14, but may also be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart device 14. Furthermore, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the smart device 14 or an external device, and the smart device 14 acquires or collects information required for processing from the data processing device 12 or an external device.

[0152] For example, the dialogue unit is realized by the control unit 46A of the smart device 14 or the specific processing unit 290 of the data processing device 12. The pronunciation analysis unit is realized by, for example, the control unit 46A of the smart device 14, and analyzes the user's pronunciation in real time. The quiz unit is realized by, for example, the specific processing unit 290 of the data processing device 12, and provides a quiz according to the user's progress. The background providing unit is realized by, for example, the control unit 46A of the smart device 14 or the specific processing unit 290 of the data processing device 12, and provides information on culture and customs. The correspondence between each unit and the device or control unit is not limited to the above-mentioned example, and various changes are possible.

[0153] [Second embodiment] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0154] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0155] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN and / or a LAN.

[0156] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0157] The microphone 238 receives instructions and the like from the user by receiving voice uttered by the user. The microphone 238 captures the voice uttered by the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs the voice according to instructions from the processor 46.

[0158] Camera 42 is a small digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the user's surroundings (e.g., an imaging range defined by an angle of view equivalent to the width of the field of vision of an average healthy person).

[0159] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is performed in a secure state.

[0160] Fig. 4 shows an example of main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0161] The processor 28 reads the specific processing program 56 from the storage 32, and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0162] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the emotion of the user using the emotion identification model 59 and perform identification processing using the user's emotion. In the emotion estimation function (emotion identification function) using the emotion identification model 59, various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, are performed, but are not limited to such examples. In addition, the estimation and prediction of the emotion also include, for example, analysis of the emotion.

[0163] In the smart glasses 214, the specific processing is performed by the processor 46. The storage 50 stores a specific processing program 60. The processor 46 reads the specific processing program 60 from the storage 50 and executes the read specific processing program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as the control unit 46A in accordance with the specific processing program 60 executed on the RAM 48. Note that the smart glasses 214 have a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can also perform processing similar to that of the specific processing unit 290 using these models.

[0164] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 obtains a processing result (such as a prediction result) using the data generation model 58 by communicating with the server device having the data generation model 58. In addition, the data processing device 12 may be a server device, or may be a terminal device (such as a mobile phone, a robot, or a home appliance) owned by a user.

[0165] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires a voice indicating a user input for the result of the specific processing. The control unit 46A transmits the voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0166] The data generation model 58 is a so-called generative AI. An example of the data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image (e.g., still image data or video data) is input. The data generation model 58 infers the input inference data according to the instruction indicated by the prompt, and outputs the inference result in one or more data formats of voice data, text data, and image data. The data generation model 58 includes, for example, a text generation AI, an image generation AI, and a multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specification processing unit 290 performs the above-mentioned specification processing while using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction, and in this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. In the data processing device 12 and the like, the data generation model 58 includes a plurality of types, and the data generation model 58 includes an AI other than the generation AI. The AI ​​other than the generation AI is, for example, a linear regression, a logistic regression, a decision tree, a random forest, a support vector machine (SVM), a k-means clustering, a convolutional neural network (CNN), a recurrent neural network (RNN), a generative adversarial network (GAN), or a naive Bayes, and can perform various processes, but is not limited to such examples. The AI ​​may be an AI agent. In addition, when the processes of each part described above are performed by an AI, the processes are performed in part or in whole by the AI, but are not limited to such examples. In addition, the processes performed by the AI ​​including the generation AI may be replaced with rule-based processes, and the rule-based processes may be replaced with processes performed by the AI ​​including the generation AI.

[0167] The data processing system 210 according to the second embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 210 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart glasses 214, but may be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart glasses 214. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the smart glasses 214 or an external device, and the smart glasses 214 acquires or collects information required for processing from the data processing device 12 or an external device.

[0168] For example, the dialogue unit is realized by the control unit 46A of the smart glasses 214 or the specific processing unit 290 of the data processing device 12. The pronunciation analysis unit is realized by, for example, the control unit 46A of the smart glasses 214, and analyzes the user's pronunciation in real time. The quiz unit is realized by, for example, the specific processing unit 290 of the data processing device 12, and provides a quiz according to the user's progress. The background providing unit is realized by, for example, the control unit 46A of the smart glasses 214 or the specific processing unit 290 of the data processing device 12, and provides information on culture and customs. The correspondence between each unit and the device or control unit is not limited to the above-mentioned example, and various changes are possible.

[0169] [Third embodiment] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0170] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0171] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN and / or a LAN.

[0172] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0173] The microphone 238 receives instructions and the like from the user by receiving voice uttered by the user. The microphone 238 captures the voice uttered by the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs the voice according to instructions from the processor 46.

[0174] Camera 42 is a small digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the user's surroundings (e.g., an imaging range defined by an angle of view equivalent to the width of the field of vision of an average healthy person).

[0175] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is performed in a secure state.

[0176] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0177] The processor 28 reads the specific processing program 56 from the storage 32, and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0178] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the emotion of the user using the emotion identification model 59 and perform identification processing using the user's emotion. In the emotion estimation function (emotion identification function) using the emotion identification model 59, various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, are performed, but are not limited to such examples. In addition, the estimation and prediction of the emotion also include, for example, analysis of the emotion.

[0179] In the headset type terminal 314, the specific processing is performed by the processor 46. The storage 50 stores a specific program 60. The processor 46 reads the specific program 60 from the storage 50, and executes the read specific program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as the control unit 46A in accordance with the specific program 60 executed on the RAM 48. Note that the headset type terminal 314 has a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can also perform processing similar to that of the identification processing unit 290 using these models.

[0180] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 obtains a processing result (such as a prediction result) using the data generation model 58 by communicating with the server device having the data generation model 58. In addition, the data processing device 12 may be a server device, or may be a terminal device (such as a mobile phone, a robot, or a home appliance) owned by a user.

[0181] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input for the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0182] The data generation model 58 is a so-called generative AI. An example of the data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image (e.g., still image data or video data) is input. The data generation model 58 infers the input inference data according to the instruction indicated by the prompt, and outputs the inference result in one or more data formats of voice data, text data, and image data. The data generation model 58 includes, for example, a text generation AI, an image generation AI, and a multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specification processing unit 290 performs the above-mentioned specification processing while using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction, and in this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. In the data processing device 12 and the like, the data generation model 58 includes a plurality of types, and the data generation model 58 includes an AI other than the generation AI. The AI ​​other than the generation AI is, for example, a linear regression, a logistic regression, a decision tree, a random forest, a support vector machine (SVM), a k-means clustering, a convolutional neural network (CNN), a recurrent neural network (RNN), a generative adversarial network (GAN), or a naive Bayes, and can perform various processes, but is not limited to such examples. The AI ​​may be an AI agent. In addition, when the processes of each part described above are performed by an AI, the processes are performed in part or in whole by the AI, but are not limited to such examples. In addition, the processes performed by the AI ​​including the generation AI may be replaced with rule-based processes, and the rule-based processes may be replaced with processes performed by the AI ​​including the generation AI.

[0183] The data processing system 310 according to the third embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 310 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the headset type terminal 314, but may be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the headset type terminal 314. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the headset type terminal 314 or an external device, and the headset type terminal 314 acquires or collects information required for processing from the data processing device 12 or an external device.

[0184] For example, the dialogue unit is realized by the control unit 46A of the headset type terminal 314 or the specific processing unit 290 of the data processing device 12. The pronunciation analysis unit is realized by, for example, the control unit 46A of the headset type terminal 314, and analyzes the user's pronunciation in real time. The quiz unit is realized by, for example, the specific processing unit 290 of the data processing device 12, and provides a quiz according to the user's progress. The background providing unit is realized by, for example, the control unit 46A of the headset type terminal 314 or the specific processing unit 290 of the data processing device 12, and provides information on culture and customs. The correspondence between each unit and the device or control unit is not limited to the above-mentioned example, and various changes are possible.

[0185] [Fourth embodiment] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[0186] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0187] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN and / or a LAN.

[0188] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[0189] The microphone 238 receives instructions and the like from the user by receiving voice uttered by the user. The microphone 238 captures the voice uttered by the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs the voice according to instructions from the processor 46.

[0190] Camera 42 is a small digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a CMOS image sensor or a CCD image sensor, and captures images of the user's surroundings (e.g., an imaging range defined by an angle of view equivalent to the width of the field of vision of a typical healthy person).

[0191] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is performed in a secure state.

[0192] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and legs. The posture and behavior of the robot 414 are controlled by controlling the motors of the arms, hands, and legs. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[0193] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0194] The processor 28 reads the specific processing program 56 from the storage 32, and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0195] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the emotion of the user using the emotion identification model 59 and perform identification processing using the user's emotion. In the emotion estimation function (emotion identification function) using the emotion identification model 59, various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, are performed, but are not limited to such examples. In addition, the estimation and prediction of the emotion also include, for example, analysis of the emotion.

[0196] In the robot 414, the specific processing is performed by the processor 46. The storage 50 stores a specific program 60. The processor 46 reads out the specific program 60 from the storage 50, and executes the specific program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as the control unit 46A in accordance with the specific program 60 executed on the RAM 48. The robot 414 has a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can also perform processing similar to that of the specific processing unit 290 using these models.

[0197] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 obtains a processing result (such as a prediction result) using the data generation model 58 by communicating with the server device having the data generation model 58. In addition, the data processing device 12 may be a server device, or may be a terminal device (such as a mobile phone, a robot, or a home appliance) owned by a user.

[0198] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires a voice indicating a user input for the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0199] The data generation model 58 is a so-called generative AI. An example of the data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image (e.g., still image data or video data) is input. The data generation model 58 infers the input inference data according to the instruction indicated by the prompt, and outputs the inference result in one or more data formats of voice data, text data, and image data. The data generation model 58 includes, for example, a text generation AI, an image generation AI, and a multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specification processing unit 290 performs the above-mentioned specification processing while using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction, and in this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. In the data processing device 12 and the like, the data generation model 58 includes a plurality of types, and the data generation model 58 includes an AI other than the generation AI. The AI ​​other than the generation AI is, for example, a linear regression, a logistic regression, a decision tree, a random forest, a support vector machine (SVM), a k-means clustering, a convolutional neural network (CNN), a recurrent neural network (RNN), a generative adversarial network (GAN), or a naive Bayes, and can perform various processes, but is not limited to such examples. The AI ​​may be an AI agent. In addition, when the processes of each part described above are performed by an AI, the processes are performed in part or in whole by the AI, but are not limited to such examples. In addition, the processes performed by the AI ​​including the generation AI may be replaced with rule-based processes, and the rule-based processes may be replaced with processes performed by the AI ​​including the generation AI.

[0200] The data processing system 410 according to the fourth embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 410 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the robot 414, but may be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the robot 414. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the robot 414 or an external device, etc., and the robot 414 acquires or collects information required for processing from the data processing device 12 or an external device, etc.

[0201] For example, the dialogue unit is realized by the control unit 46A of the robot 414 or the specific processing unit 290 of the data processing device 12. The pronunciation analysis unit is realized by, for example, the control unit 46A of the robot 414, and analyzes the user's pronunciation in real time. The quiz unit is realized by, for example, the specific processing unit 290 of the data processing device 12, and provides a quiz according to the user's progress. The background providing unit is realized by, for example, the control unit 46A of the robot 414 or the specific processing unit 290 of the data processing device 12, and provides information on culture and customs. The correspondence between each unit and the device or control unit is not limited to the above-mentioned example, and various changes are possible.

[0202] The emotion identification model 59 as an emotion engine may determine the emotion of the user according to a specific mapping. Specifically, the emotion identification model 59 may determine the emotion of the user according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the emotion of the robot, and the identification processing unit 290 may perform identification processing using the emotion of the robot.

[0203] FIG. 9 is a diagram showing an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive emotions are arranged. The more outside the concentric circles, the more emotions that represent states and actions that arise from a state of mind are arranged. Emotions are a concept that includes emotions and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions that occur in the brain are arranged. On the right side of the concentric circles, emotions that are generally induced by situational judgment are arranged. On the upper and lower sides of the concentric circles, emotions that are generally generated from reactions that occur in the brain and are induced by situational judgment are arranged. In addition, on the upper side of the concentric circles, emotions of "pleasure" are arranged, and on the lower side, emotions of "discomfort" are arranged. In this way, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[0204] These emotions are distributed in the three o'clock direction of emotion map 400 and usually fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[0205] The inside of emotion map 400 represents what is going on inside one's mind, and the outside of emotion map 400 represents behavior, so the further out you go on emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[0206] Here, human emotions are based on various balances such as posture and blood sugar level, and when these balances are far from the ideal, it indicates an unpleasant state, and when they are close to the ideal, it indicates a pleasant state. Emotions can also be created for robots, cars, motorcycles, etc., based on various balances such as posture and battery level, so that when these balances are far from the ideal, it indicates an unpleasant state, and when they are close to the ideal, it indicates a pleasant state. The emotion map may be generated, for example, based on the emotion map of Dr. Mitsuyoshi (Research on speech emotion recognition and emotion brain physiological signal analysis system, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). On the left half of the emotion map, emotions belonging to an area called "reaction" where sensation is dominant are lined up. On the right half of the emotion map, emotions belonging to an area called "situation" where situation recognition is dominant are lined up.

[0207] The emotion map defines two emotions that promote learning. The first is the negative emotion around the middle of "repentance" or "remorse" on the situation side. In other words, this is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the positive emotion around "desire" on the response side. In other words, this is when the robot has positive feelings such as "I want more" or "I want to know more."

[0208] The emotion identification model 59 inputs the user input to a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the emotion of the user. This neural network is pre-trained based on multiple learning data that are combinations of the user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in Fig. 10. Fig. 10 shows an example in which multiple emotions, "relief," "calm," and "encouraging," have similar emotion values.

[0209] In the above embodiment, an example was given in which a specific process is performed by one computer 22, but the technology disclosed herein is not limited to this, and distributed processing of the specific process may be performed by multiple computers, including computer 22.

[0210] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable computer-readable non-transitory storage medium such as a Universal Serial Bus (USB) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[0211] In addition, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 upon request from the data processing device 12.

[0212] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[0213] As the hardware resource for executing the specific process, various processors as shown below can be used. An example of the processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing the specific process by executing software, i.e., a program. Another example of the processor is a dedicated electric circuit, which is a processor having a circuit configuration designed exclusively for executing a specific process, such as a Field-Programmable Gate Array (FPGA), a Programmable Logic Device (PLD), or an Application Specific Integrated Circuit (ASIC). Each processor has a built-in or connected memory, and each processor executes the specific process by using the memory.

[0214] The hardware resource that executes the specific process may be one of these various processors, or may be a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific process may be a single processor.

[0215] As an example of a configuration using one processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a configuration using a processor that realizes the functions of the entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[0216] Furthermore, more specifically, the hardware structure of these various processors can be an electric circuit that combines circuit elements such as semiconductor elements. The above specific processes are merely examples. It goes without saying that unnecessary steps may be deleted, new steps may be added, or the order of processes may be changed without departing from the spirit of the invention.

[0217] In the above example, the first to fourth embodiments are described separately, but some or all of these embodiments may be combined. The smart device 14, the smart glasses 214, the headset terminal 314, and the robot 414 are only examples, and they may be combined with each other, or may be other devices. In the above example, the first and second embodiments are described separately, but they may be combined.

[0218] The above description and illustrations are detailed descriptions of the parts related to the technology of the present disclosure, and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, function, action, and effect is an example of the configuration, function, action, and effect of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replaced with respect to the description and illustrations shown above, within the scope of the gist of the technology of the present disclosure. In addition, in order to avoid confusion and to facilitate understanding of the parts related to the technology of the present disclosure, the description and illustrations shown above omit explanations of technical common sense that do not require particular explanation in enabling the implementation of the technology of the present disclosure.

[0219] All publications, patent applications, and standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, and standard was specifically and individually indicated to be incorporated by reference.

[0220] (Appendix 1) A dialogue unit that dialogues with the user in a language selected by the user; a pronunciation analysis unit that analyzes the pronunciation of the user's dialogue with the dialogue unit in real time and provides specific improvements; a quiz section that provides interactive quizzes on grammar and vocabulary according to a progress of the language learning of the user through the dialogue section; a background providing unit that provides information about the culture and customs of the language selected by the user through the dialogue unit. A system characterized by: (Appendix 2) A situation-specific learning section is provided for carrying out language practice based on specific situations through the dialogue section. 2. The system of claim 1, (Appendix 3) A daily task learning unit that allows the user to practice specific daily tasks in the language selected by the user through the dialogue unit. 2. The system of claim 1, (Appendix 4) A story providing unit provides specific interactive content in which the user is the main character and progresses the story using language. 2. The system of claim 1, (Appendix 5) The dialogue unit includes: Combine voice, text, and video to engage in specific interactions with said users 2. The system of claim 1, (Appendix 6) A progress tracking unit that tracks the user's learning progress and ability and recommends specific lessons and exercises. 2. The system of claim 1, (Appendix 7) A feedback providing unit that provides specific feedback based on the user's learning progress and ability. 7. The system of claim 6, (Appendix 8) The pronunciation analysis unit is Displaying the results of the analysis of the user's pronunciation in detail 2. The system of claim 1, (Appendix 9) The story providing unit is Providing an interactive story with a specific ending based on the user's selection. 5. The system of claim 4. (Appendix 10) The pronunciation analysis unit is Suggesting improvements based on a comparison of the user's pronunciation with specific native speakers 2. The system of claim 1, (Appendix 11) The dialogue unit includes: Infer user emotions and adjust the tone and pace of dialogue based on the inferred user emotions 2. The system of claim 1, (Appendix 12) The dialogue unit includes: Analyze the user's past dialogue history and select the optimal dialogue scenario 2. The system of claim 1, (Appendix 13) The dialogue unit includes: Providing additional information based on the user's interests and concerns in response to the content of the interaction 2. The system of claim 1, (Appendix 14) The dialogue unit includes: Estimate the user's emotions and select a conversation topic based on the estimated user's emotions. 2. The system of claim 1, (Appendix 15) The dialogue unit includes: During a conversation, the app takes into account the user's geographic location to provide relevant topics 2. The system of claim 1, (Appendix 16) The dialogue unit includes: Analyzes the user's social media activity and provides relevant topics during interactions 2. The system of claim 1, (Appendix 17) The pronunciation analysis unit is Estimating a user's emotion and adjusting the presentation of the pronunciation feedback based on the estimated user's emotion 2. The system of claim 1, (Appendix 18) The pronunciation analysis unit is When analyzing pronunciation, the system refers to the user's past pronunciation data and suggests areas for improvement. 2. The system of claim 1, (Appendix 19) The pronunciation analysis unit is Pronunciation analysis is tailored to different accents and dialects 2. The system of claim 1, (Appendix 20) The pronunciation analysis unit is Estimating a user's emotion and prioritizing pronunciation feedback based on the estimated user's emotion 2. The system of claim 1, (Appendix 21) The pronunciation analysis unit is During pronunciation analysis, the geographic location of the user is taken into account to provide relevant pronunciation feedback 2. The system of claim 1, (Appendix 22) The pronunciation analysis unit is During pronunciation analysis, the social media activity of the user is analyzed and relevant pronunciation feedback is provided. 2. The system of claim 1, (Appendix 23) The quiz section includes: Inferring a user's emotions and adjusting the difficulty of the quiz based on the estimated user's emotions 2. The system of claim 1, (Appendix 24) The quiz section includes: When providing a quiz, select the most suitable questions by referring to the user's past quiz results. 2. The system of claim 1, (Appendix 25) The quiz section includes: When providing quizzes, provide quiz formats that cater to different learning styles 2. The system of claim 1, (Appendix 26) The quiz section includes: The user's emotions are estimated, and the order of questions in the quiz is adjusted based on the estimated user's emotions. 2. The system of claim 1, (Appendix 27) The quiz section includes: When providing a quiz, provide relevant questions by considering the user's geographic location information 2. The system of claim 1, (Appendix 28) The quiz section includes: When providing a quiz, analyze the user's social media activity to provide relevant questions 2. The system of claim 1, (Appendix 29) The background providing unit is To estimate a user's emotion and adjust a method of providing cultural information based on the estimated user's emotion. 2. The system of claim 1, (Appendix 30) The background providing unit is When providing background information, the most appropriate information is selected by referring to the user's past learning history. 2. The system of claim 1, (Appendix 31) The background providing unit is When providing background information, provide information that is relevant to different cultural backgrounds 2. The system of claim 1, (Appendix 32) The background providing unit is Estimate user's emotions and prioritize cultural information based on the estimated user's emotions. 2. The system of claim 1, (Appendix 33) The background providing unit is When providing contextual information, the app takes into account the user's geographic location to provide relevant information. 2. The system of claim 1, (Appendix 34) The background providing unit is Analyze your social media activity and provide relevant information when you provide context 2. The system of claim 1, (Appendix 35) The situation-specific learning unit includes: Estimate the user's emotions and select a situation based on the estimated user's emotions. 3. The system of claim 2, (Appendix 36) The situation-specific learning unit includes: When learning by situation, the most appropriate situation is selected by referring to the user's past learning history. 3. The system of claim 2, (Appendix 37) The situation-specific learning unit includes: Estimate the user's emotions and adjust the order of situation practice based on the estimated user's emotions. 3. The system of claim 2, (Appendix 38) The situation-specific learning unit includes: When learning in a situation, the system provides relevant situations by taking into account the user's geographic location. 3. The system of claim 2, (Appendix 39) The daily task learning unit includes: Estimate the user's emotions and select daily tasks based on the estimated user emotions. 4. The system of claim 3. (Appendix 40) The daily task learning unit includes: When learning daily tasks, the system selects the most appropriate tasks by referring to the user's past learning history. 4. The system of claim 3. (Appendix 41) The daily task learning unit includes: Estimating a user's emotions and adjusting the order of daily task practice based on the estimated user's emotions 4. The system of claim 3. (Appendix 42) The daily task learning unit includes: When learning daily tasks, provide relevant tasks considering the user's geographic location 4. The system of claim 3. (Appendix 43) The story providing unit is Infer the user's emotions and adjust the story progression based on the inferred user emotions 5. The system of claim 4. (Appendix 44) The story providing unit is When providing a story, the system selects the optimal story development by referring to the user's past selection history. 5. The system of claim 4. (Appendix 45) The story providing unit is Infer user emotions and adjust story choices based on the inferred user emotions 5. The system of claim 4. (Appendix 46) The story providing unit is When serving stories, the app takes into account the user's geographic location to provide relevant stories. 5. The system of claim 4. (Appendix 47) The progress tracking unit is Inferring a user's emotion and adjusting a progress tracking method based on the inferred user emotion 7. The system of claim 6, (Appendix 48) The progress tracking unit is When tracking progress, select the most appropriate tracking method by referring to the user's past learning history 7. The system of claim 6, (Appendix 49) The progress tracking unit is Inferring a user's emotions and adjusting the way the progress is displayed based on the estimated user's emotions 7. The system of claim 6, (Appendix 50) The progress tracking unit is When tracking progress, take into account the user's geographic location to provide relevant tracking information 7. The system of claim 6, (Appendix 51) The feedback providing unit, Infer user emotions and adjust feedback presentation based on the estimated user emotions 8. The system of claim 7, (Appendix 52) The feedback providing unit, When providing feedback, the most appropriate feedback is selected by referring to the user's past learning history. 8. The system of claim 7, (Appendix 53) The feedback providing unit, Estimate user emotions and prioritize feedback based on the estimated user emotions 8. The system of claim 7, (Appendix 54) The feedback providing unit, When providing feedback, consider the user's geographic location to provide relevant feedback 8. The system of claim 7, [Explanation of symbols]

[0221] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot

Claims

1. A dialogue unit that dialogues with the user in a language selected by the user; a pronunciation analysis unit that analyzes the pronunciation of the user's dialogue with the dialogue unit in real time and provides specific improvements; a quiz section that provides interactive quizzes on grammar and vocabulary according to a progress of the language learning of the user through the dialogue section; a background providing unit that provides information about the culture and customs of the language selected by the user through the dialogue unit. A system characterized by:

2. A situation-specific learning section is provided for carrying out language practice based on specific situations through the dialogue section.

2. The system of claim 1 .

3. A daily task learning unit that allows the user to practice specific daily tasks in the language selected by the user through the dialogue unit.

2. The system of claim 1 .

4. A story providing unit provides specific interactive content in which the user is the main character and progresses the story using language.

2. The system of claim 1 .

5. The dialogue unit includes: Combine voice, text, and video to engage in specific interactions with said users 2. The system of claim 1 .

6. A progress tracking unit that tracks the user's learning progress and ability and recommends specific lessons and exercises.

2. The system of claim 1 .

7. A feedback providing unit that provides specific feedback based on the user's learning progress and ability.

7. The system of claim 6.

8. The pronunciation analysis unit is Displaying the results of the analysis of the user's pronunciation in detail 2. The system of claim 1 .

Images