System

Abstract

A system is provided.SOLUTION: This system includes a means for generating questions for evaluating the level of a learner and preparing a learning program on the basis of the answer of the learner, a means for generating daily learning contents and providing them to the learner, a means for analyzing the answer data of the learner and timely reconstructing the learning program, and a means for executing a periodical level check as necessary and adjusting the learning program on the basis of the evaluation.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Traditional English learning methods make it difficult for learners to find learning materials and questions at the right level for them and to manage their learning progress appropriately, making it difficult to improve their skills efficiently. In particular, it takes a lot of time and effort to improve the four skills of listening, writing, reading, and speaking in a balanced manner, and there are limited options for customized learning plans that address individual learning progress and weaknesses. This leads to issues such as insufficient learning effectiveness or students abandoning their studies midway. [Means for solving the problem]

[0005] In order to solve the above-mentioned problems, the present invention provides a self-contained learning support system for efficiently and continuously improving learners' English skills. The present invention includes the following means:

[0006] 1. A means of generating questions to assess learners' levels and creating learning programs based on their answers.

[0007] 2. A means of generating and delivering daily learning content to learners.

[0008] 3. A means of analyzing learner response data and reconstructing learning programs in a timely manner.

[0009] 4. A means of conducting regular level checks and adjusting the study program based on the assessment, if necessary.

[0010] This allows learners to always be provided with a learning plan optimized for their level and progress, enabling them to improve their English skills efficiently and effectively. It also provides a comprehensive learning environment that strengthens the four skills of reading, writing, listening, and speaking in a balanced manner, thereby enhancing the overall effectiveness of English learning. Furthermore, by dynamically adjusting the learning program schedule according to the goals and achievement dates set by the learner, it clarifies the path to achieving the goal and maintains motivation to study.

[0011] "Questions" are questions used to assess learners' English skills and knowledge.

[0012] A "response" is the answer or response that a learner provides to a question.

[0013] A "study program" is a set of lessons or exercises designed to improve a learner's English proficiency.

[0014] "Learning content" refers to the specific teaching materials and questions provided to learners based on the learning program.

[0015] "Analysis" refers to the process of analyzing learner response data in detail and evaluating and reflecting the results.

[0016] "Restructuring" refers to updating or reorganizing existing learning programs based on learner progress and assessment results.

[0017] A "regular level check" is a series of questions or tests that are used to reassess a learner's English level at regular intervals.

[0018] "Adjustment" means changing or modifying the learning program or schedule according to the learner's needs and progress.

[0019] "Skills" refers to abilities and techniques related to reading, writing, listening, and speaking English. [Brief explanation of the drawings]

[0020] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7]FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram illustrating a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0021] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0022] First, the terms used in the following description will be explained.

[0023] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0024] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0025] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0026] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0027] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0028] [First embodiment]

[0029] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0030] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0031] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0032] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0033] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0034] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0035] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0036] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0037] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0038] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0039] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0040] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0041] The self-contained English skill improvement learning support system of the present invention is implemented as follows: The system is made up of a server, terminals, and users, and each element operates in cooperation with each other.

[0042] 1. Initial setup and level check

[0043] A user accesses the system using their terminal and logs in.

[0044] The server verifies the user's credentials and approves the login.

[0045] The server uses a generative AI model to generate questions for an initial level check, which assess each of the four skills: reading, writing, listening, and speaking.

[0046] The user answers the questions and sends the answers from the terminal to the server.

[0047] The server analyzes the user's response data and evaluates the user's initial level in each skill, thereby understanding the learner's current level.

[0048] 2. Setting goals and creating a learning program

[0049] The user inputs the goal they are aiming for (e.g., "Aiming for 800 points on the TOEIC") and the date by which they will achieve it on their device, and sends it to the server.

[0050] The server generates an optimal learning program based on the user's current level and set goals. This program includes practice content for each skill required to achieve the user's goal.

[0051] For example, it includes problem sets to strengthen reading skills and audio materials to improve listening skills.

[0052] 3. Providing daily learning content and analyzing learning results

[0053] The server generates daily learning content using a generative AI model and sends it to the device.

[0054] Users check the learning content through their devices and work on each assignment.

[0055] For example, "Today's listening questions" and "Today's writing assignments" are presented.

[0056] The user inputs the learning results and answers and sends them from the terminal to the server.

[0057] The server analyzes the user's response data and evaluates the user's learning outcomes for that day. Based on this evaluation, the learning content for the next day is adjusted.

[0058] 4. Regular level checks and program restructuring

[0059] The server periodically generates level check questions and sends them to the terminal.

[0060] The user answers the questions and sends the answer data from the terminal to the server.

[0061] The server analyzes the response data and evaluates the user's current English level.

[0062] The server reconstructs the learning program based on the latest evaluation and sends it to the terminal, allowing the user to always receive a program that meets their latest learning needs.

[0063] Specific examples

[0064] For example, let's consider a case where a user sets a goal of "aiming for 800 points on the TOEIC." In the initial level check, the user's reading ability is assessed as intermediate and their listening ability as beginner. In this case, the server provides intermediate-level reading comprehension questions to strengthen reading ability, while constructing a program that includes many beginner-level audio materials for listening ability.

[0065] In the daily learning session, a beginner-level audio file is provided as a listening test for the day, and the user is asked to answer the questions. When the user submits their answers, the server analyzes them and adjusts the program to provide audio materials with a slightly higher level of difficulty the following day.

[0066] Regular level checks evaluate progress since the last level check and check how much reading and listening skills have improved. Based on the results, a new learning program is set up so that users can continue to study appropriately toward their goals.

[0067] In this way, this system constantly monitors the progress of each individual user and dynamically optimizes their learning content, enabling them to improve their English skills efficiently and effectively.

[0068] The processing flow will be explained below.

[0069] Step 1:

[0070] A user accesses the system using their terminal and logs in.

[0071] The server verifies the user's credentials and approves the login.

[0072] Step 2:

[0073] The server uses a generative AI model to generate questions for an initial level check.

[0074] Specifically, it automatically generates questions to assess the four skills of reading, writing, listening, and speaking.

[0075] Step 3:

[0076] The server sends the generated question to the terminal.

[0077] The user checks the questions on the terminal and enters the answers.

[0078] Step 4:

[0079] The user sends the answer data from the terminal to the server.

[0080] The server receives the user's response data and analyzes it.

[0081] Step 5:

[0082] The server evaluates the user's initial level of the four skills based on the response data.

[0083] Based on the evaluation results, the user's current skill level is recorded.

[0084] Step 6:

[0085] The user uses the terminal to input the goal and the date of achievement into the server and transmit it.

[0086] Specifically, set a goal such as "aiming for 800 points on the TOEIC."

[0087] Step 7:

[0088] The server generates an optimal learning program based on the user's current level and set goals.

[0089] The program includes practice tailored to the skills of reading, listening, writing and speaking.

[0090] Step 8:

[0091] The server transmits the generated learning program to the terminal.

[0092] The user checks and practices the learning program on the device.

[0093] Step 9:

[0094] The server generates daily learning content using a generative AI model and sends it to the device.

[0095] The user works on the day's learning content (for example, listening questions and writing assignments) on the terminal.

[0096] Step 10:

[0097] The user inputs the results of that day's learning into the terminal and sends them to the server.

[0098] The server receives this and analyzes it.

[0099] Step 11:

[0100] The server evaluates the user's progress based on the learning results.

[0101] Based on the evaluation results, the learning content for the next day is adjusted and provided to the user.

[0102] Step 12:

[0103] The server generates questions for regular level checks as needed and sends them to the terminal.

[0104] The user answers this and sends the results to the server.

[0105] Step 13:

[0106] The server analyzes the results of regular level checks and evaluates the user's latest English level.

[0107] The learning program is reconstructed based on the evaluation results and sent to the terminal.

[0108] Step 14:

[0109] The cycle repeats when the user continues learning with a new learning program and logs in again.

[0110] Through these processing steps, the user is always provided with a study plan that is optimized for his or her level and progress, and is able to efficiently improve his or her English skills.

[0111] Example 1

[0112] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0113] Conventional learning support systems have difficulty dynamically and flexibly adjusting learning programs according to the user's learning progress. Furthermore, they lack automation to provide individualized learning programs based on each user's level and goals, making it difficult to effectively improve users' English skills.

[0114] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0115] In this invention, the server includes: means for generating questions to assess a user's level and creating a study program based on the user's answers; means for generating daily study content and providing it to the user; means for analyzing the user's answer data and reconstructing the study program as needed; means for conducting periodic level checks as needed and adjusting the study program based on the evaluation; means for the user to access the server through a terminal, input authentication information, and the server to verify the login against a database; means for the server to generate questions using a generative AI model and present them to the user through the terminal; and means for the server to analyze the user's study results and generate feedback to adjust the study content for the next day, thereby enabling efficient and effective improvement of English skills according to the individual needs of the user.

[0116] "User" refers to a person who uses the learning support system.

[0117] "Terminal" refers to a device such as a computer or smartphone that a user uses to access the system, display learning content, and enter answers.

[0118] A "server" refers to a computer system that manages the entire system and performs various processes in cooperation with a database.

[0119] A "generative AI model" is a model that uses artificial intelligence technology and refers to a program used to generate learning content and analyze responses.

[0120] "Questions" refer to problems and tasks generated by the system to assist users in assessing their level and learning.

[0121] "Study program" means a set of learning activities or tasks designed to engage a user in English language learning.

[0122] "Response data" refers to the response information entered by the user in response to questions or learning content.

[0123] "Evaluation" refers to the process of determining the progress and level of learning based on the user's response data.

[0124] "Feedback" refers to information about areas for improvement and next learning content provided to the user based on the results of the server's analysis.

[0125] "Login" refers to the process by which a user enters authentication information to gain access to a system.

[0126] "Learning content" refers to the specific tasks and learning materials that users must address.

[0127] "Regular level checks" refers to tests or questions administered to periodically assess a user's learning progress.

[0128] The self-contained English skill improvement learning support system of the present invention is implemented as follows: This system is made up of a server, terminals, and users, and each element operates in cooperation with each other.

[0129] The server includes a means for requiring a user to enter authentication information when accessing the system through a terminal and for approving the login by checking this information against a database, thereby allowing the user to access the system. For example, the process involves a user entering an email address and password on a login screen, which is then verified by the server.

[0130] The server provides a means to generate questions using a generative AI model (e.g., GPT-4) to assess the user's level. The generated questions assess the four skills of reading, writing, listening, and speaking individually. This allows for a comprehensive understanding of the user's skill level. For example, intermediate-level written questions are provided for reading, and easy audio questions are provided for listening.

[0131] Users use their devices to answer questions and send the results to the server. The server analyzes the answers and uses NLP technology to assess the user's initial skill level. The server also includes a means to customize a learning program based on the user's set goal (e.g., "800 points on the TOEIC") and the date by which it is achieved. The program includes intermediate-level problem sets to strengthen reading and beginner-level audio materials to improve listening comprehension.

[0132] The server generates daily learning content using a generative AI model and provides a means to send it to the device. For example, "Today's listening questions" and "Today's writing assignments" are presented. The user can check and work on these learning content through the device.

[0133] The user inputs the results of their study into their device and sends them to the server. The server analyzes the user's response data and evaluates the day's study results. The system includes a means for adjusting the study content for the next day based on this evaluation. Specifically, if the user correctly answers a beginner-level listening question, a slightly more difficult question will be presented the next day.

[0134] Additionally, the server provides a means to conduct regular level checks as needed. These checks also use the generative AI model to generate questions and present them to the user. The server then analyzes the user's answers again to assess their current skill level. Based on the results, it reconstructs the learning program and sends it to the device. This ensures that the user's learning plan is always up to date.

[0135] An example of a specific prompt might be, "The user wants to learn intermediate-level reading comprehension questions to improve their reading. Please generate appropriate questions." This prompt is input into a generative AI model, which then generates appropriate reading questions.

[0136] In this way, this system provides a high level of collaboration between the server, terminals, and users, enabling efficient and effective improvement of English skills according to the individual needs of each user.

[0137] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0138] System program processing flow

[0139] Step 1: User Login and Authentication

[0140] Input: The user enters their email address and password into the terminal and presses the login button.

[0141] Action: The device sends the entered authentication information to the server.

[0142] Processing: The server checks the authentication information against the database and authorizes the user to log in.

[0143] Output: Successful login information is sent to the terminal and displayed to the user.

[0144] Step 2: Generate questions for initial level check

[0145] Input: After the server verifies the user's login, it starts the initial level check.

[0146] How it works: The server inputs prompts into the generative AI model, which generates questions for reading, writing, listening, and speaking.

[0147] Processing: The generative AI model generates a question based on the prompt and returns the result to the server.

[0148] Output: The generated questions are sent to the terminal and displayed to the user.

[0149] Step 3: Conduct an initial level check

[0150] Input: The user answers the questions and enters the answer data into the terminal.

[0151] Operation: The device sends the response data to the server.

[0152] Processing: The server analyzes the response data and evaluates the user's initial level for each skill (reading, writing, listening, and speaking).

[0153] Output: Evaluation results and feedback are sent to the device.

[0154] Step 4: Set goals and create a learning program

[0155] Input: The user inputs the goal (e.g., "800 points on the TOEIC") and the deadline for achieving it into the terminal and submits it.

[0156] Operation: The device sends the goal and deadline information to the server.

[0157] Processing: The server generates a learning program based on the user's initial level and goals. In this process, the generative AI model is used again.

[0158] Output: The customized learning program is sent to the terminal and displayed to the user.

[0159] Step 5: Provide daily learning content

[0160] Input: The server generates today's learning content based on the learning program.

[0161] How it works: The generative AI model generates learning content based on the prompt.

[0162] Processing: The server sends the generated learning content to the terminal.

[0163] Output: Today's learning content will be displayed on the terminal.

[0164] Step 6: Input and analysis of training results

[0165] Input: The user works on the learning content and inputs the results into the terminal.

[0166] Operation: The device sends the learning results to the server.

[0167] Processing: The server analyzes the learning results, evaluates the learning outcomes of the day, and generates feedback to adjust the learning content for the next day.

[0168] Output: Analysis results and applied feedback are sent to the device.

[0169] Step 7: Regular level checks and program restructuring

[0170] Input: After a certain period of time has passed, the server generates questions for regular level checks.

[0171] How it works: The regenerative AI model generates a question based on the prompt.

[0172] Processing: The server sends the generated questions to the terminal, and the user answers them. The answer data is sent to the server and analyzed.

[0173] Output: The latest skill level evaluation results are sent to the terminal, and the learning program is reconstructed.

[0174] Through the above processing steps, the system of the present invention efficiently and effectively supports users in improving their English skills in accordance with their individual needs.

[0175] (Application example 1)

[0176] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0177] Improving listening and speaking skills in English learning is a difficult task for many learners. In particular, there is a demand for efficient use of time while riding in an autonomous vehicle to improve English skills. To solve this problem, an effective English learning support system using smart devices is required.

[0178] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0179] In this invention, the server includes a means for generating questions to assess the learner's level and creating a learning program based on the learner's answers, a means for generating daily learning content and providing it to the learner, and a means for analyzing the learner's answer data and reconstructing the learning program as needed, thereby enabling a means for receiving and analyzing voice input using a smart device, a means for dynamically generating questions and answers using a generative AI model, and a means for providing audio and video content to the learner using a smart device.

[0180] "Learner" refers to an individual who uses the System to improve their English language skills.

[0181] "Level assessment questions" are questions that are asked to understand the learner's current level of reading, writing, listening, and speaking skills.

[0182] A "learning program" is a plan that combines the practice and materials necessary for a learner to achieve their desired English skills.

[0183] A "generative AI model" is an algorithm that uses artificial intelligence technology to automatically generate optimal learning content and questions.

[0184] "Smart devices" refer to electronic devices that can connect to the Internet and have voice input and video display functions, and examples include smartphones and smart glasses.

[0185] "Means for receiving and analyzing voice input" refers to the function of collecting learners' speech through the microphone of a smart device and recognizing and analyzing its content.

[0186] "Means for reviewing and dynamically providing learning content" refers to a function that generates and delivers optimal practice questions and teaching materials in real time according to the learner's level and progress.

[0187] "Means for dynamically generating questions and answers" refers to a function that uses a generative AI model to generate appropriate questions and answers in real time according to the learner's level and learning progress.

[0188] "Means for providing audio and video content" refers to a function that supports learning by allowing learners to listen to audio or watch videos using smart devices.

[0189] "Regular level checks" refer to tests conducted at regular intervals to assess learners' progress and determine their current English skill level.

[0190] The English conversation learning support system using smart devices of the present invention is implemented as follows: This system is made up of a server, a terminal, and a user, and each element works in cooperation with each other.

[0191] 1. Initial setup and level check

[0192] The user accesses the system using their own terminal (smart device) and logs in.

[0193] The server verifies the user's credentials and approves the login.

[0194] The server uses a generative AI model to generate questions for an initial level check, which assess each of the four skills: reading, writing, listening, and speaking.

[0195] The user answers the questions and sends the answers from the terminal to the server.

[0196] The server analyzes the user's response data and evaluates the user's initial level in each skill, thereby understanding the learner's current level.

[0197] 2. Setting goals and creating a learning program

[0198] The user inputs the goal (e.g., "to acquire English conversation skills at a daily conversation level") and the date by which it will be achieved from the terminal and sends it to the server.

[0199] The server generates an optimal learning program based on the user's current level and set goals. This program includes problem sets and audio materials to improve listening and speaking skills.

[0200] 3. Providing daily learning content and analyzing learning results

[0201] The server generates daily learning content using a generative AI model and sends it to the device.

[0202] The user checks the learning content through the device and works on each task. For example, "Today's listening questions" and "Today's speaking tasks" are presented.

[0203] The user responds by voice input, and the voice data is sent from the terminal to the server.

[0204] The server analyzes the audio data and evaluates the learning outcomes for that day, and the learning content for the next day is adjusted based on this evaluation.

[0205] 4. Regular level checks and program restructuring

[0206] The server periodically generates level check questions and sends them to the terminal.

[0207] The user answers the questions and sends the answer data from the terminal to the server.

[0208] The server analyzes the response data and evaluates the user's current English level.

[0209] The server reconstructs the learning program based on the latest evaluation and sends it to the terminal, allowing the user to always receive a program that meets their latest learning needs.

[0210] Hardware and software used

[0211] Hardware: Smart devices (smartphones, smart glasses, etc.)

[0212] Software: Generative AI models, speech recognition libraries (e.g., speech_recognition)

[0213] The system uses a generative AI model to dynamically generate questions and collects and analyzes user speech data through speech recognition, allowing users to efficiently improve their English conversation skills.

[0214] Specific examples

[0215] When a user uses this system to learn English conversation while riding in an autonomous vehicle, the following prompt sentences are generated:

[0216] "Please generate a list of English conversation phrases suitable for a beginner level. These phrases should be commonly used in travel situations."

[0217] For example, a passenger repeats the following phrase presented as "Today's Listening Question."

[0218] "How can I get to the nearest train station?"

[0219] Once the passenger has spoken correctly, their speech is analyzed and the next phrase is presented.

[0220] "Thank you! Let's move on to the next phrase."

[0221] In this way, you can efficiently improve your English conversation skills while managing your progress.

[0222] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0223] Step 1:

[0224] Initial setup and level check

[0225] When a user accesses the system using their own device, the server verifies the user's authentication information and approves the login. Next, it uses the generative AI model to generate questions for an initial level check and sends them to the device. The user answers the questions and sends the answers from the device to the server. The server then analyzes the user's answer data and evaluates the user's initial level in each skill. In this process, the user's answers are the input data, the generative AI model generates questions, and the server analyzes the data to evaluate the learner's level.

[0226] Step 2:

[0227] Setting goals and creating learning programs

[0228] The user inputs their desired goal (e.g., "Acquire English conversation skills at a daily conversation level") and the date by which they will achieve this goal from their device and sends it to the server. The server generates an optimal learning program based on the user's current level and the set goal. Specifically, it uses a generative AI model to dynamically create problem sets and audio materials to improve listening and speaking skills. In this process, the user's goal and current level are input data, and the generative AI model is used to generate a learning program, which outputs an optimal learning plan.

[0229] Step 3:

[0230] Providing daily learning content and analyzing learning results

[0231] The server generates daily learning content using a generative AI model and sends it to the device. The user checks the learning content through the device and works on each assignment. The user answers through voice input, and the voice data is sent from the device to the server. The server analyzes the voice data and evaluates the learning outcome for that day. Based on this evaluation, the learning content for the next day is adjusted. In this process, the user's voice data is the input data, and the server analyzes and evaluates the learning outcome, adjusting the learning content for the next day to obtain the output.

[0232] Step 4:

[0233] Regular level checks and program restructuring

[0234] The server periodically generates level-check questions and sends them to the terminal. The user answers the questions and sends the answer data from the terminal to the server. The server analyzes the answer data and evaluates the user's latest English level. The server reconstructs a learning program based on the latest evaluation and sends it to the terminal. This allows the user to always receive a program that meets their latest learning needs. In this process, the user's answer data is the input data, and the server analyzes it to evaluate the user's latest English level, then generates and outputs a new learning program.

[0235] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0236] The self-contained English skill improvement learning support system of this invention achieves a higher level of personalization by combining a conventional learning support system with an emotion engine that recognizes the user's emotions. The system is composed of a server, terminals, and users, and each element works in cooperation with each other.

[0237] 1. Initial setup and level check

[0238] A user accesses the system using their terminal and logs in.

[0239] The server verifies the user's credentials and approves the login.

[0240] The server uses a generative AI model to generate questions for an initial level check, which automatically generate questions to assess the four skills of reading, writing, listening, and speaking.

[0241] The user answers the questions and sends the answers from the terminal to the server.

[0242] The server analyzes the user's response data and evaluates the user's initial level in each skill, thereby understanding the learner's current level.

[0243] 2. Setting goals and creating a learning program

[0244] The user inputs the goal they are aiming for (e.g., "Aiming for 800 points on the TOEIC") and the date by which they will achieve it on their device, and sends it to the server.

[0245] The server generates an optimal learning program based on the user's current level and set goals. This program includes practice content for each skill required to achieve the user's goal.

[0246] For example, it includes problem sets to strengthen reading skills and audio materials to improve listening skills.

[0247] 3. Providing daily learning content and analyzing learning results

[0248] The server generates daily learning content using a generative AI model and sends it to the device.

[0249] Users check the learning content through their devices and work on each assignment.

[0250] For example, "Today's listening questions" and "Today's writing assignments" are presented.

[0251] The user inputs the learning results and answers and sends them from the terminal to the server.

[0252] The server receives this and analyzes it.

[0253] 4. Implementation and Use of Emotion Engine

[0254] The server uses an emotion engine to recognize the user's emotions in real time, specifically assessing the user's current emotions (e.g., stress, concentration, fatigue, etc.) through facial recognition technology and voice analysis.

[0255] The server then adjusts learning content and break suggestions based on the recognized emotion data. For example, if the user feels tired, a pop-up will appear recommending a break.

[0256] The user accepts the sentiment-based suggestions and chooses whether to pause or continue learning.

[0257] 5. Regular level checks and program restructuring

[0258] The server periodically generates level check questions and sends them to the terminal.

[0259] The user answers the questions and sends the answer data from the terminal to the server.

[0260] The server analyzes the response data and evaluates the user's current English level.

[0261] The server reconstructs the learning program based on the latest evaluation and sends it to the terminal, allowing the user to always receive a program that meets their latest learning needs.

[0262] Specific examples

[0263] For example, consider a case where a user sets a goal of "aiming for 800 points on the TOEIC." In the initial level check, the user's reading ability is assessed as intermediate and their listening ability as beginner. In this case, the server provides intermediate-level reading comprehension questions to strengthen reading ability, while constructing a program that includes many beginner-level audio materials for listening ability.

[0264] During daily learning, the emotion engine detects the user's stress level, and if high stress is detected, the server reduces the learning content and switches to more relaxing content. If the user feels tired, a notification is displayed recommending a break.

[0265] Regular level checks evaluate progress since the last level check and check how much reading and listening skills have improved. Based on the results, a new learning program is set up so that users can continue to study appropriately toward their goals.

[0266] In this way, the system constantly monitors the user's progress and emotional state, dynamically optimizing learning content to improve English skills efficiently and effectively.

[0267] The processing flow will be explained below.

[0268] Step 1:

[0269] A user accesses the system using their terminal and logs in.

[0270] The server verifies the user's credentials and approves the login.

[0271] Step 2:

[0272] The server uses a generative AI model to generate questions for an initial level check.

[0273] Specifically, it automatically generates questions to assess the four skills of reading, writing, listening, and speaking.

[0274] Step 3:

[0275] The server sends the generated question to the terminal.

[0276] The user checks the questions on the terminal and enters the answers.

[0277] Step 4:

[0278] The user sends the answer data from the terminal to the server.

[0279] The server receives the user's response data and analyzes it.

[0280] Step 5:

[0281] The server evaluates the user's initial level of the four skills based on the response data.

[0282] Based on the evaluation results, the user's current skill level is recorded.

[0283] Step 6:

[0284] The user uses the terminal to input the goal and the date of achievement into the server and transmit it.

[0285] Specifically, set a goal such as "aiming for 800 points on the TOEIC."

[0286] Step 7:

[0287] The server generates an optimal learning program based on the user's current level and set goals.

[0288] The program includes practice tailored to the skills of reading, listening, writing and speaking.

[0289] Step 8:

[0290] The server transmits the generated learning program to the terminal.

[0291] The user checks and practices the learning program on the device.

[0292] Step 9:

[0293] The server generates daily learning content using a generative AI model and sends it to the device.

[0294] The user works on the day's learning content (for example, listening questions and writing assignments) on the terminal.

[0295] Step 10:

[0296] The user inputs the results of that day's learning into the terminal and sends them to the server.

[0297] The server receives this and analyzes it.

[0298] Step 11:

[0299] The server evaluates the user's progress based on the learning results.

[0300] Based on the evaluation results, the learning content for the next day is adjusted and provided to the user.

[0301] Step 12:

[0302] The server uses an emotion engine to recognize the user's emotions in real time.

[0303] Specifically, it assesses the user's current emotions (e.g., stress, concentration, fatigue, etc.) through facial recognition technology and voice analysis.

[0304] Step 13:

[0305] The server adjusts learning content and break suggestions based on the recognized emotion data.

[0306] For example, if stress levels are high, the system will provide less difficult questions and switch to content that has a relaxing effect.

[0307] Step 14:

[0308] The user chooses whether to accept sentiment-based suggestions.

[0309] If the user feels they need a break, they can pause their learning and refresh.

[0310] Step 15:

[0311] The server generates questions for regular level checks as needed and sends them to the terminal.

[0312] The user answers this and sends the results to the server.

[0313] Step 16:

[0314] The server analyzes the results of regular level checks and evaluates the user's latest English level.

[0315] The learning program is reconstructed based on the evaluation results and sent to the terminal.

[0316] Step 17:

[0317] The user continues learning with a new learning program.

[0318] When you log in again, the cycle repeats.

[0319] Through these processing steps, users are always provided with a learning plan that is optimized for their level, progress, and emotional state, allowing them to efficiently improve their English skills.

[0320] Example 2

[0321] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0322] Conventional learning support systems provide learning programs tailored to a learner's current level and goals, but they are unable to recognize the learner's emotional state and adjust the learning content based on that emotion. This creates the problem that appropriate support is not provided even when the learner feels fatigued or stressed, making efficient learning difficult. In addition, rebuilding learning programs and conducting regular level checks is cumbersome, creating a need for automatic dynamic adjustments.

[0323] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0324] In this invention, the server includes means for generating questions to assess the learner's level and creating a learning program based on the learner's answers, means for generating daily learning content and providing it to the learner, means for recognizing the learner's emotional state in real time and adjusting the learning content and break suggestions, means for analyzing the learner's answer data and reconstructing the learning program as appropriate, and means for conducting regular level checks as necessary and adjusting the learning program based on the evaluation, thereby enabling efficient and personalized learning support that takes the learner's emotional state into consideration.

[0325] The term "learner" refers to an individual who uses the learning system for the purpose of improving a particular skill (mainly language skill in the present invention).

[0326] A "server" refers to a computer system that provides various services to learners via a network, and in the present invention, it mainly generates and provides learning programs.

[0327] "Question" refers to a question or problem generated by the server to assess a learner's skill or level.

[0328] "Learning program" refers to a set of educational curriculum and assignments created by the server based on the learner's current level and goals.

[0329] "Learning content" refers to specific learning tasks and learning materials that are generated by the server daily and provided to learners.

[0330] "Response data" refers to the responses entered by learners to questions and assignments.

[0331] "Emotional state" refers to the learner's current psychological or emotional state (e.g., stress, concentration, fatigue).

[0332] An "emotion engine" refers to a system that uses facial recognition technology, voice analysis technology, and other techniques to recognize a learner's emotional state in real time.

[0333] "Dynamic adjustment" refers to changing and optimizing learning programs and content in real time based on the learner's progress and emotional state.

[0334] "Level check" refers to a test or assessment to assess a learner's current skill level.

[0335] This invention relates to a self-contained English skill improvement learning support system that assesses learners' levels and creates and provides learning programs. The system consists of a server, terminals, and learners, and each element works in conjunction with the others. The server uses a generative AI model to generate questions and learning content, and an emotion engine to recognize the learner's emotional state and dynamically adjust the learning content.

[0336] Initial setup and level check

[0337] The learner accesses the system using a device and logs in. The server verifies the learner's authentication information and approves the login. The server uses a generative AI model to generate questions to assess the four skills of reading, writing, listening, and speaking. The learner answers the questions and sends the data from their device to the server. The server analyzes the response data and evaluates the initial level of each skill to determine the learner's current skill level.

[0338] Setting goals and creating learning programs

[0339] Learners input their goal (e.g., "Aim for 800 points on the TOEIC") and the target date for achieving it on their device and send it to the server. The server then generates an optimal learning program based on the learner's current level and the set goal. This program might include, for example, a problem set to improve reading skills or audio materials to improve listening ability.

[0340] Providing daily learning content and analyzing learning results

[0341] The server uses a generative AI model to generate daily learning content and sends it to the device. The learner checks the learning content through the device and works on each task. For example, they are presented with a "today's listening question" or a "today's writing task." The learner inputs their learning results and answers and sends them from their device to the server. The server analyzes the received data and evaluates the learner's progress.

[0342] Implementing and utilizing an emotion engine

[0343] The server uses an emotion engine to recognize the learner's emotional state in real time. Specifically, it uses facial recognition technology and voice analysis to assess the learner's current emotions (e.g., stress, concentration, fatigue, etc.). The server adjusts learning content and break suggestions based on the emotion data. For example, if the learner feels fatigued, a pop-up recommending a break will be displayed. The learner can accept the emotion-based suggestion and choose whether to pause or continue learning.

[0344] Regular level checks and program restructuring

[0345] The server periodically generates level-check questions and sends them to the device. The learner answers the questions and sends the answer data from the device to the server. The server analyzes the data and evaluates the learner's current English level. Based on this evaluation, the server reconstructs the learning program and sends it to the device. This ensures that the learner always receives a program that meets their latest learning needs.

[0346] Specific examples and examples of prompts for the generative AI model

[0347] For example, consider a case where a learner sets a goal of "achieving 800 points on the TOEIC." Suppose that the initial level check assesses reading as intermediate and listening as beginner. In this case, the server provides intermediate-level reading comprehension questions to strengthen reading, while constructing a program that includes many beginner-level audio materials for listening. During daily learning, if the emotion engine detects high stress, the server reduces the learning content and switches to content with a relaxing effect. Furthermore, if the learner feels tired, a notification is displayed recommending a break.

[0348] Example prompts for generative AI models

[0349] "A student is aiming for 800 points on the TOEIC, and their current reading level is intermediate and their listening level is beginner. Please generate daily study content based on the student's progress."

[0350] In this way, the system constantly monitors learners' progress and emotional state, dynamically optimizing learning content to help them improve their English skills efficiently and effectively.

[0351] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0352] Step 1:

[0353] A user accesses the system using their terminal and logs in.

[0354] Input: User ID and password.

[0355] How it works: The user opens a browser or a dedicated app, accesses the login screen, and enters their user ID and password.

[0356] Output: The entered authentication information is sent to the server.

[0357] Step 2:

[0358] The server verifies the user's credentials and approves the login.

[0359] Input: User ID and password.

[0360] How it works: The server accesses the database and checks the entered credentials.

[0361] Output: If authentication is successful, the login is approved and the user is shown the dashboard.

[0362] Step 3:

[0363] The server uses a generative AI model to generate questions for an initial level check.

[0364] Input: User login information.

[0365] How it works: The server sends prompts to a generative AI model to generate questions to assess the four skills of reading, writing, listening, and speaking.

[0366] Output: The generated questions are sent to the user's terminal.

[0367] Step 4:

[0368] The user answers the questions and sends the answers from the terminal to the server.

[0369] Input: The generated question.

[0370] How it works: The user reviews the questions on the terminal and enters answers to each question.

[0371] Output: The answer data is sent from the device to the server.

[0372] Step 5:

[0373] The server analyzes the user's response data and evaluates the user's initial level in each skill.

[0374] Input: User response data.

[0375] How it works: The server uses natural language processing (NLP) technology to analyze the response data and assess the initial level of each skill (reading, writing, listening, and speaking).

[0376] Output: User's initial level assessment result.

[0377] Step 6:

[0378] The user inputs the goal and the date of achievement from the terminal and transmits it to the server.

[0379] Input: Your goal and the date you want to reach it.

[0380] How it works: The user enters the goal and target achievement date into a dedicated input form and presses the submit button.

[0381] Output: The goal and the target date are sent to the server.

[0382] Step 7:

[0383] The server generates an optimal learning program based on the user's current level and set goals.

[0384] Input: User's initial level assessment results and goal / target achievement date.

[0385] How it works: The server sends prompts to the generative AI model to generate a learning program appropriate for the user's current level and goals.

[0386] Output: The generated learning program is sent to the user's terminal.

[0387] Step 8:

[0388] The server generates daily learning content using a generative AI model and sends it to the device.

[0389] Input: The user's study program.

[0390] How it works: The server sends prompts to the generative AI model to generate learnings for the day.

[0391] Output: The generated learning content is sent to the user's device.

[0392] Step 9:

[0393] Users check the learning content through their devices and work on each assignment.

[0394] Input: What you learned that day.

[0395] Action: The user checks the learning content on the device and works on the assignments. For example, they play an audio file and answer listening questions.

[0396] Output: Learning results and answer data.

[0397] Step 10:

[0398] The user inputs the learning results and answers and sends them from the terminal to the server.

[0399] Input: Learning results and answer data.

[0400] Operation: The user enters the learning results and answers and presses the submit button.

[0401] Output: Results and response data are sent to the server.

[0402] Step 11:

[0403] The server uses an emotion engine to recognize the learner's emotional state in real time.

[0404] Input: Learner's video and audio data.

[0405] Operation: The server runs the emotion engine to assess the learner's current emotional state through facial recognition technology and voice analysis.

[0406] Output: Emotional state assessment results.

[0407] Step 12:

[0408] The server adjusts learning content and break suggestions based on the recognized emotion data.

[0409] Input: Emotional state assessment results.

[0410] How it works: The server adjusts the difficulty of the learning content and suggests breaks based on the user's emotional state. For example, if the user is highly fatigued, a pop-up will appear recommending a break.

[0411] Output: Tailored learning content and break suggestions.

[0412] Step 13:

[0413] The user chooses whether to accept sentiment-based suggestions.

[0414] Input: Study content and break suggestions.

[0415] Action: The user reviews the suggestions and selects "Take a break" or "Continue learning."

[0416] Output: Next action based on user selection.

[0417] Step 14:

[0418] The server periodically generates level check questions and sends them to the terminal.

[0419] Input: Learner progress data.

[0420] How it works: The server uses the generative AI model to generate questions for regular level checks.

[0421] Output: Generated level check questions.

[0422] Step 15:

[0423] The user answers the questions and sends the answer data from the terminal to the server.

[0424] Input: Questions for regular level checks.

[0425] Operation: The user answers the questions and sends the answer data from the terminal to the server.

[0426] Output: The answer data is sent to the server.

[0427] Step 16:

[0428] The server analyzes the response data and evaluates the user's current English level.

[0429] Input: Response data.

[0430] How it works: The server analyzes the response data and evaluates your current English level.

[0431] Output: Latest English level assessment results.

[0432] Step 17:

[0433] The server reconstructs the learning program based on the latest evaluation and sends it to the device.

[0434] Input: Your most recent English level assessment results.

[0435] How it works: The server uses the generative AI model to reconstruct a learning program based on the latest English level.

[0436] Output: A reconstructed learning program.

[0437] (Application example 2)

[0438] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0439] Conventional learning support systems provide learning content mechanically without considering the learner's emotional state, resulting in problems such as reduced learning efficiency due to learner fatigue or stress. While these systems dynamically adjust learning programs based on the learner's progress and ability, they do not monitor the learner's emotional state, resulting in an insufficient optimization of the individual learning experience. In particular, in work environments using robots, such as factories, a system is needed that can optimize work efficiency according to the worker's emotional state.

[0440] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: a means for generating questions to evaluate the learner's level and creating a learning program based on the learner's answers; a means for generating daily learning content and providing it to the learner; a means for analyzing the learner's answer data and reconstructing the learning program as needed; a means for conducting periodic level checks as needed and adjusting the learning program based on the evaluation; and a means for recognizing the learner's emotional state in real time and dynamically optimizing the learning content based on the learner's emotional state. This makes it possible to provide a learning program that takes the learner's emotional state into consideration, thereby reducing learner stress and fatigue and improving learning efficiency. Furthermore, by applying this to factory work sites, it is possible to optimize work schedules and robot operations according to the worker's emotional state.

[0441] "Assessing the learner's level" means measuring the level of knowledge and skills that the learner currently possesses and providing appropriate learning content and programs based on that.

[0442] "Question generation" means automatically creating questions or problems to assess a learner's ability.

[0443] "Creating a learning program" means planning and designing learning content and activities based on the learner's goals and current abilities.

[0444] "Generating daily learning content" means automatically creating and providing learning tasks and materials that learners should tackle every day.

[0445] "Providing to learners" means displaying and transmitting learning content and assignments to the devices and systems used by learners.

[0446] "Analyzing learner response data" means analyzing the responses and results submitted by learners and evaluating their learning progress and level of understanding based on the results.

[0447] "Reconstructing learning programs in a timely manner" means dynamically updating and redesigning learning content and programs according to learners' progress and assessment results.

[0448] "Conducting regular level checks" means administering tests and questions to assess learners' abilities on a regular basis.

[0449] "Recognizing the learner's emotional state in real time" means instantly assessing the learner's current emotional and psychological state using information such as their facial expressions and voice.

[0450] "Dynamic optimization of learning content based on emotional state" means flexibly adjusting the learning content and difficulty level by reflecting the learner's emotional and psychological state.

[0451] "Facial recognition technology" is a technology that detects an individual's face from camera footage and analyzes their facial expressions and characteristics.

[0452] "Voice analysis technology" is a technology that analyzes voice data and evaluates its content and the speaker's emotional state.

[0453] "Detecting fatigue and stress" means determining the level of fatigue and stress experienced by learners through facial recognition and voice analysis.

[0454] "Displaying a notification recommending a break" means that when the system detects that the learner is fatigued or stressed, a message urging the learner to take a break will be displayed on the device.

[0455] The present invention provides a learning support system that recognizes the emotional state of a learner in real time and dynamically optimizes the generation and provision of learning programs. The system is composed of a server, terminals, and users.

[0456] 1. Initial setup and level check

[0457] A user accesses the system using a terminal and logs in. The server verifies the user's authentication information and approves the login. The server then uses a generative AI model to generate questions for an initial level check. These questions are designed to assess the four skills of reading, writing, listening, and speaking. The user answers the questions and sends the answers from the terminal to the server. The server analyzes the user's response data and evaluates the initial level in each skill.

[0458] 2. Setting goals and creating a learning program

[0459] The user inputs their goal (for example, "Aiming for 800 points on the TOEIC") and the date they wish to achieve it, and sends it to the server. The server then generates an optimal learning program based on the user's current level and the goal they set. This program includes practice content for each skill the user needs to achieve their goal.

[0460] 3. Providing daily learning content and analyzing learning results

[0461] The server uses a generative AI model to generate daily learning content and sends it to the device. The user checks the learning content through the device and works on each task. The device then sends the learning results and answers to the server, which receives and analyzes them.

[0462] 4. Implementation and Use of Emotion Engine

[0463] The server uses an emotion engine to recognize the user's emotions in real time. The hardware used includes a camera and microphone. The software uses OpenCV for facial recognition technology and TensorFlow / Keras for emotion recognition models. Using these technologies, the server evaluates the user's current emotions (e.g., stress, concentration, fatigue, etc.). Based on the recognized emotion data, the server adjusts the learning content and suggests breaks. For example, if the server recognizes that the user is feeling fatigued, it will display a notification on the device recommending a break.

[0464] 5. Regular level checks and program restructuring

[0465] The server periodically generates level-check questions and sends them to the device. The user answers the questions and sends the answer data from the device to the server. The server analyzes the answer data and evaluates the user's latest English level. Based on the latest evaluation, the server reconstructs the learning program and sends it to the device. This ensures that the user always receives a program that meets their latest learning needs.

[0466] Specific examples

[0467] For example, consider a user who sets a goal of "achieving 800 points on the TOEIC." Suppose the user's reading and listening skills are assessed as intermediate and beginner levels in the initial level check. In this case, the server provides intermediate-level reading comprehension questions to strengthen reading, while constructing a program that includes many beginner-level audio materials for listening. During daily study, the emotion engine detects the user's stress level. If high stress is recognized, the server reduces the study content and switches to content with a relaxing effect. If the user feels tired, the server displays a notification recommending a break. Periodic level checks evaluate progress since the previous level check and confirm the degree of improvement in reading and listening skills. Based on the results, a new study program is created, allowing the user to continue studying appropriately toward their goal.

[0468] Prompt Sentence Examples

[0469] "Please explain how to recognize the emotions of factory workers in real time and optimize labor efficiency based on their emotions. I would like to build a system that sends emotional data to a server, which then adjusts optimal work schedules and robot operations. As an example, please explain in detail how you combine facial recognition technology with an emotion engine."

[0470] This allows learners' progress and emotional state to be constantly monitored, and learning content to be dynamically optimized, enabling them to improve their English skills efficiently and effectively.

[0471] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0472] Step 1:

[0473] A user accesses the system using a terminal and logs in. The user enters login information (user ID and password), which the terminal sends to the server. The server verifies the user's authentication information and approves the login. The input data is the user's authentication information, and the output data is the login success or failure status.

[0474] Step 2:

[0475] The server uses a generative AI model to generate questions for an initial level check. The server uses prompts to request questions to assess the skills of reading, writing, listening, and speaking from the generative AI model, and sends the generated questions to the terminal. The input data is the prompts, and the output data is the generated questions.

[0476] Step 3:

[0477] The user answers questions through the terminal and sends the answers to the server. The terminal acquires the user's input and sends the data to the server. The input data is the user's answer, and the output data is a notification that the answer data has been sent.

[0478] Step 4:

[0479] The server analyzes the user's response data and evaluates their initial level. The server then inputs the response data into an analysis algorithm to calculate a score for each skill. The input data is the user's response data, and the output data is the evaluation result of the initial level for each skill.

[0480] Step 5:

[0481] The user inputs the goal they are aiming for and the date by which they will reach it, and sends it to the server. The user inputs the goal they have set (for example, "Aim for 800 points on the TOEIC") and the date by which they plan to reach it from their terminal, and sends it to the server. The input data is the goal and the date by which it will be reached, and the output data is a notification that the goal has been set.

[0482] Step 6:

[0483] The server generates an optimal learning program based on the user's current level and set goals. Based on the initial level assessment results and goals, the server requests the AI ​​model to generate learning content suitable for the user, and creates that content. The input data is the initial level assessment results and the user's goals, and the output data is the generated learning program.

[0484] Step 7:

[0485] The server generates daily learning content using a generative AI model and sends it to the device. The server dynamically adjusts the daily learning content taking into account the user's progress and emotional state and sends it to the device. The input data is the user's progress and emotional state, and the output data is the generated daily learning program.

[0486] Step 8:

[0487] The user works on daily learning content through the terminal and inputs the results. The user performs the learning tasks displayed on the terminal and inputs the answers. The input data is the user's learning results, and the output data is a notification that the learning results have been sent.

[0488] Step 9:

[0489] The server receives and analyzes the learning result data. The server analyzes the received learning results and evaluates the progress. The input data is the learning result data, and the output data is the progress evaluation result.

[0490] Step 10:

[0491] The server uses an emotion engine to recognize the user's emotions in real time. The server then activates the emotion engine and analyzes the camera video and audio data to evaluate the user's emotional state. The input data are the camera video and audio data, and the output data are the emotion evaluation results.

[0492] Step 11:

[0493] The server adjusts the learning content and break suggestions based on the emotion evaluation results. The server dynamically adjusts the learning content based on the emotion evaluation results, and in some cases displays a notification on the device recommending a break. The input data is the emotion evaluation results, and the output data is the adjusted learning content and break suggestions.

[0494] Step 12:

[0495] The server periodically generates level check questions and sends them to the terminal. The server also generates questions to reassess the user's skills at regular intervals and sends them to the terminal. The input data is the prompt text, and the output data is the generated level check questions.

[0496] Step 13:

[0497] The user answers questions for the regular level check and sends the results to the server. The user answers questions and sends the data to the server. The input data is the answers to the regular level check, and the output data is a notification that the answer data has been sent.

[0498] Step 14:

[0499] The server analyzes the response data and evaluates the user's latest English level. The server analyzes the response data and evaluates the user's progress. The input data is the response data from the regular level check, and the output data is the latest English level evaluation result.

[0500] Step 15:

[0501] The server reconstructs the learning program based on the latest evaluation results and sends it to the terminal. The server generates a new learning program that reflects the latest evaluation results and sends it to the terminal. The input data is the latest evaluation results, and the output data is the reconstructed learning program.

[0502] This makes it possible to provide an optimal learning environment that takes into account the user's emotional state, maximizing learning efficiency.

[0503] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0504] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0505] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0506] [Second embodiment]

[0507] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0508] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0509] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0510] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0511] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0512] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0513] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0514] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0515] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0516] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0517] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0518] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0519] The self-contained English skill improvement learning support system of the present invention is implemented as follows: The system is made up of a server, terminals, and users, and each element operates in cooperation with each other.

[0520] 1. Initial setup and level check

[0521] A user accesses the system using their terminal and logs in.

[0522] The server verifies the user's credentials and approves the login.

[0523] The server uses a generative AI model to generate questions for an initial level check, which assess each of the four skills: reading, writing, listening, and speaking.

[0524] The user answers the questions and sends the answers from the terminal to the server.

[0525] The server analyzes the user's response data and evaluates the user's initial level in each skill, thereby understanding the learner's current level.

[0526] 2. Setting goals and creating a learning program

[0527] The user inputs the goal they are aiming for (e.g., "Aiming for 800 points on the TOEIC") and the date by which they will achieve it on their device, and sends it to the server.

[0528] The server generates an optimal learning program based on the user's current level and set goals. This program includes practice content for each skill required to achieve the user's goal.

[0529] For example, it includes problem sets to strengthen reading skills and audio materials to improve listening skills.

[0530] 3. Providing daily learning content and analyzing learning results

[0531] The server generates daily learning content using a generative AI model and sends it to the device.

[0532] Users check the learning content through their devices and work on each assignment.

[0533] For example, "Today's listening questions" and "Today's writing assignments" are presented.

[0534] The user inputs the learning results and answers and sends them from the terminal to the server.

[0535] The server analyzes the user's response data and evaluates the user's learning outcomes for that day. Based on this evaluation, the learning content for the next day is adjusted.

[0536] 4. Regular level checks and program restructuring

[0537] The server periodically generates level check questions and sends them to the terminal.

[0538] The user answers the questions and sends the answer data from the terminal to the server.

[0539] The server analyzes the response data and evaluates the user's current English level.

[0540] The server reconstructs the learning program based on the latest evaluation and sends it to the terminal, allowing the user to always receive a program that meets their latest learning needs.

[0541] Specific examples

[0542] For example, let's consider a case where a user sets a goal of "aiming for 800 points on the TOEIC." In the initial level check, the user's reading ability is assessed as intermediate and their listening ability as beginner. In this case, the server provides intermediate-level reading comprehension questions to strengthen reading ability, while constructing a program that includes many beginner-level audio materials for listening ability.

[0543] In the daily learning session, a beginner-level audio file is provided as a listening test for the day, and the user is asked to answer the questions. When the user submits their answers, the server analyzes them and adjusts the program to provide audio materials with a slightly higher level of difficulty the following day.

[0544] Regular level checks evaluate progress since the last level check and check how much reading and listening skills have improved. Based on the results, a new learning program is set up so that users can continue to study appropriately toward their goals.

[0545] In this way, this system constantly monitors the progress of each individual user and dynamically optimizes their learning content, enabling them to improve their English skills efficiently and effectively.

[0546] The processing flow will be explained below.

[0547] Step 1:

[0548] A user accesses the system using their terminal and logs in.

[0549] The server verifies the user's credentials and approves the login.

[0550] Step 2:

[0551] The server uses a generative AI model to generate questions for an initial level check.

[0552] Specifically, it automatically generates questions to assess the four skills of reading, writing, listening, and speaking.

[0553] Step 3:

[0554] The server sends the generated question to the terminal.

[0555] The user checks the questions on the terminal and enters the answers.

[0556] Step 4:

[0557] The user sends the answer data from the terminal to the server.

[0558] The server receives the user's response data and analyzes it.

[0559] Step 5:

[0560] The server evaluates the user's initial level of the four skills based on the response data.

[0561] Based on the evaluation results, the user's current skill level is recorded.

[0562] Step 6:

[0563] The user uses the terminal to input the goal and the date of achievement into the server and transmit it.

[0564] Specifically, set a goal such as "aiming for 800 points on the TOEIC."

[0565] Step 7:

[0566] The server generates an optimal learning program based on the user's current level and set goals.

[0567] The program includes practice tailored to the skills of reading, listening, writing and speaking.

[0568] Step 8:

[0569] The server transmits the generated learning program to the terminal.

[0570] The user checks and practices the learning program on the device.

[0571] Step 9:

[0572] The server generates daily learning content using a generative AI model and sends it to the device.

[0573] The user works on the day's learning content (for example, listening questions and writing assignments) on the terminal.

[0574] Step 10:

[0575] The user inputs the results of that day's learning into the terminal and sends them to the server.

[0576] The server receives this and analyzes it.

[0577] Step 11:

[0578] The server evaluates the user's progress based on the learning results.

[0579] Based on the evaluation results, the learning content for the next day is adjusted and provided to the user.

[0580] Step 12:

[0581] The server generates questions for regular level checks as needed and sends them to the terminal.

[0582] The user answers this and sends the results to the server.

[0583] Step 13:

[0584] The server analyzes the results of regular level checks and evaluates the user's latest English level.

[0585] The learning program is reconstructed based on the evaluation results and sent to the terminal.

[0586] Step 14:

[0587] The cycle repeats when the user continues learning with a new learning program and logs in again.

[0588] Through these processing steps, the user is always provided with a study plan that is optimized for his or her level and progress, and is able to efficiently improve his or her English skills.

[0589] Example 1

[0590] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0591] Conventional learning support systems have difficulty dynamically and flexibly adjusting learning programs according to the user's learning progress. Furthermore, they lack automation to provide individualized learning programs based on each user's level and goals, making it difficult to effectively improve users' English skills.

[0592] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0593] In this invention, the server includes: means for generating questions to assess a user's level and creating a study program based on the user's answers; means for generating daily study content and providing it to the user; means for analyzing the user's answer data and reconstructing the study program as needed; means for conducting periodic level checks as needed and adjusting the study program based on the evaluation; means for the user to access the server through a terminal, input authentication information, and the server to verify the login against a database; means for the server to generate questions using a generative AI model and present them to the user through the terminal; and means for the server to analyze the user's study results and generate feedback to adjust the study content for the next day, thereby enabling efficient and effective improvement of English skills according to the individual needs of the user.

[0594] "User" refers to a person who uses the learning support system.

[0595] "Terminal" refers to a device such as a computer or smartphone that a user uses to access the system, display learning content, and enter answers.

[0596] A "server" refers to a computer system that manages the entire system and performs various processes in cooperation with a database.

[0597] A "generative AI model" is a model that uses artificial intelligence technology and refers to a program used to generate learning content and analyze responses.

[0598] "Questions" refer to problems and tasks generated by the system to assist users in assessing their level and learning.

[0599] "Study program" means a set of learning activities or tasks designed to engage a user in English language learning.

[0600] "Response data" refers to the response information entered by the user in response to questions or learning content.

[0601] "Evaluation" refers to the process of determining the progress and level of learning based on the user's response data.

[0602] "Feedback" refers to information about areas for improvement and next learning content provided to the user based on the results of the server's analysis.

[0603] "Login" refers to the process by which a user enters authentication information to gain access to a system.

[0604] "Learning content" refers to the specific tasks and learning materials that users must address.

[0605] "Regular level checks" refers to tests or questions administered to periodically assess a user's learning progress.

[0606] The self-contained English skill improvement learning support system of the present invention is implemented as follows: This system is made up of a server, terminals, and users, and each element operates in cooperation with each other.

[0607] The server includes a means for requiring a user to enter authentication information when accessing the system through a terminal and for approving the login by checking this information against a database, thereby allowing the user to access the system. For example, the process involves a user entering an email address and password on a login screen, which is then verified by the server.

[0608] The server provides a means to generate questions using a generative AI model (e.g., GPT-4) to assess the user's level. The generated questions assess the four skills of reading, writing, listening, and speaking individually. This allows for a comprehensive understanding of the user's skill level. For example, intermediate-level written questions are provided for reading, and easy audio questions are provided for listening.

[0609] Users use their devices to answer questions and send the results to the server. The server analyzes the answers and uses NLP technology to assess the user's initial skill level. The server also includes a means to customize a learning program based on the user's set goal (e.g., "800 points on the TOEIC") and the date by which it is achieved. The program includes intermediate-level problem sets to strengthen reading and beginner-level audio materials to improve listening comprehension.

[0610] The server generates daily learning content using a generative AI model and provides a means to send it to the device. For example, "Today's listening questions" and "Today's writing assignments" are presented. The user can check and work on these learning content through the device.

[0611] The user inputs the results of their study into their device and sends them to the server. The server analyzes the user's response data and evaluates the day's study results. The system includes a means for adjusting the study content for the next day based on this evaluation. Specifically, if the user correctly answers a beginner-level listening question, a slightly more difficult question will be presented the next day.

[0612] Additionally, the server provides a means to conduct regular level checks as needed. These checks also use the generative AI model to generate questions and present them to the user. The server then analyzes the user's answers again to assess their current skill level. Based on the results, it reconstructs the learning program and sends it to the device. This ensures that the user's learning plan is always up to date.

[0613] An example of a specific prompt might be, "The user wants to learn intermediate-level reading comprehension questions to improve their reading. Please generate appropriate questions." This prompt is input into a generative AI model, which then generates appropriate reading questions.

[0614] In this way, this system provides a high level of collaboration between the server, terminals, and users, enabling efficient and effective improvement of English skills according to the individual needs of each user.

[0615] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0616] System program processing flow

[0617] Step 1: User Login and Authentication

[0618] Input: The user enters their email address and password into the terminal and presses the login button.

[0619] Action: The device sends the entered authentication information to the server.

[0620] Processing: The server checks the authentication information against the database and authorizes the user to log in.

[0621] Output: Successful login information is sent to the terminal and displayed to the user.

[0622] Step 2: Generate questions for initial level check

[0623] Input: After the server verifies the user's login, it starts the initial level check.

[0624] How it works: The server inputs prompts into the generative AI model, which generates questions for reading, writing, listening, and speaking.

[0625] Processing: The generative AI model generates a question based on the prompt and returns the result to the server.

[0626] Output: The generated questions are sent to the terminal and displayed to the user.

[0627] Step 3: Conduct an initial level check

[0628] Input: The user answers the questions and enters the answer data into the terminal.

[0629] Operation: The device sends the response data to the server.

[0630] Processing: The server analyzes the response data and evaluates the user's initial level for each skill (reading, writing, listening, and speaking).

[0631] Output: Evaluation results and feedback are sent to the device.

[0632] Step 4: Set goals and create a learning program

[0633] Input: The user inputs the goal (e.g., "800 points on the TOEIC") and the deadline for achieving it into the terminal and submits it.

[0634] Operation: The device sends the goal and deadline information to the server.

[0635] Processing: The server generates a learning program based on the user's initial level and goals. In this process, the generative AI model is used again.

[0636] Output: The customized learning program is sent to the terminal and displayed to the user.

[0637] Step 5: Provide daily learning content

[0638] Input: The server generates today's learning content based on the learning program.

[0639] How it works: The generative AI model generates learning content based on the prompt.

[0640] Processing: The server sends the generated learning content to the terminal.

[0641] Output: Today's learning content will be displayed on the terminal.

[0642] Step 6: Input and analysis of training results

[0643] Input: The user works on the learning content and inputs the results into the terminal.

[0644] Operation: The device sends the learning results to the server.

[0645] Processing: The server analyzes the learning results, evaluates the learning outcomes of the day, and generates feedback to adjust the learning content for the next day.

[0646] Output: Analysis results and applied feedback are sent to the device.

[0647] Step 7: Regular level checks and program restructuring

[0648] Input: After a certain period of time has passed, the server generates questions for regular level checks.

[0649] How it works: The regenerative AI model generates a question based on the prompt.

[0650] Processing: The server sends the generated questions to the terminal, and the user answers them. The answer data is sent to the server and analyzed.

[0651] Output: The latest skill level evaluation results are sent to the terminal, and the learning program is reconstructed.

[0652] Through the above processing steps, the system of the present invention efficiently and effectively supports users in improving their English skills in accordance with their individual needs.

[0653] (Application example 1)

[0654] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0655] Improving listening and speaking skills in English learning is a difficult task for many learners. In particular, there is a demand for efficient use of time while riding in an autonomous vehicle to improve English skills. To solve this problem, an effective English learning support system using smart devices is required.

[0656] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0657] In this invention, the server includes a means for generating questions to assess the learner's level and creating a learning program based on the learner's answers, a means for generating daily learning content and providing it to the learner, and a means for analyzing the learner's answer data and reconstructing the learning program as needed, thereby enabling a means for receiving and analyzing voice input using a smart device, a means for dynamically generating questions and answers using a generative AI model, and a means for providing audio and video content to the learner using a smart device.

[0658] "Learner" refers to an individual who uses the System to improve their English language skills.

[0659] "Level assessment questions" are questions that are asked to understand the learner's current level of reading, writing, listening, and speaking skills.

[0660] A "learning program" is a plan that combines the practice and materials necessary for a learner to achieve their desired English skills.

[0661] A "generative AI model" is an algorithm that uses artificial intelligence technology to automatically generate optimal learning content and questions.

[0662] "Smart devices" refer to electronic devices that can connect to the Internet and have voice input and video display functions, and examples include smartphones and smart glasses.

[0663] "Means for receiving and analyzing voice input" refers to the function of collecting learners' speech through the microphone of a smart device and recognizing and analyzing its content.

[0664] "Means for reviewing and dynamically providing learning content" refers to a function that generates and delivers optimal practice questions and teaching materials in real time according to the learner's level and progress.

[0665] "Means for dynamically generating questions and answers" refers to a function that uses a generative AI model to generate appropriate questions and answers in real time according to the learner's level and learning progress.

[0666] "Means for providing audio and video content" refers to a function that supports learning by allowing learners to listen to audio or watch videos using smart devices.

[0667] "Regular level checks" refer to tests conducted at regular intervals to assess learners' progress and determine their current English skill level.

[0668] The English conversation learning support system using smart devices of the present invention is implemented as follows: This system is made up of a server, a terminal, and a user, and each element works in cooperation with each other.

[0669] 1. Initial setup and level check

[0670] The user accesses the system using their own terminal (smart device) and logs in.

[0671] The server verifies the user's credentials and approves the login.

[0672] The server uses a generative AI model to generate questions for an initial level check, which assess each of the four skills: reading, writing, listening, and speaking.

[0673] The user answers the questions and sends the answers from the terminal to the server.

[0674] The server analyzes the user's response data and evaluates the user's initial level in each skill, thereby understanding the learner's current level.

[0675] 2. Setting goals and creating a learning program

[0676] The user inputs the goal (e.g., "to acquire English conversation skills at a daily conversation level") and the date by which it will be achieved from the terminal and sends it to the server.

[0677] The server generates an optimal learning program based on the user's current level and set goals. This program includes problem sets and audio materials to improve listening and speaking skills.

[0678] 3. Providing daily learning content and analyzing learning results

[0679] The server generates daily learning content using a generative AI model and sends it to the device.

[0680] The user checks the learning content through the device and works on each task. For example, "Today's listening questions" and "Today's speaking tasks" are presented.

[0681] The user responds by voice input, and the voice data is sent from the terminal to the server.

[0682] The server analyzes the audio data and evaluates the learning outcomes for that day, and the learning content for the next day is adjusted based on this evaluation.

[0683] 4. Regular level checks and program restructuring

[0684] The server periodically generates level check questions and sends them to the terminal.

[0685] The user answers the questions and sends the answer data from the terminal to the server.

[0686] The server analyzes the response data and evaluates the user's current English level.

[0687] The server reconstructs the learning program based on the latest evaluation and sends it to the terminal, allowing the user to always receive a program that meets their latest learning needs.

[0688] Hardware and software used

[0689] Hardware: Smart devices (smartphones, smart glasses, etc.)

[0690] Software: Generative AI models, speech recognition libraries (e.g., speech_recognition)

[0691] The system uses a generative AI model to dynamically generate questions and collects and analyzes user speech data through speech recognition, allowing users to efficiently improve their English conversation skills.

[0692] Specific examples

[0693] When a user uses this system to learn English conversation while riding in an autonomous vehicle, the following prompt sentences are generated:

[0694] "Please generate a list of English conversation phrases suitable for a beginner level. These phrases should be commonly used in travel situations."

[0695] For example, a passenger repeats the following phrase presented as "Today's Listening Question."

[0696] "How can I get to the nearest train station?"

[0697] Once the passenger has spoken correctly, their speech is analyzed and the next phrase is presented.

[0698] "Thank you! Let's move on to the next phrase."

[0699] In this way, you can efficiently improve your English conversation skills while managing your progress.

[0700] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0701] Step 1:

[0702] Initial setup and level check

[0703] When a user accesses the system using their own device, the server verifies the user's authentication information and approves the login. Next, it uses the generative AI model to generate questions for an initial level check and sends them to the device. The user answers the questions and sends the answers from the device to the server. The server then analyzes the user's answer data and evaluates the user's initial level in each skill. In this process, the user's answers are the input data, the generative AI model generates questions, and the server analyzes the data to evaluate the learner's level.

[0704] Step 2:

[0705] Setting goals and creating learning programs

[0706] The user inputs their desired goal (e.g., "Acquire English conversation skills at a daily conversation level") and the date by which they will achieve this goal from their device and sends it to the server. The server generates an optimal learning program based on the user's current level and the set goal. Specifically, it uses a generative AI model to dynamically create problem sets and audio materials to improve listening and speaking skills. In this process, the user's goal and current level are input data, and the generative AI model is used to generate a learning program, which outputs an optimal learning plan.

[0707] Step 3:

[0708] Providing daily learning content and analyzing learning results

[0709] The server generates daily learning content using a generative AI model and sends it to the device. The user checks the learning content through the device and works on each assignment. The user answers through voice input, and the voice data is sent from the device to the server. The server analyzes the voice data and evaluates the learning outcome for that day. Based on this evaluation, the learning content for the next day is adjusted. In this process, the user's voice data is the input data, and the server analyzes and evaluates the learning outcome, adjusting the learning content for the next day to obtain the output.

[0710] Step 4:

[0711] Regular level checks and program restructuring

[0712] The server periodically generates level-check questions and sends them to the terminal. The user answers the questions and sends the answer data from the terminal to the server. The server analyzes the answer data and evaluates the user's latest English level. The server reconstructs a learning program based on the latest evaluation and sends it to the terminal. This allows the user to always receive a program that meets their latest learning needs. In this process, the user's answer data is the input data, and the server analyzes it to evaluate the user's latest English level, then generates and outputs a new learning program.

[0713] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0714] The self-contained English skill improvement learning support system of this invention achieves a higher level of personalization by combining a conventional learning support system with an emotion engine that recognizes the user's emotions. The system is composed of a server, terminals, and users, and each element works in cooperation with each other.

[0715] 1. Initial setup and level check

[0716] A user accesses the system using their terminal and logs in.

[0717] The server verifies the user's credentials and approves the login.

[0718] The server uses a generative AI model to generate questions for an initial level check, which automatically generate questions to assess the four skills of reading, writing, listening, and speaking.

[0719] The user answers the questions and sends the answers from the terminal to the server.

[0720] The server analyzes the user's response data and evaluates the user's initial level in each skill, thereby understanding the learner's current level.

[0721] 2. Setting goals and creating a learning program

[0722] The user inputs the goal they are aiming for (e.g., "Aiming for 800 points on the TOEIC") and the date by which they will achieve it on their device, and sends it to the server.

[0723] The server generates an optimal learning program based on the user's current level and set goals. This program includes practice content for each skill required to achieve the user's goal.

[0724] For example, it includes problem sets to strengthen reading skills and audio materials to improve listening skills.

[0725] 3. Providing daily learning content and analyzing learning results

[0726] The server generates daily learning content using a generative AI model and sends it to the device.

[0727] Users check the learning content through their devices and work on each assignment.

[0728] For example, "Today's listening questions" and "Today's writing assignments" are presented.

[0729] The user inputs the learning results and answers and sends them from the terminal to the server.

[0730] The server receives this and analyzes it.

[0731] 4. Implementation and Use of Emotion Engine

[0732] The server uses an emotion engine to recognize the user's emotions in real time, specifically assessing the user's current emotions (e.g., stress, concentration, fatigue, etc.) through facial recognition technology and voice analysis.

[0733] The server then adjusts learning content and break suggestions based on the recognized emotion data. For example, if the user feels tired, a pop-up will appear recommending a break.

[0734] The user accepts the sentiment-based suggestions and chooses whether to pause or continue learning.

[0735] 5. Regular level checks and program restructuring

[0736] The server periodically generates level check questions and sends them to the terminal.

[0737] The user answers the questions and sends the answer data from the terminal to the server.

[0738] The server analyzes the response data and evaluates the user's current English level.

[0739] The server reconstructs the learning program based on the latest evaluation and sends it to the terminal, allowing the user to always receive a program that meets their latest learning needs.

[0740] Specific examples

[0741] For example, consider a case where a user sets a goal of "aiming for 800 points on the TOEIC." In the initial level check, the user's reading ability is assessed as intermediate and their listening ability as beginner. In this case, the server provides intermediate-level reading comprehension questions to strengthen reading ability, while constructing a program that includes many beginner-level audio materials for listening ability.

[0742] During daily learning, the emotion engine detects the user's stress level, and if high stress is detected, the server reduces the learning content and switches to more relaxing content. If the user feels tired, a notification is displayed recommending a break.

[0743] Regular level checks evaluate progress since the last level check and check how much reading and listening skills have improved. Based on the results, a new learning program is set up so that users can continue to study appropriately toward their goals.

[0744] In this way, the system constantly monitors the user's progress and emotional state, dynamically optimizing learning content to improve English skills efficiently and effectively.

[0745] The processing flow will be explained below.

[0746] Step 1:

[0747] A user accesses the system using their terminal and logs in.

[0748] The server verifies the user's credentials and approves the login.

[0749] Step 2:

[0750] The server uses a generative AI model to generate questions for an initial level check.

[0751] Specifically, it automatically generates questions to assess the four skills of reading, writing, listening, and speaking.

[0752] Step 3:

[0753] The server sends the generated question to the terminal.

[0754] The user checks the questions on the terminal and enters the answers.

[0755] Step 4:

[0756] The user sends the answer data from the terminal to the server.

[0757] The server receives the user's response data and analyzes it.

[0758] Step 5:

[0759] The server evaluates the user's initial level of the four skills based on the response data.

[0760] Based on the evaluation results, the user's current skill level is recorded.

[0761] Step 6:

[0762] The user uses the terminal to input the goal and the date of achievement into the server and transmit it.

[0763] Specifically, set a goal such as "aiming for 800 points on the TOEIC."

[0764] Step 7:

[0765] The server generates an optimal learning program based on the user's current level and set goals.

[0766] The program includes practice tailored to the skills of reading, listening, writing and speaking.

[0767] Step 8:

[0768] The server transmits the generated learning program to the terminal.

[0769] The user checks and practices the learning program on the device.

[0770] Step 9:

[0771] The server generates daily learning content using a generative AI model and sends it to the device.

[0772] The user works on the day's learning content (for example, listening questions and writing assignments) on the terminal.

[0773] Step 10:

[0774] The user inputs the results of that day's learning into the terminal and sends them to the server.

[0775] The server receives this and analyzes it.

[0776] Step 11:

[0777] The server evaluates the user's progress based on the learning results.

[0778] Based on the evaluation results, the learning content for the next day is adjusted and provided to the user.

[0779] Step 12:

[0780] The server uses an emotion engine to recognize the user's emotions in real time.

[0781] Specifically, it assesses the user's current emotions (e.g., stress, concentration, fatigue, etc.) through facial recognition technology and voice analysis.

[0782] Step 13:

[0783] The server adjusts learning content and break suggestions based on the recognized emotion data.

[0784] For example, if stress levels are high, the system will provide less difficult questions and switch to content that has a relaxing effect.

[0785] Step 14:

[0786] The user chooses whether to accept sentiment-based suggestions.

[0787] If the user feels they need a break, they can pause their learning and refresh.

[0788] Step 15:

[0789] The server generates questions for regular level checks as needed and sends them to the terminal.

[0790] The user answers this and sends the results to the server.

[0791] Step 16:

[0792] The server analyzes the results of regular level checks and evaluates the user's latest English level.

[0793] The learning program is reconstructed based on the evaluation results and sent to the terminal.

[0794] Step 17:

[0795] The user continues learning with a new learning program.

[0796] When you log in again, the cycle repeats.

[0797] Through these processing steps, users are always provided with a learning plan that is optimized for their level, progress, and emotional state, allowing them to efficiently improve their English skills.

[0798] Example 2

[0799] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0800] Conventional learning support systems provide learning programs tailored to a learner's current level and goals, but they are unable to recognize the learner's emotional state and adjust the learning content based on that emotion. This creates the problem that appropriate support is not provided even when the learner feels fatigued or stressed, making efficient learning difficult. In addition, rebuilding learning programs and conducting regular level checks is cumbersome, creating a need for automatic dynamic adjustments.

[0801] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0802] In this invention, the server includes means for generating questions to assess the learner's level and creating a learning program based on the learner's answers, means for generating daily learning content and providing it to the learner, means for recognizing the learner's emotional state in real time and adjusting the learning content and break suggestions, means for analyzing the learner's answer data and reconstructing the learning program as appropriate, and means for conducting regular level checks as necessary and adjusting the learning program based on the evaluation, thereby enabling efficient and personalized learning support that takes the learner's emotional state into consideration.

[0803] The term "learner" refers to an individual who uses the learning system for the purpose of improving a particular skill (mainly language skill in the present invention).

[0804] A "server" refers to a computer system that provides various services to learners via a network, and in the present invention, it mainly generates and provides learning programs.

[0805] "Question" refers to a question or problem generated by the server to assess a learner's skill or level.

[0806] "Learning program" refers to a set of educational curriculum and assignments created by the server based on the learner's current level and goals.

[0807] "Learning content" refers to specific learning tasks and learning materials that are generated by the server daily and provided to learners.

[0808] "Response data" refers to the responses entered by learners to questions and assignments.

[0809] "Emotional state" refers to the learner's current psychological or emotional state (e.g., stress, concentration, fatigue).

[0810] An "emotion engine" refers to a system that uses facial recognition technology, voice analysis technology, and other techniques to recognize a learner's emotional state in real time.

[0811] "Dynamic adjustment" refers to changing and optimizing learning programs and content in real time based on the learner's progress and emotional state.

[0812] "Level check" refers to a test or assessment to assess a learner's current skill level.

[0813] This invention relates to a self-contained English skill improvement learning support system that assesses learners' levels and creates and provides learning programs. The system consists of a server, terminals, and learners, and each element works in conjunction with the others. The server uses a generative AI model to generate questions and learning content, and an emotion engine to recognize the learner's emotional state and dynamically adjust the learning content.

[0814] Initial setup and level check

[0815] The learner accesses the system using a device and logs in. The server verifies the learner's authentication information and approves the login. The server uses a generative AI model to generate questions to assess the four skills of reading, writing, listening, and speaking. The learner answers the questions and sends the data from their device to the server. The server analyzes the response data and evaluates the initial level of each skill to determine the learner's current skill level.

[0816] Setting goals and creating learning programs

[0817] Learners input their goal (e.g., "Aim for 800 points on the TOEIC") and the target date for achieving it on their device and send it to the server. The server then generates an optimal learning program based on the learner's current level and the set goal. This program might include, for example, a problem set to improve reading skills or audio materials to improve listening ability.

[0818] Providing daily learning content and analyzing learning results

[0819] The server uses a generative AI model to generate daily learning content and sends it to the device. The learner checks the learning content through the device and works on each task. For example, they are presented with a "today's listening question" or a "today's writing task." The learner inputs their learning results and answers and sends them from their device to the server. The server analyzes the received data and evaluates the learner's progress.

[0820] Implementing and utilizing an emotion engine

[0821] The server uses an emotion engine to recognize the learner's emotional state in real time. Specifically, it uses facial recognition technology and voice analysis to assess the learner's current emotions (e.g., stress, concentration, fatigue, etc.). The server adjusts learning content and break suggestions based on the emotion data. For example, if the learner feels fatigued, a pop-up recommending a break will be displayed. The learner can accept the emotion-based suggestion and choose whether to pause or continue learning.

[0822] Regular level checks and program restructuring

[0823] The server periodically generates level-check questions and sends them to the device. The learner answers the questions and sends the answer data from the device to the server. The server analyzes the data and evaluates the learner's current English level. Based on this evaluation, the server reconstructs the learning program and sends it to the device. This ensures that the learner always receives a program that meets their latest learning needs.

[0824] Specific examples and examples of prompts for the generative AI model

[0825] For example, consider a case where a learner sets a goal of "achieving 800 points on the TOEIC." Suppose that the initial level check assesses reading as intermediate and listening as beginner. In this case, the server provides intermediate-level reading comprehension questions to strengthen reading, while constructing a program that includes many beginner-level audio materials for listening. During daily learning, if the emotion engine detects high stress, the server reduces the learning content and switches to content with a relaxing effect. Furthermore, if the learner feels tired, a notification is displayed recommending a break.

[0826] Example prompts for generative AI models

[0827] "A student is aiming for 800 points on the TOEIC, and their current reading level is intermediate and their listening level is beginner. Please generate daily study content based on the student's progress."

[0828] In this way, the system constantly monitors learners' progress and emotional state, dynamically optimizing learning content to help them improve their English skills efficiently and effectively.

[0829] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0830] Step 1:

[0831] A user accesses the system using their terminal and logs in.

[0832] Input: User ID and password.

[0833] How it works: The user opens a browser or a dedicated app, accesses the login screen, and enters their user ID and password.

[0834] Output: The entered authentication information is sent to the server.

[0835] Step 2:

[0836] The server verifies the user's credentials and approves the login.

[0837] Input: User ID and password.

[0838] How it works: The server accesses the database and checks the entered credentials.

[0839] Output: If authentication is successful, the login is approved and the user is shown the dashboard.

[0840] Step 3:

[0841] The server uses a generative AI model to generate questions for an initial level check.

[0842] Input: User login information.

[0843] How it works: The server sends prompts to a generative AI model to generate questions to assess the four skills of reading, writing, listening, and speaking.

[0844] Output: The generated questions are sent to the user's terminal.

[0845] Step 4:

[0846] The user answers the questions and sends the answers from the terminal to the server.

[0847] Input: The generated question.

[0848] How it works: The user reviews the questions on the terminal and enters answers to each question.

[0849] Output: The answer data is sent from the device to the server.

[0850] Step 5:

[0851] The server analyzes the user's response data and evaluates the user's initial level in each skill.

[0852] Input: User response data.

[0853] How it works: The server uses natural language processing (NLP) technology to analyze the response data and assess the initial level of each skill (reading, writing, listening, and speaking).

[0854] Output: User's initial level assessment result.

[0855] Step 6:

[0856] The user inputs the goal and the date of achievement from the terminal and transmits it to the server.

[0857] Input: Your goal and the date you want to reach it.

[0858] How it works: The user enters the goal and target achievement date into a dedicated input form and presses the submit button.

[0859] Output: The goal and the target date are sent to the server.

[0860] Step 7:

[0861] The server generates an optimal learning program based on the user's current level and set goals.

[0862] Input: User's initial level assessment results and goal / target achievement date.

[0863] How it works: The server sends prompts to the generative AI model to generate a learning program appropriate for the user's current level and goals.

[0864] Output: The generated learning program is sent to the user's terminal.

[0865] Step 8:

[0866] The server generates daily learning content using a generative AI model and sends it to the device.

[0867] Input: The user's study program.

[0868] How it works: The server sends prompts to the generative AI model to generate learnings for the day.

[0869] Output: The generated learning content is sent to the user's device.

[0870] Step 9:

[0871] Users check the learning content through their devices and work on each assignment.

[0872] Input: What you learned that day.

[0873] Action: The user checks the learning content on the device and works on the assignments. For example, they play an audio file and answer listening questions.

[0874] Output: Learning results and answer data.

[0875] Step 10:

[0876] The user inputs the learning results and answers and sends them from the terminal to the server.

[0877] Input: Learning results and answer data.

[0878] Operation: The user enters the learning results and answers and presses the submit button.

[0879] Output: Results and response data are sent to the server.

[0880] Step 11:

[0881] The server uses an emotion engine to recognize the learner's emotional state in real time.

[0882] Input: Learner's video and audio data.

[0883] Operation: The server runs the emotion engine to assess the learner's current emotional state through facial recognition technology and voice analysis.

[0884] Output: Emotional state assessment results.

[0885] Step 12:

[0886] The server adjusts learning content and break suggestions based on the recognized emotion data.

[0887] Input: Emotional state assessment results.

[0888] How it works: The server adjusts the difficulty of the learning content and suggests breaks based on the user's emotional state. For example, if the user is highly fatigued, a pop-up will appear recommending a break.

[0889] Output: Tailored learning content and break suggestions.

[0890] Step 13:

[0891] The user chooses whether to accept sentiment-based suggestions.

[0892] Input: Study content and break suggestions.

[0893] Action: The user reviews the suggestions and selects "Take a break" or "Continue learning."

[0894] Output: Next action based on user selection.

[0895] Step 14:

[0896] The server periodically generates level check questions and sends them to the terminal.

[0897] Input: Learner progress data.

[0898] How it works: The server uses the generative AI model to generate questions for regular level checks.

[0899] Output: Generated level check questions.

[0900] Step 15:

[0901] The user answers the questions and sends the answer data from the terminal to the server.

[0902] Input: Questions for regular level checks.

[0903] Operation: The user answers the questions and sends the answer data from the terminal to the server.

[0904] Output: The answer data is sent to the server.

[0905] Step 16:

[0906] The server analyzes the response data and evaluates the user's current English level.

[0907] Input: Response data.

[0908] How it works: The server analyzes the response data and evaluates your current English level.

[0909] Output: Latest English level assessment results.

[0910] Step 17:

[0911] The server reconstructs the learning program based on the latest evaluation and sends it to the device.

[0912] Input: Your most recent English level assessment results.

[0913] How it works: The server uses the generative AI model to reconstruct a learning program based on the latest English level.

[0914] Output: A reconstructed learning program.

[0915] (Application example 2)

[0916] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0917] Conventional learning support systems provide learning content mechanically without considering the learner's emotional state, resulting in problems such as reduced learning efficiency due to learner fatigue or stress. While these systems dynamically adjust learning programs based on the learner's progress and ability, they do not monitor the learner's emotional state, resulting in an insufficient optimization of the individual learning experience. In particular, in work environments using robots, such as factories, a system is needed that can optimize work efficiency according to the worker's emotional state.

[0918] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: a means for generating questions to evaluate the learner's level and creating a learning program based on the learner's answers; a means for generating daily learning content and providing it to the learner; a means for analyzing the learner's answer data and reconstructing the learning program as needed; a means for conducting periodic level checks as needed and adjusting the learning program based on the evaluation; and a means for recognizing the learner's emotional state in real time and dynamically optimizing the learning content based on the learner's emotional state. This makes it possible to provide a learning program that takes the learner's emotional state into consideration, thereby reducing learner stress and fatigue and improving learning efficiency. Furthermore, by applying this to factory work sites, it is possible to optimize work schedules and robot operations according to the worker's emotional state.

[0919] "Assessing the learner's level" means measuring the level of knowledge and skills that the learner currently possesses and providing appropriate learning content and programs based on that.

[0920] "Question generation" means automatically creating questions or problems to assess a learner's ability.

[0921] "Creating a learning program" means planning and designing learning content and activities based on the learner's goals and current abilities.

[0922] "Generating daily learning content" means automatically creating and providing learning tasks and materials that learners should tackle every day.

[0923] "Providing to learners" means displaying and transmitting learning content and assignments to the devices and systems used by learners.

[0924] "Analyzing learner response data" means analyzing the responses and results submitted by learners and evaluating their learning progress and level of understanding based on the results.

[0925] "Reconstructing learning programs in a timely manner" means dynamically updating and redesigning learning content and programs according to learners' progress and assessment results.

[0926] "Conducting regular level checks" means administering tests and questions to assess learners' abilities on a regular basis.

[0927] "Recognizing the learner's emotional state in real time" means instantly assessing the learner's current emotional and psychological state using information such as their facial expressions and voice.

[0928] "Dynamic optimization of learning content based on emotional state" means flexibly adjusting the learning content and difficulty level by reflecting the learner's emotional and psychological state.

[0929] "Facial recognition technology" is a technology that detects an individual's face from camera footage and analyzes their facial expressions and characteristics.

[0930] "Voice analysis technology" is a technology that analyzes voice data and evaluates its content and the speaker's emotional state.

[0931] "Detecting fatigue and stress" means determining the level of fatigue and stress experienced by learners through facial recognition and voice analysis.

[0932] "Displaying a notification recommending a break" means that when the system detects that the learner is fatigued or stressed, a message urging the learner to take a break will be displayed on the device.

[0933] The present invention provides a learning support system that recognizes the emotional state of a learner in real time and dynamically optimizes the generation and provision of learning programs. The system is composed of a server, terminals, and users.

[0934] 1. Initial setup and level check

[0935] A user accesses the system using a terminal and logs in. The server verifies the user's authentication information and approves the login. The server then uses a generative AI model to generate questions for an initial level check. These questions are designed to assess the four skills of reading, writing, listening, and speaking. The user answers the questions and sends the answers from the terminal to the server. The server analyzes the user's response data and evaluates the initial level in each skill.

[0936] 2. Setting goals and creating a learning program

[0937] The user inputs their goal (for example, "Aiming for 800 points on the TOEIC") and the date they wish to achieve it, and sends it to the server. The server then generates an optimal learning program based on the user's current level and the goal they set. This program includes practice content for each skill the user needs to achieve their goal.

[0938] 3. Providing daily learning content and analyzing learning results

[0939] The server uses a generative AI model to generate daily learning content and sends it to the device. The user checks the learning content through the device and works on each task. The device then sends the learning results and answers to the server, which receives and analyzes them.

[0940] 4. Implementation and Use of Emotion Engine

[0941] The server uses an emotion engine to recognize the user's emotions in real time. The hardware used includes a camera and microphone. The software uses OpenCV for facial recognition technology and TensorFlow / Keras for emotion recognition models. Using these technologies, the server evaluates the user's current emotions (e.g., stress, concentration, fatigue, etc.). Based on the recognized emotion data, the server adjusts the learning content and suggests breaks. For example, if the server recognizes that the user is feeling fatigued, it will display a notification on the device recommending a break.

[0942] 5. Regular level checks and program restructuring

[0943] The server periodically generates level-check questions and sends them to the device. The user answers the questions and sends the answer data from the device to the server. The server analyzes the answer data and evaluates the user's latest English level. Based on the latest evaluation, the server reconstructs the learning program and sends it to the device. This ensures that the user always receives a program that meets their latest learning needs.

[0944] Specific examples

[0945] For example, consider a user who sets a goal of "achieving 800 points on the TOEIC." Suppose the user's reading and listening skills are assessed as intermediate and beginner levels in the initial level check. In this case, the server provides intermediate-level reading comprehension questions to strengthen reading, while constructing a program that includes many beginner-level audio materials for listening. During daily study, the emotion engine detects the user's stress level. If high stress is recognized, the server reduces the study content and switches to content with a relaxing effect. If the user feels tired, the server displays a notification recommending a break. Periodic level checks evaluate progress since the previous level check and confirm the degree of improvement in reading and listening skills. Based on the results, a new study program is created, allowing the user to continue studying appropriately toward their goal.

[0946] Prompt Sentence Examples

[0947] "Please explain how to recognize the emotions of factory workers in real time and optimize labor efficiency based on their emotions. I would like to build a system that sends emotional data to a server, which then adjusts optimal work schedules and robot operations. As an example, please explain in detail how you combine facial recognition technology with an emotion engine."

[0948] This allows learners' progress and emotional state to be constantly monitored, and learning content to be dynamically optimized, enabling them to improve their English skills efficiently and effectively.

[0949] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0950] Step 1:

[0951] A user accesses the system using a terminal and logs in. The user enters login information (user ID and password), which the terminal sends to the server. The server verifies the user's authentication information and approves the login. The input data is the user's authentication information, and the output data is the login success or failure status.

[0952] Step 2:

[0953] The server uses a generative AI model to generate questions for an initial level check. The server uses prompts to request questions to assess the skills of reading, writing, listening, and speaking from the generative AI model, and sends the generated questions to the terminal. The input data is the prompts, and the output data is the generated questions.

[0954] Step 3:

[0955] The user answers questions through the terminal and sends the answers to the server. The terminal acquires the user's input and sends the data to the server. The input data is the user's answer, and the output data is a notification that the answer data has been sent.

[0956] Step 4:

[0957] The server analyzes the user's response data and evaluates their initial level. The server then inputs the response data into an analysis algorithm to calculate a score for each skill. The input data is the user's response data, and the output data is the evaluation result of the initial level for each skill.

[0958] Step 5:

[0959] The user inputs the goal they are aiming for and the date by which they will reach it, and sends it to the server. The user inputs the goal they have set (for example, "Aim for 800 points on the TOEIC") and the date by which they plan to reach it from their terminal, and sends it to the server. The input data is the goal and the date by which it will be reached, and the output data is a notification that the goal has been set.

[0960] Step 6:

[0961] The server generates an optimal learning program based on the user's current level and set goals. Based on the initial level assessment results and goals, the server requests the AI ​​model to generate learning content suitable for the user, and creates that content. The input data is the initial level assessment results and the user's goals, and the output data is the generated learning program.

[0962] Step 7:

[0963] The server generates daily learning content using a generative AI model and sends it to the device. The server dynamically adjusts the daily learning content taking into account the user's progress and emotional state and sends it to the device. The input data is the user's progress and emotional state, and the output data is the generated daily learning program.

[0964] Step 8:

[0965] The user works on daily learning content through the terminal and inputs the results. The user performs the learning tasks displayed on the terminal and inputs the answers. The input data is the user's learning results, and the output data is a notification that the learning results have been sent.

[0966] Step 9:

[0967] The server receives and analyzes the learning result data. The server analyzes the received learning results and evaluates the progress. The input data is the learning result data, and the output data is the progress evaluation result.

[0968] Step 10:

[0969] The server uses an emotion engine to recognize the user's emotions in real time. The server then activates the emotion engine and analyzes the camera video and audio data to evaluate the user's emotional state. The input data are the camera video and audio data, and the output data are the emotion evaluation results.

[0970] Step 11:

[0971] The server adjusts the learning content and break suggestions based on the emotion evaluation results. The server dynamically adjusts the learning content based on the emotion evaluation results, and in some cases displays a notification on the device recommending a break. The input data is the emotion evaluation results, and the output data is the adjusted learning content and break suggestions.

[0972] Step 12:

[0973] The server periodically generates level check questions and sends them to the terminal. The server also generates questions to reassess the user's skills at regular intervals and sends them to the terminal. The input data is the prompt text, and the output data is the generated level check questions.

[0974] Step 13:

[0975] The user answers questions for the regular level check and sends the results to the server. The user answers questions and sends the data to the server. The input data is the answers to the regular level check, and the output data is a notification that the answer data has been sent.

[0976] Step 14:

[0977] The server analyzes the response data and evaluates the user's latest English level. The server analyzes the response data and evaluates the user's progress. The input data is the response data from the regular level check, and the output data is the latest English level evaluation result.

[0978] Step 15:

[0979] The server reconstructs the learning program based on the latest evaluation results and sends it to the terminal. The server generates a new learning program that reflects the latest evaluation results and sends it to the terminal. The input data is the latest evaluation results, and the output data is the reconstructed learning program.

[0980] This makes it possible to provide an optimal learning environment that takes into account the user's emotional state, maximizing learning efficiency.

[0981] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0982] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0983] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0984] [Third embodiment]

[0985] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0986] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0987] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0988] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0989] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0990] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0991] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0992] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0993] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0994] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0995] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0996] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0997] The self-contained English skill improvement learning support system of the present invention is implemented as follows: The system is made up of a server, terminals, and users, and each element operates in cooperation with each other.

[0998] 1. Initial setup and level check

[0999] A user accesses the system using their terminal and logs in.

[1000] The server verifies the user's credentials and approves the login.

[1001] The server uses a generative AI model to generate questions for an initial level check, which assess each of the four skills: reading, writing, listening, and speaking.

[1002] The user answers the questions and sends the answers from the terminal to the server.

[1003] The server analyzes the user's response data and evaluates the user's initial level in each skill, thereby understanding the learner's current level.

[1004] 2. Setting goals and creating a learning program

[1005] The user inputs the goal they are aiming for (e.g., "Aiming for 800 points on the TOEIC") and the date by which they will achieve it on their device, and sends it to the server.

[1006] The server generates an optimal learning program based on the user's current level and set goals. This program includes practice content for each skill required to achieve the user's goal.

[1007] For example, it includes problem sets to strengthen reading skills and audio materials to improve listening skills.

[1008] 3. Providing daily learning content and analyzing learning results

[1009] The server generates daily learning content using a generative AI model and sends it to the device.

[1010] Users check the learning content through their devices and work on each assignment.

[1011] For example, "Today's listening questions" and "Today's writing assignments" are presented.

[1012] The user inputs the learning results and answers and sends them from the terminal to the server.

[1013] The server analyzes the user's response data and evaluates the user's learning outcomes for that day. Based on this evaluation, the learning content for the next day is adjusted.

[1014] 4. Regular level checks and program restructuring

[1015] The server periodically generates level check questions and sends them to the terminal.

[1016] The user answers the questions and sends the answer data from the terminal to the server.

[1017] The server analyzes the response data and evaluates the user's current English level.

[1018] The server reconstructs the learning program based on the latest evaluation and sends it to the terminal, allowing the user to always receive a program that meets their latest learning needs.

[1019] Specific examples

[1020] For example, let's consider a case where a user sets a goal of "aiming for 800 points on the TOEIC." In the initial level check, the user's reading ability is assessed as intermediate and their listening ability as beginner. In this case, the server provides intermediate-level reading comprehension questions to strengthen reading ability, while constructing a program that includes many beginner-level audio materials for listening ability.

[1021] In the daily learning session, a beginner-level audio file is provided as a listening test for the day, and the user is asked to answer the questions. When the user submits their answers, the server analyzes them and adjusts the program to provide audio materials with a slightly higher level of difficulty the following day.

[1022] Regular level checks evaluate progress since the last level check and check how much reading and listening skills have improved. Based on the results, a new learning program is set up so that users can continue to study appropriately toward their goals.

[1023] In this way, this system constantly monitors the progress of each individual user and dynamically optimizes their learning content, enabling them to improve their English skills efficiently and effectively.

[1024] The processing flow will be explained below.

[1025] Step 1:

[1026] A user accesses the system using their terminal and logs in.

[1027] The server verifies the user's credentials and approves the login.

[1028] Step 2:

[1029] The server uses a generative AI model to generate questions for an initial level check.

[1030] Specifically, it automatically generates questions to assess the four skills of reading, writing, listening, and speaking.

[1031] Step 3:

[1032] The server sends the generated question to the terminal.

[1033] The user checks the questions on the terminal and enters the answers.

[1034] Step 4:

[1035] The user sends the answer data from the terminal to the server.

[1036] The server receives the user's response data and analyzes it.

[1037] Step 5:

[1038] The server evaluates the user's initial level of the four skills based on the response data.

[1039] Based on the evaluation results, the user's current skill level is recorded.

[1040] Step 6:

[1041] The user uses the terminal to input the goal and the date of achievement into the server and transmit it.

[1042] Specifically, set a goal such as "aiming for 800 points on the TOEIC."

[1043] Step 7:

[1044] The server generates an optimal learning program based on the user's current level and set goals.

[1045] The program includes practice tailored to the skills of reading, listening, writing and speaking.

[1046] Step 8:

[1047] The server transmits the generated learning program to the terminal.

[1048] The user checks and practices the learning program on the device.

[1049] Step 9:

[1050] The server generates daily learning content using a generative AI model and sends it to the device.

[1051] The user works on the day's learning content (for example, listening questions and writing assignments) on the terminal.

[1052] Step 10:

[1053] The user inputs the results of that day's learning into the terminal and sends them to the server.

[1054] The server receives this and analyzes it.

[1055] Step 11:

[1056] The server evaluates the user's progress based on the learning results.

[1057] Based on the evaluation results, the learning content for the next day is adjusted and provided to the user.

[1058] Step 12:

[1059] The server generates questions for regular level checks as needed and sends them to the terminal.

[1060] The user answers this and sends the results to the server.

[1061] Step 13:

[1062] The server analyzes the results of regular level checks and evaluates the user's latest English level.

[1063] The learning program is reconstructed based on the evaluation results and sent to the terminal.

[1064] Step 14:

[1065] The cycle repeats when the user continues learning with a new learning program and logs in again.

[1066] Through these processing steps, the user is always provided with a study plan that is optimized for his or her level and progress, and is able to efficiently improve his or her English skills.

[1067] Example 1

[1068] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1069] Conventional learning support systems have difficulty dynamically and flexibly adjusting learning programs according to the user's learning progress. Furthermore, they lack automation to provide individualized learning programs based on each user's level and goals, making it difficult to effectively improve users' English skills.

[1070] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1071] In this invention, the server includes: means for generating questions to assess a user's level and creating a study program based on the user's answers; means for generating daily study content and providing it to the user; means for analyzing the user's answer data and reconstructing the study program as needed; means for conducting periodic level checks as needed and adjusting the study program based on the evaluation; means for the user to access the server through a terminal, input authentication information, and the server to verify the login against a database; means for the server to generate questions using a generative AI model and present them to the user through the terminal; and means for the server to analyze the user's study results and generate feedback to adjust the study content for the next day, thereby enabling efficient and effective improvement of English skills according to the individual needs of the user.

[1072] "User" refers to a person who uses the learning support system.

[1073] "Terminal" refers to a device such as a computer or smartphone that a user uses to access the system, display learning content, and enter answers.

[1074] A "server" refers to a computer system that manages the entire system and performs various processes in cooperation with a database.

[1075] A "generative AI model" is a model that uses artificial intelligence technology and refers to a program used to generate learning content and analyze responses.

[1076] "Questions" refer to problems and tasks generated by the system to assist users in assessing their level and learning.

[1077] "Study program" means a set of learning activities or tasks designed to engage a user in English language learning.

[1078] "Response data" refers to the response information entered by the user in response to questions or learning content.

[1079] "Evaluation" refers to the process of determining the progress and level of learning based on the user's response data.

[1080] "Feedback" refers to information about areas for improvement and next learning content provided to the user based on the results of the server's analysis.

[1081] "Login" refers to the process by which a user enters authentication information to gain access to a system.

[1082] "Learning content" refers to the specific tasks and learning materials that users must address.

[1083] "Regular level checks" refers to tests or questions administered to periodically assess a user's learning progress.

[1084] The self-contained English skill improvement learning support system of the present invention is implemented as follows: This system is made up of a server, terminals, and users, and each element operates in cooperation with each other.

[1085] The server includes a means for requiring a user to enter authentication information when accessing the system through a terminal and for approving the login by checking this information against a database, thereby allowing the user to access the system. For example, the process involves a user entering an email address and password on a login screen, which is then verified by the server.

[1086] The server provides a means to generate questions using a generative AI model (e.g., GPT-4) to assess the user's level. The generated questions assess the four skills of reading, writing, listening, and speaking individually. This allows for a comprehensive understanding of the user's skill level. For example, intermediate-level written questions are provided for reading, and easy audio questions are provided for listening.

[1087] Users use their devices to answer questions and send the results to the server. The server analyzes the answers and uses NLP technology to assess the user's initial skill level. The server also includes a means to customize a learning program based on the user's set goal (e.g., "800 points on the TOEIC") and the date by which it is achieved. The program includes intermediate-level problem sets to strengthen reading and beginner-level audio materials to improve listening comprehension.

[1088] The server generates daily learning content using a generative AI model and provides a means to send it to the device. For example, "Today's listening questions" and "Today's writing assignments" are presented. The user can check and work on these learning content through the device.

[1089] The user inputs the results of their study into their device and sends them to the server. The server analyzes the user's response data and evaluates the day's study results. The system includes a means for adjusting the study content for the next day based on this evaluation. Specifically, if the user correctly answers a beginner-level listening question, a slightly more difficult question will be presented the next day.

[1090] Additionally, the server provides a means to conduct regular level checks as needed. These checks also use the generative AI model to generate questions and present them to the user. The server then analyzes the user's answers again to assess their current skill level. Based on the results, it reconstructs the learning program and sends it to the device. This ensures that the user's learning plan is always up to date.

[1091] An example of a specific prompt might be, "The user wants to learn intermediate-level reading comprehension questions to improve their reading. Please generate appropriate questions." This prompt is input into a generative AI model, which then generates appropriate reading questions.

[1092] In this way, this system provides a high level of collaboration between the server, terminals, and users, enabling efficient and effective improvement of English skills according to the individual needs of each user.

[1093] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1094] System program processing flow

[1095] Step 1: User Login and Authentication

[1096] Input: The user enters their email address and password into the terminal and presses the login button.

[1097] Action: The device sends the entered authentication information to the server.

[1098] Processing: The server checks the authentication information against the database and authorizes the user to log in.

[1099] Output: Successful login information is sent to the terminal and displayed to the user.

[1100] Step 2: Generate questions for initial level check

[1101] Input: After the server verifies the user's login, it starts the initial level check.

[1102] How it works: The server inputs prompts into the generative AI model, which generates questions for reading, writing, listening, and speaking.

[1103] Processing: The generative AI model generates a question based on the prompt and returns the result to the server.

[1104] Output: The generated questions are sent to the terminal and displayed to the user.

[1105] Step 3: Conduct an initial level check

[1106] Input: The user answers the questions and enters the answer data into the terminal.

[1107] Operation: The device sends the response data to the server.

[1108] Processing: The server analyzes the response data and evaluates the user's initial level for each skill (reading, writing, listening, and speaking).

[1109] Output: Evaluation results and feedback are sent to the device.

[1110] Step 4: Set goals and create a learning program

[1111] Input: The user inputs the goal (e.g., "800 points on the TOEIC") and the deadline for achieving it into the terminal and submits it.

[1112] Operation: The device sends the goal and deadline information to the server.

[1113] Processing: The server generates a learning program based on the user's initial level and goals. In this process, the generative AI model is used again.

[1114] Output: The customized learning program is sent to the terminal and displayed to the user.

[1115] Step 5: Provide daily learning content

[1116] Input: The server generates today's learning content based on the learning program.

[1117] How it works: The generative AI model generates learning content based on the prompt.

[1118] Processing: The server sends the generated learning content to the terminal.

[1119] Output: Today's learning content will be displayed on the terminal.

[1120] Step 6: Input and analysis of training results

[1121] Input: The user works on the learning content and inputs the results into the terminal.

[1122] Operation: The device sends the learning results to the server.

[1123] Processing: The server analyzes the learning results, evaluates the learning outcomes of the day, and generates feedback to adjust the learning content for the next day.

[1124] Output: Analysis results and applied feedback are sent to the device.

[1125] Step 7: Regular level checks and program restructuring

[1126] Input: After a certain period of time has passed, the server generates questions for regular level checks.

[1127] How it works: The regenerative AI model generates a question based on the prompt.

[1128] Processing: The server sends the generated questions to the terminal, and the user answers them. The answer data is sent to the server and analyzed.

[1129] Output: The latest skill level evaluation results are sent to the terminal, and the learning program is reconstructed.

[1130] Through the above processing steps, the system of the present invention efficiently and effectively supports users in improving their English skills in accordance with their individual needs.

[1131] (Application example 1)

[1132] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1133] Improving listening and speaking skills in English learning is a difficult task for many learners. In particular, there is a demand for efficient use of time while riding in an autonomous vehicle to improve English skills. To solve this problem, an effective English learning support system using smart devices is required.

[1134] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1135] In this invention, the server includes a means for generating questions to assess the learner's level and creating a learning program based on the learner's answers, a means for generating daily learning content and providing it to the learner, and a means for analyzing the learner's answer data and reconstructing the learning program as needed, thereby enabling a means for receiving and analyzing voice input using a smart device, a means for dynamically generating questions and answers using a generative AI model, and a means for providing audio and video content to the learner using a smart device.

[1136] "Learner" refers to an individual who uses the System to improve their English language skills.

[1137] "Level assessment questions" are questions that are asked to understand the learner's current level of reading, writing, listening, and speaking skills.

[1138] A "learning program" is a plan that combines the practice and materials necessary for a learner to achieve their desired English skills.

[1139] A "generative AI model" is an algorithm that uses artificial intelligence technology to automatically generate optimal learning content and questions.

[1140] "Smart devices" refer to electronic devices that can connect to the Internet and have voice input and video display functions, and examples include smartphones and smart glasses.

[1141] "Means for receiving and analyzing voice input" refers to the function of collecting learners' speech through the microphone of a smart device and recognizing and analyzing its content.

[1142] "Means for reviewing and dynamically providing learning content" refers to a function that generates and delivers optimal practice questions and teaching materials in real time according to the learner's level and progress.

[1143] "Means for dynamically generating questions and answers" refers to a function that uses a generative AI model to generate appropriate questions and answers in real time according to the learner's level and learning progress.

[1144] "Means for providing audio and video content" refers to a function that supports learning by allowing learners to listen to audio or watch videos using smart devices.

[1145] "Regular level checks" refer to tests conducted at regular intervals to assess learners' progress and determine their current English skill level.

[1146] The English conversation learning support system using smart devices of the present invention is implemented as follows: This system is made up of a server, a terminal, and a user, and each element works in cooperation with each other.

[1147] 1. Initial setup and level check

[1148] The user accesses the system using their own terminal (smart device) and logs in.

[1149] The server verifies the user's credentials and approves the login.

[1150] The server uses a generative AI model to generate questions for an initial level check, which assess each of the four skills: reading, writing, listening, and speaking.

[1151] The user answers the questions and sends the answers from the terminal to the server.

[1152] The server analyzes the user's response data and evaluates the user's initial level in each skill, thereby understanding the learner's current level.

[1153] 2. Setting goals and creating a learning program

[1154] The user inputs the goal (e.g., "to acquire English conversation skills at a daily conversation level") and the date by which it will be achieved from the terminal and sends it to the server.

[1155] The server generates an optimal learning program based on the user's current level and set goals. This program includes problem sets and audio materials to improve listening and speaking skills.

[1156] 3. Providing daily learning content and analyzing learning results

[1157] The server generates daily learning content using a generative AI model and sends it to the device.

[1158] The user checks the learning content through the device and works on each task. For example, "Today's listening questions" and "Today's speaking tasks" are presented.

[1159] The user responds by voice input, and the voice data is sent from the terminal to the server.

[1160] The server analyzes the audio data and evaluates the learning outcomes for that day, and the learning content for the next day is adjusted based on this evaluation.

[1161] 4. Regular level checks and program restructuring

[1162] The server periodically generates level check questions and sends them to the terminal.

[1163] The user answers the questions and sends the answer data from the terminal to the server.

[1164] The server analyzes the response data and evaluates the user's current English level.

[1165] The server reconstructs the learning program based on the latest evaluation and sends it to the terminal, allowing the user to always receive a program that meets their latest learning needs.

[1166] Hardware and software used

[1167] Hardware: Smart devices (smartphones, smart glasses, etc.)

[1168] Software: Generative AI models, speech recognition libraries (e.g., speech_recognition)

[1169] The system uses a generative AI model to dynamically generate questions and collects and analyzes user speech data through speech recognition, allowing users to efficiently improve their English conversation skills.

[1170] Specific examples

[1171] When a user uses this system to learn English conversation while riding in an autonomous vehicle, the following prompt sentences are generated:

[1172] "Please generate a list of English conversation phrases suitable for a beginner level. These phrases should be commonly used in travel situations."

[1173] For example, a passenger repeats the following phrase presented as "Today's Listening Question."

[1174] "How can I get to the nearest train station?"

[1175] Once the passenger has spoken correctly, their speech is analyzed and the next phrase is presented.

[1176] "Thank you! Let's move on to the next phrase."

[1177] In this way, you can efficiently improve your English conversation skills while managing your progress.

[1178] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1179] Step 1:

[1180] Initial setup and level check

[1181] When a user accesses the system using their own device, the server verifies the user's authentication information and approves the login. Next, it uses the generative AI model to generate questions for an initial level check and sends them to the device. The user answers the questions and sends the answers from the device to the server. The server then analyzes the user's answer data and evaluates the user's initial level in each skill. In this process, the user's answers are the input data, the generative AI model generates questions, and the server analyzes the data to evaluate the learner's level.

[1182] Step 2:

[1183] Setting goals and creating learning programs

[1184] The user inputs their desired goal (e.g., "Acquire English conversation skills at a daily conversation level") and the date by which they will achieve this goal from their device and sends it to the server. The server generates an optimal learning program based on the user's current level and the set goal. Specifically, it uses a generative AI model to dynamically create problem sets and audio materials to improve listening and speaking skills. In this process, the user's goal and current level are input data, and the generative AI model is used to generate a learning program, which outputs an optimal learning plan.

[1185] Step 3:

[1186] Providing daily learning content and analyzing learning results

[1187] The server generates daily learning content using a generative AI model and sends it to the device. The user checks the learning content through the device and works on each assignment. The user answers through voice input, and the voice data is sent from the device to the server. The server analyzes the voice data and evaluates the learning outcome for that day. Based on this evaluation, the learning content for the next day is adjusted. In this process, the user's voice data is the input data, and the server analyzes and evaluates the learning outcome, adjusting the learning content for the next day to obtain the output.

[1188] Step 4:

[1189] Regular level checks and program restructuring

[1190] The server periodically generates level-check questions and sends them to the terminal. The user answers the questions and sends the answer data from the terminal to the server. The server analyzes the answer data and evaluates the user's latest English level. The server reconstructs a learning program based on the latest evaluation and sends it to the terminal. This allows the user to always receive a program that meets their latest learning needs. In this process, the user's answer data is the input data, and the server analyzes it to evaluate the user's latest English level, then generates and outputs a new learning program.

[1191] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1192] The self-contained English skill improvement learning support system of this invention achieves a higher level of personalization by combining a conventional learning support system with an emotion engine that recognizes the user's emotions. The system is composed of a server, terminals, and users, and each element works in cooperation with each other.

[1193] 1. Initial setup and level check

[1194] A user accesses the system using their terminal and logs in.

[1195] The server verifies the user's credentials and approves the login.

[1196] The server uses a generative AI model to generate questions for an initial level check, which automatically generate questions to assess the four skills of reading, writing, listening, and speaking.

[1197] The user answers the questions and sends the answers from the terminal to the server.

[1198] The server analyzes the user's response data and evaluates the user's initial level in each skill, thereby understanding the learner's current level.

[1199] 2. Setting goals and creating a learning program

[1200] The user inputs the goal they are aiming for (e.g., "Aiming for 800 points on the TOEIC") and the date by which they will achieve it on their device, and sends it to the server.

[1201] The server generates an optimal learning program based on the user's current level and set goals. This program includes practice content for each skill required to achieve the user's goal.

[1202] For example, it includes problem sets to strengthen reading skills and audio materials to improve listening skills.

[1203] 3. Providing daily learning content and analyzing learning results

[1204] The server generates daily learning content using a generative AI model and sends it to the device.

[1205] Users check the learning content through their devices and work on each assignment.

[1206] For example, "Today's listening questions" and "Today's writing assignments" are presented.

[1207] The user inputs the learning results and answers and sends them from the terminal to the server.

[1208] The server receives this and analyzes it.

[1209] 4. Implementation and Use of Emotion Engine

[1210] The server uses an emotion engine to recognize the user's emotions in real time, specifically assessing the user's current emotions (e.g., stress, concentration, fatigue, etc.) through facial recognition technology and voice analysis.

[1211] The server then adjusts learning content and break suggestions based on the recognized emotion data. For example, if the user feels tired, a pop-up will appear recommending a break.

[1212] The user accepts the sentiment-based suggestions and chooses whether to pause or continue learning.

[1213] 5. Regular level checks and program restructuring

[1214] The server periodically generates level check questions and sends them to the terminal.

[1215] The user answers the questions and sends the answer data from the terminal to the server.

[1216] The server analyzes the response data and evaluates the user's current English level.

[1217] The server reconstructs the learning program based on the latest evaluation and sends it to the terminal, allowing the user to always receive a program that meets their latest learning needs.

[1218] Specific examples

[1219] For example, consider a case where a user sets a goal of "aiming for 800 points on the TOEIC." In the initial level check, the user's reading ability is assessed as intermediate and their listening ability as beginner. In this case, the server provides intermediate-level reading comprehension questions to strengthen reading ability, while constructing a program that includes many beginner-level audio materials for listening ability.

[1220] During daily learning, the emotion engine detects the user's stress level, and if high stress is detected, the server reduces the learning content and switches to more relaxing content. If the user feels tired, a notification is displayed recommending a break.

[1221] Regular level checks evaluate progress since the last level check and check how much reading and listening skills have improved. Based on the results, a new learning program is set up so that users can continue to study appropriately toward their goals.

[1222] In this way, the system constantly monitors the user's progress and emotional state, dynamically optimizing learning content to improve English skills efficiently and effectively.

[1223] The processing flow will be explained below.

[1224] Step 1:

[1225] A user accesses the system using their terminal and logs in.

[1226] The server verifies the user's credentials and approves the login.

[1227] Step 2:

[1228] The server uses a generative AI model to generate questions for an initial level check.

[1229] Specifically, it automatically generates questions to assess the four skills of reading, writing, listening, and speaking.

[1230] Step 3:

[1231] The server sends the generated question to the terminal.

[1232] The user checks the questions on the terminal and enters the answers.

[1233] Step 4:

[1234] The user sends the answer data from the terminal to the server.

[1235] The server receives the user's response data and analyzes it.

[1236] Step 5:

[1237] The server evaluates the user's initial level of the four skills based on the response data.

[1238] Based on the evaluation results, the user's current skill level is recorded.

[1239] Step 6:

[1240] The user uses the terminal to input the goal and the date of achievement into the server and transmit it.

[1241] Specifically, set a goal such as "aiming for 800 points on the TOEIC."

[1242] Step 7:

[1243] The server generates an optimal learning program based on the user's current level and set goals.

[1244] The program includes practice tailored to the skills of reading, listening, writing and speaking.

[1245] Step 8:

[1246] The server transmits the generated learning program to the terminal.

[1247] The user checks and practices the learning program on the device.

[1248] Step 9:

[1249] The server generates daily learning content using a generative AI model and sends it to the device.

[1250] The user works on the day's learning content (for example, listening questions and writing assignments) on the terminal.

[1251] Step 10:

[1252] The user inputs the results of that day's learning into the terminal and sends them to the server.

[1253] The server receives this and analyzes it.

[1254] Step 11:

[1255] The server evaluates the user's progress based on the learning results.

[1256] Based on the evaluation results, the learning content for the next day is adjusted and provided to the user.

[1257] Step 12:

[1258] The server uses an emotion engine to recognize the user's emotions in real time.

[1259] Specifically, it assesses the user's current emotions (e.g., stress, concentration, fatigue, etc.) through facial recognition technology and voice analysis.

[1260] Step 13:

[1261] The server adjusts learning content and break suggestions based on the recognized emotion data.

[1262] For example, if stress levels are high, the system will provide less difficult questions and switch to content that has a relaxing effect.

[1263] Step 14:

[1264] The user chooses whether to accept sentiment-based suggestions.

[1265] If the user feels they need a break, they can pause their learning and refresh.

[1266] Step 15:

[1267] The server generates questions for regular level checks as needed and sends them to the terminal.

[1268] The user answers this and sends the results to the server.

[1269] Step 16:

[1270] The server analyzes the results of regular level checks and evaluates the user's latest English level.

[1271] The learning program is reconstructed based on the evaluation results and sent to the terminal.

[1272] Step 17:

[1273] The user continues learning with a new learning program.

[1274] When you log in again, the cycle repeats.

[1275] Through these processing steps, users are always provided with a learning plan that is optimized for their level, progress, and emotional state, allowing them to efficiently improve their English skills.

[1276] Example 2

[1277] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1278] Conventional learning support systems provide learning programs tailored to a learner's current level and goals, but they are unable to recognize the learner's emotional state and adjust the learning content based on that emotion. This creates the problem that appropriate support is not provided even when the learner feels fatigued or stressed, making efficient learning difficult. In addition, rebuilding learning programs and conducting regular level checks is cumbersome, creating a need for automatic dynamic adjustments.

[1279] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1280] In this invention, the server includes means for generating questions to assess the learner's level and creating a learning program based on the learner's answers, means for generating daily learning content and providing it to the learner, means for recognizing the learner's emotional state in real time and adjusting the learning content and break suggestions, means for analyzing the learner's answer data and reconstructing the learning program as appropriate, and means for conducting regular level checks as necessary and adjusting the learning program based on the evaluation, thereby enabling efficient and personalized learning support that takes the learner's emotional state into consideration.

[1281] The term "learner" refers to an individual who uses the learning system for the purpose of improving a particular skill (mainly language skill in the present invention).

[1282] A "server" refers to a computer system that provides various services to learners via a network, and in the present invention, it mainly generates and provides learning programs.

[1283] "Question" refers to a question or problem generated by the server to assess a learner's skill or level.

[1284] "Learning program" refers to a set of educational curriculum and assignments created by the server based on the learner's current level and goals.

[1285] "Learning content" refers to specific learning tasks and learning materials that are generated by the server daily and provided to learners.

[1286] "Response data" refers to the responses entered by learners to questions and assignments.

[1287] "Emotional state" refers to the learner's current psychological or emotional state (e.g., stress, concentration, fatigue).

[1288] An "emotion engine" refers to a system that uses facial recognition technology, voice analysis technology, and other techniques to recognize a learner's emotional state in real time.

[1289] "Dynamic adjustment" refers to changing and optimizing learning programs and content in real time based on the learner's progress and emotional state.

[1290] "Level check" refers to a test or assessment to assess a learner's current skill level.

[1291] This invention relates to a self-contained English skill improvement learning support system that assesses learners' levels and creates and provides learning programs. The system consists of a server, terminals, and learners, and each element works in conjunction with the others. The server uses a generative AI model to generate questions and learning content, and an emotion engine to recognize the learner's emotional state and dynamically adjust the learning content.

[1292] Initial setup and level check

[1293] The learner accesses the system using a device and logs in. The server verifies the learner's authentication information and approves the login. The server uses a generative AI model to generate questions to assess the four skills of reading, writing, listening, and speaking. The learner answers the questions and sends the data from their device to the server. The server analyzes the response data and evaluates the initial level of each skill to determine the learner's current skill level.

[1294] Setting goals and creating learning programs

[1295] Learners input their goal (e.g., "Aim for 800 points on the TOEIC") and the target date for achieving it on their device and send it to the server. The server then generates an optimal learning program based on the learner's current level and the set goal. This program might include, for example, a problem set to improve reading skills or audio materials to improve listening ability.

[1296] Providing daily learning content and analyzing learning results

[1297] The server uses a generative AI model to generate daily learning content and sends it to the device. The learner checks the learning content through the device and works on each task. For example, they are presented with a "today's listening question" or a "today's writing task." The learner inputs their learning results and answers and sends them from their device to the server. The server analyzes the received data and evaluates the learner's progress.

[1298] Implementing and utilizing an emotion engine

[1299] The server uses an emotion engine to recognize the learner's emotional state in real time. Specifically, it uses facial recognition technology and voice analysis to assess the learner's current emotions (e.g., stress, concentration, fatigue, etc.). The server adjusts learning content and break suggestions based on the emotion data. For example, if the learner feels fatigued, a pop-up recommending a break will be displayed. The learner can accept the emotion-based suggestion and choose whether to pause or continue learning.

[1300] Regular level checks and program restructuring

[1301] The server periodically generates level-check questions and sends them to the device. The learner answers the questions and sends the answer data from the device to the server. The server analyzes the data and evaluates the learner's current English level. Based on this evaluation, the server reconstructs the learning program and sends it to the device. This ensures that the learner always receives a program that meets their latest learning needs.

[1302] Specific examples and examples of prompts for the generative AI model

[1303] For example, consider a case where a learner sets a goal of "achieving 800 points on the TOEIC." Suppose that the initial level check assesses reading as intermediate and listening as beginner. In this case, the server provides intermediate-level reading comprehension questions to strengthen reading, while constructing a program that includes many beginner-level audio materials for listening. During daily learning, if the emotion engine detects high stress, the server reduces the learning content and switches to content with a relaxing effect. Furthermore, if the learner feels tired, a notification is displayed recommending a break.

[1304] Example prompts for generative AI models

[1305] "A student is aiming for 800 points on the TOEIC, and their current reading level is intermediate and their listening level is beginner. Please generate daily study content based on the student's progress."

[1306] In this way, the system constantly monitors learners' progress and emotional state, dynamically optimizing learning content to help them improve their English skills efficiently and effectively.

[1307] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1308] Step 1:

[1309] A user accesses the system using their terminal and logs in.

[1310] Input: User ID and password.

[1311] How it works: The user opens a browser or a dedicated app, accesses the login screen, and enters their user ID and password.

[1312] Output: The entered authentication information is sent to the server.

[1313] Step 2:

[1314] The server verifies the user's credentials and approves the login.

[1315] Input: User ID and password.

[1316] How it works: The server accesses the database and checks the entered credentials.

[1317] Output: If authentication is successful, the login is approved and the user is shown the dashboard.

[1318] Step 3:

[1319] The server uses a generative AI model to generate questions for an initial level check.

[1320] Input: User login information.

[1321] How it works: The server sends prompts to a generative AI model to generate questions to assess the four skills of reading, writing, listening, and speaking.

[1322] Output: The generated questions are sent to the user's terminal.

[1323] Step 4:

[1324] The user answers the questions and sends the answers from the terminal to the server.

[1325] Input: The generated question.

[1326] How it works: The user reviews the questions on the terminal and enters answers to each question.

[1327] Output: The answer data is sent from the device to the server.

[1328] Step 5:

[1329] The server analyzes the user's response data and evaluates the user's initial level in each skill.

[1330] Input: User response data.

[1331] How it works: The server uses natural language processing (NLP) technology to analyze the response data and assess the initial level of each skill (reading, writing, listening, and speaking).

[1332] Output: User's initial level assessment result.

[1333] Step 6:

[1334] The user inputs the goal and the date of achievement from the terminal and transmits it to the server.

[1335] Input: Your goal and the date you want to reach it.

[1336] How it works: The user enters the goal and target achievement date into a dedicated input form and presses the submit button.

[1337] Output: The goal and the target date are sent to the server.

[1338] Step 7:

[1339] The server generates an optimal learning program based on the user's current level and set goals.

[1340] Input: User's initial level assessment results and goal / target achievement date.

[1341] How it works: The server sends prompts to the generative AI model to generate a learning program appropriate for the user's current level and goals.

[1342] Output: The generated learning program is sent to the user's terminal.

[1343] Step 8:

[1344] The server generates daily learning content using a generative AI model and sends it to the device.

[1345] Input: The user's study program.

[1346] How it works: The server sends prompts to the generative AI model to generate learnings for the day.

[1347] Output: The generated learning content is sent to the user's device.

[1348] Step 9:

[1349] Users check the learning content through their devices and work on each assignment.

[1350] Input: What you learned that day.

[1351] Action: The user checks the learning content on the device and works on the assignments. For example, they play an audio file and answer listening questions.

[1352] Output: Learning results and answer data.

[1353] Step 10:

[1354] The user inputs the learning results and answers and sends them from the terminal to the server.

[1355] Input: Learning results and answer data.

[1356] Operation: The user enters the learning results and answers and presses the submit button.

[1357] Output: Results and response data are sent to the server.

[1358] Step 11:

[1359] The server uses an emotion engine to recognize the learner's emotional state in real time.

[1360] Input: Learner's video and audio data.

[1361] Operation: The server runs the emotion engine to assess the learner's current emotional state through facial recognition technology and voice analysis.

[1362] Output: Emotional state assessment results.

[1363] Step 12:

[1364] The server adjusts learning content and break suggestions based on the recognized emotion data.

[1365] Input: Emotional state assessment results.

[1366] How it works: The server adjusts the difficulty of the learning content and suggests breaks based on the user's emotional state. For example, if the user is highly fatigued, a pop-up will appear recommending a break.

[1367] Output: Tailored learning content and break suggestions.

[1368] Step 13:

[1369] The user chooses whether to accept sentiment-based suggestions.

[1370] Input: Study content and break suggestions.

[1371] Action: The user reviews the suggestions and selects "Take a break" or "Continue learning."

[1372] Output: Next action based on user selection.

[1373] Step 14:

[1374] The server periodically generates level check questions and sends them to the terminal.

[1375] Input: Learner progress data.

[1376] How it works: The server uses the generative AI model to generate questions for regular level checks.

[1377] Output: Generated level check questions.

[1378] Step 15:

[1379] The user answers the questions and sends the answer data from the terminal to the server.

[1380] Input: Questions for regular level checks.

[1381] Operation: The user answers the questions and sends the answer data from the terminal to the server.

[1382] Output: The answer data is sent to the server.

[1383] Step 16:

[1384] The server analyzes the response data and evaluates the user's current English level.

[1385] Input: Response data.

[1386] How it works: The server analyzes the response data and evaluates your current English level.

[1387] Output: Latest English level assessment results.

[1388] Step 17:

[1389] The server reconstructs the learning program based on the latest evaluation and sends it to the device.

[1390] Input: Your most recent English level assessment results.

[1391] How it works: The server uses the generative AI model to reconstruct a learning program based on the latest English level.

[1392] Output: A reconstructed learning program.

[1393] (Application example 2)

[1394] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1395] Conventional learning support systems provide learning content mechanically without considering the learner's emotional state, resulting in problems such as reduced learning efficiency due to learner fatigue or stress. While these systems dynamically adjust learning programs based on the learner's progress and ability, they do not monitor the learner's emotional state, resulting in an insufficient optimization of the individual learning experience. In particular, in work environments using robots, such as factories, a system is needed that can optimize work efficiency according to the worker's emotional state.

[1396] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: a means for generating questions to evaluate the learner's level and creating a learning program based on the learner's answers; a means for generating daily learning content and providing it to the learner; a means for analyzing the learner's answer data and reconstructing the learning program as needed; a means for conducting periodic level checks as needed and adjusting the learning program based on the evaluation; and a means for recognizing the learner's emotional state in real time and dynamically optimizing the learning content based on the learner's emotional state. This makes it possible to provide a learning program that takes the learner's emotional state into consideration, thereby reducing learner stress and fatigue and improving learning efficiency. Furthermore, by applying this to factory work sites, it is possible to optimize work schedules and robot operations according to the worker's emotional state.

[1397] "Assessing the learner's level" means measuring the level of knowledge and skills that the learner currently possesses and providing appropriate learning content and programs based on that.

[1398] "Question generation" means automatically creating questions or problems to assess a learner's ability.

[1399] "Creating a learning program" means planning and designing learning content and activities based on the learner's goals and current abilities.

[1400] "Generating daily learning content" means automatically creating and providing learning tasks and materials that learners should tackle every day.

[1401] "Providing to learners" means displaying and transmitting learning content and assignments to the devices and systems used by learners.

[1402] "Analyzing learner response data" means analyzing the responses and results submitted by learners and evaluating their learning progress and level of understanding based on the results.

[1403] "Reconstructing learning programs in a timely manner" means dynamically updating and redesigning learning content and programs according to learners' progress and assessment results.

[1404] "Conducting regular level checks" means administering tests and questions to assess learners' abilities on a regular basis.

[1405] "Recognizing the learner's emotional state in real time" means instantly assessing the learner's current emotional and psychological state using information such as their facial expressions and voice.

[1406] "Dynamic optimization of learning content based on emotional state" means flexibly adjusting the learning content and difficulty level by reflecting the learner's emotional and psychological state.

[1407] "Facial recognition technology" is a technology that detects an individual's face from camera footage and analyzes their facial expressions and characteristics.

[1408] "Voice analysis technology" is a technology that analyzes voice data and evaluates its content and the speaker's emotional state.

[1409] "Detecting fatigue and stress" means determining the level of fatigue and stress experienced by learners through facial recognition and voice analysis.

[1410] "Displaying a notification recommending a break" means that when the system detects that the learner is fatigued or stressed, a message urging the learner to take a break will be displayed on the device.

[1411] The present invention provides a learning support system that recognizes the emotional state of a learner in real time and dynamically optimizes the generation and provision of learning programs. The system is composed of a server, terminals, and users.

[1412] 1. Initial setup and level check

[1413] A user accesses the system using a terminal and logs in. The server verifies the user's authentication information and approves the login. The server then uses a generative AI model to generate questions for an initial level check. These questions are designed to assess the four skills of reading, writing, listening, and speaking. The user answers the questions and sends the answers from the terminal to the server. The server analyzes the user's response data and evaluates the initial level in each skill.

[1414] 2. Setting goals and creating a learning program

[1415] The user inputs their goal (for example, "Aiming for 800 points on the TOEIC") and the date they wish to achieve it, and sends it to the server. The server then generates an optimal learning program based on the user's current level and the goal they set. This program includes practice content for each skill the user needs to achieve their goal.

[1416] 3. Providing daily learning content and analyzing learning results

[1417] The server uses a generative AI model to generate daily learning content and sends it to the device. The user checks the learning content through the device and works on each task. The device then sends the learning results and answers to the server, which receives and analyzes them.

[1418] 4. Implementation and Use of Emotion Engine

[1419] The server uses an emotion engine to recognize the user's emotions in real time. The hardware used includes a camera and microphone. The software uses OpenCV for facial recognition technology and TensorFlow / Keras for emotion recognition models. Using these technologies, the server evaluates the user's current emotions (e.g., stress, concentration, fatigue, etc.). Based on the recognized emotion data, the server adjusts the learning content and suggests breaks. For example, if the server recognizes that the user is feeling fatigued, it will display a notification on the device recommending a break.

[1420] 5. Regular level checks and program restructuring

[1421] The server periodically generates level-check questions and sends them to the device. The user answers the questions and sends the answer data from the device to the server. The server analyzes the answer data and evaluates the user's latest English level. Based on the latest evaluation, the server reconstructs the learning program and sends it to the device. This ensures that the user always receives a program that meets their latest learning needs.

[1422] Specific examples

[1423] For example, consider a user who sets a goal of "achieving 800 points on the TOEIC." Suppose the user's reading and listening skills are assessed as intermediate and beginner levels in the initial level check. In this case, the server provides intermediate-level reading comprehension questions to strengthen reading, while constructing a program that includes many beginner-level audio materials for listening. During daily study, the emotion engine detects the user's stress level. If high stress is recognized, the server reduces the study content and switches to content with a relaxing effect. If the user feels tired, the server displays a notification recommending a break. Periodic level checks evaluate progress since the previous level check and confirm the degree of improvement in reading and listening skills. Based on the results, a new study program is created, allowing the user to continue studying appropriately toward their goal.

[1424] Prompt Sentence Examples

[1425] "Please explain how to recognize the emotions of factory workers in real time and optimize labor efficiency based on their emotions. I would like to build a system that sends emotional data to a server, which then adjusts optimal work schedules and robot operations. As an example, please explain in detail how you combine facial recognition technology with an emotion engine."

[1426] This allows learners' progress and emotional state to be constantly monitored, and learning content to be dynamically optimized, enabling them to improve their English skills efficiently and effectively.

[1427] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1428] Step 1:

[1429] A user accesses the system using a terminal and logs in. The user enters login information (user ID and password), which the terminal sends to the server. The server verifies the user's authentication information and approves the login. The input data is the user's authentication information, and the output data is the login success or failure status.

[1430] Step 2:

[1431] The server uses a generative AI model to generate questions for an initial level check. The server uses prompts to request questions to assess the skills of reading, writing, listening, and speaking from the generative AI model, and sends the generated questions to the terminal. The input data is the prompts, and the output data is the generated questions.

[1432] Step 3:

[1433] The user answers questions through the terminal and sends the answers to the server. The terminal acquires the user's input and sends the data to the server. The input data is the user's answer, and the output data is a notification that the answer data has been sent.

[1434] Step 4:

[1435] The server analyzes the user's response data and evaluates their initial level. The server then inputs the response data into an analysis algorithm to calculate a score for each skill. The input data is the user's response data, and the output data is the evaluation result of the initial level for each skill.

[1436] Step 5:

[1437] The user inputs the goal they are aiming for and the date by which they will reach it, and sends it to the server. The user inputs the goal they have set (for example, "Aim for 800 points on the TOEIC") and the date by which they plan to reach it from their terminal, and sends it to the server. The input data is the goal and the date by which it will be reached, and the output data is a notification that the goal has been set.

[1438] Step 6:

[1439] The server generates an optimal learning program based on the user's current level and set goals. Based on the initial level assessment results and goals, the server requests the AI ​​model to generate learning content suitable for the user, and creates that content. The input data is the initial level assessment results and the user's goals, and the output data is the generated learning program.

[1440] Step 7:

[1441] The server generates daily learning content using a generative AI model and sends it to the device. The server dynamically adjusts the daily learning content taking into account the user's progress and emotional state and sends it to the device. The input data is the user's progress and emotional state, and the output data is the generated daily learning program.

[1442] Step 8:

[1443] The user works on daily learning content through the terminal and inputs the results. The user performs the learning tasks displayed on the terminal and inputs the answers. The input data is the user's learning results, and the output data is a notification that the learning results have been sent.

[1444] Step 9:

[1445] The server receives and analyzes the learning result data. The server analyzes the received learning results and evaluates the progress. The input data is the learning result data, and the output data is the progress evaluation result.

[1446] Step 10:

[1447] The server uses an emotion engine to recognize the user's emotions in real time. The server then activates the emotion engine and analyzes the camera video and audio data to evaluate the user's emotional state. The input data are the camera video and audio data, and the output data are the emotion evaluation results.

[1448] Step 11:

[1449] The server adjusts the learning content and break suggestions based on the emotion evaluation results. The server dynamically adjusts the learning content based on the emotion evaluation results, and in some cases displays a notification on the device recommending a break. The input data is the emotion evaluation results, and the output data is the adjusted learning content and break suggestions.

[1450] Step 12:

[1451] The server periodically generates level check questions and sends them to the terminal. The server also generates questions to reassess the user's skills at regular intervals and sends them to the terminal. The input data is the prompt text, and the output data is the generated level check questions.

[1452] Step 13:

[1453] The user answers questions for the regular level check and sends the results to the server. The user answers questions and sends the data to the server. The input data is the answers to the regular level check, and the output data is a notification that the answer data has been sent.

[1454] Step 14:

[1455] The server analyzes the response data and evaluates the user's latest English level. The server analyzes the response data and evaluates the user's progress. The input data is the response data from the regular level check, and the output data is the latest English level evaluation result.

[1456] Step 15:

[1457] The server reconstructs the learning program based on the latest evaluation results and sends it to the terminal. The server generates a new learning program that reflects the latest evaluation results and sends it to the terminal. The input data is the latest evaluation results, and the output data is the reconstructed learning program.

[1458] This makes it possible to provide an optimal learning environment that takes into account the user's emotional state, maximizing learning efficiency.

[1459] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1460] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1461] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1462] [Fourth embodiment]

[1463] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1464] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1465] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1466] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1467] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1468] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1469] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1470] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1471] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1472] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1473] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1474] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1475] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1476] The self-contained English skill improvement learning support system of the present invention is implemented as follows: The system is made up of a server, terminals, and users, and each element operates in cooperation with each other.

[1477] 1. Initial setup and level check

[1478] A user accesses the system using their terminal and logs in.

[1479] The server verifies the user's credentials and approves the login.

[1480] The server uses a generative AI model to generate questions for an initial level check, which assess each of the four skills: reading, writing, listening, and speaking.

[1481] The user answers the questions and sends the answers from the terminal to the server.

[1482] The server analyzes the user's response data and evaluates the user's initial level in each skill, thereby understanding the learner's current level.

[1483] 2. Setting goals and creating a learning program

[1484] The user inputs the goal they are aiming for (e.g., "Aiming for 800 points on the TOEIC") and the date by which they will achieve it on their device, and sends it to the server.

[1485] The server generates an optimal learning program based on the user's current level and set goals. This program includes practice content for each skill required to achieve the user's goal.

[1486] For example, it includes problem sets to strengthen reading skills and audio materials to improve listening skills.

[1487] 3. Providing daily learning content and analyzing learning results

[1488] The server generates daily learning content using a generative AI model and sends it to the device.

[1489] Users check the learning content through their devices and work on each assignment.

[1490] For example, "Today's listening questions" and "Today's writing assignments" are presented.

[1491] The user inputs the learning results and answers and sends them from the terminal to the server.

[1492] The server analyzes the user's response data and evaluates the user's learning outcomes for that day. Based on this evaluation, the learning content for the next day is adjusted.

[1493] 4. Regular level checks and program restructuring

[1494] The server periodically generates level check questions and sends them to the terminal.

[1495] The user answers the questions and sends the answer data from the terminal to the server.

[1496] The server analyzes the response data and evaluates the user's current English level.

[1497] The server reconstructs the learning program based on the latest evaluation and sends it to the terminal, allowing the user to always receive a program that meets their latest learning needs.

[1498] Specific examples

[1499] For example, let's consider a case where a user sets a goal of "aiming for 800 points on the TOEIC." In the initial level check, the user's reading ability is assessed as intermediate and their listening ability as beginner. In this case, the server provides intermediate-level reading comprehension questions to strengthen reading ability, while constructing a program that includes many beginner-level audio materials for listening ability.

[1500] In the daily learning session, a beginner-level audio file is provided as a listening test for the day, and the user is asked to answer the questions. When the user submits their answers, the server analyzes them and adjusts the program to provide audio materials with a slightly higher level of difficulty the following day.

[1501] Regular level checks evaluate progress since the last level check and check how much reading and listening skills have improved. Based on the results, a new learning program is set up so that users can continue to study appropriately toward their goals.

[1502] In this way, this system constantly monitors the progress of each individual user and dynamically optimizes their learning content, enabling them to improve their English skills efficiently and effectively.

[1503] The processing flow will be explained below.

[1504] Step 1:

[1505] A user accesses the system using their terminal and logs in.

[1506] The server verifies the user's credentials and approves the login.

[1507] Step 2:

[1508] The server uses a generative AI model to generate questions for an initial level check.

[1509] Specifically, it automatically generates questions to assess the four skills of reading, writing, listening, and speaking.

[1510] Step 3:

[1511] The server sends the generated question to the terminal.

[1512] The user checks the questions on the terminal and enters the answers.

[1513] Step 4:

[1514] The user sends the answer data from the terminal to the server.

[1515] The server receives the user's response data and analyzes it.

[1516] Step 5:

[1517] The server evaluates the user's initial level of the four skills based on the response data.

[1518] Based on the evaluation results, the user's current skill level is recorded.

[1519] Step 6:

[1520] The user uses the terminal to input the goal and the date of achievement into the server and transmit it.

[1521] Specifically, set a goal such as "aiming for 800 points on the TOEIC."

[1522] Step 7:

[1523] The server generates an optimal learning program based on the user's current level and set goals.

[1524] The program includes practice tailored to the skills of reading, listening, writing and speaking.

[1525] Step 8:

[1526] The server transmits the generated learning program to the terminal.

[1527] The user checks and practices the learning program on the device.

[1528] Step 9:

[1529] The server generates daily learning content using a generative AI model and sends it to the device.

[1530] The user works on the day's learning content (for example, listening questions and writing assignments) on the terminal.

[1531] Step 10:

[1532] The user inputs the results of that day's learning into the terminal and sends them to the server.

[1533] The server receives this and analyzes it.

[1534] Step 11:

[1535] The server evaluates the user's progress based on the learning results.

[1536] Based on the evaluation results, the learning content for the next day is adjusted and provided to the user.

[1537] Step 12:

[1538] The server generates questions for regular level checks as needed and sends them to the terminal.

[1539] The user answers this and sends the results to the server.

[1540] Step 13:

[1541] The server analyzes the results of regular level checks and evaluates the user's latest English level.

[1542] The learning program is reconstructed based on the evaluation results and sent to the terminal.

[1543] Step 14:

[1544] The cycle repeats when the user continues learning with a new learning program and logs in again.

[1545] Through these processing steps, the user is always provided with a study plan that is optimized for his or her level and progress, and is able to efficiently improve his or her English skills.

[1546] Example 1

[1547] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1548] Conventional learning support systems have difficulty dynamically and flexibly adjusting learning programs according to the user's learning progress. Furthermore, they lack automation to provide individualized learning programs based on each user's level and goals, making it difficult to effectively improve users' English skills.

[1549] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1550] In this invention, the server includes: means for generating questions to assess a user's level and creating a study program based on the user's answers; means for generating daily study content and providing it to the user; means for analyzing the user's answer data and reconstructing the study program as needed; means for conducting periodic level checks as needed and adjusting the study program based on the evaluation; means for the user to access the server through a terminal, input authentication information, and the server to verify the login against a database; means for the server to generate questions using a generative AI model and present them to the user through the terminal; and means for the server to analyze the user's study results and generate feedback to adjust the study content for the next day, thereby enabling efficient and effective improvement of English skills according to the individual needs of the user.

[1551] "User" refers to a person who uses the learning support system.

[1552] "Terminal" refers to a device such as a computer or smartphone that a user uses to access the system, display learning content, and enter answers.

[1553] A "server" refers to a computer system that manages the entire system and performs various processes in cooperation with a database.

[1554] A "generative AI model" is a model that uses artificial intelligence technology and refers to a program used to generate learning content and analyze responses.

[1555] "Questions" refer to problems and tasks generated by the system to assist users in assessing their level and learning.

[1556] "Study program" means a set of learning activities or tasks designed to engage a user in English language learning.

[1557] "Response data" refers to the response information entered by the user in response to questions or learning content.

[1558] "Evaluation" refers to the process of determining the progress and level of learning based on the user's response data.

[1559] "Feedback" refers to information about areas for improvement and next learning content provided to the user based on the results of the server's analysis.

[1560] "Login" refers to the process by which a user enters authentication information to gain access to a system.

[1561] "Learning content" refers to the specific tasks and learning materials that users must address.

[1562] "Regular level checks" refers to tests or questions administered to periodically assess a user's learning progress.

[1563] The self-contained English skill improvement learning support system of the present invention is implemented as follows: This system is made up of a server, terminals, and users, and each element operates in cooperation with each other.

[1564] The server includes a means for requiring a user to enter authentication information when accessing the system through a terminal and for approving the login by checking this information against a database, thereby allowing the user to access the system. For example, the process involves a user entering an email address and password on a login screen, which is then verified by the server.

[1565] The server provides a means to generate questions using a generative AI model (e.g., GPT-4) to assess the user's level. The generated questions assess the four skills of reading, writing, listening, and speaking individually. This allows for a comprehensive understanding of the user's skill level. For example, intermediate-level written questions are provided for reading, and easy audio questions are provided for listening.

[1566] Users use their devices to answer questions and send the results to the server. The server analyzes the answers and uses NLP technology to assess the user's initial skill level. The server also includes a means to customize a learning program based on the user's set goal (e.g., "800 points on the TOEIC") and the date by which it is achieved. The program includes intermediate-level problem sets to strengthen reading and beginner-level audio materials to improve listening comprehension.

[1567] The server generates daily learning content using a generative AI model and provides a means to send it to the device. For example, "Today's listening questions" and "Today's writing assignments" are presented. The user can check and work on these learning content through the device.

[1568] The user inputs the results of their study into their device and sends them to the server. The server analyzes the user's response data and evaluates the day's study results. The system includes a means for adjusting the study content for the next day based on this evaluation. Specifically, if the user correctly answers a beginner-level listening question, a slightly more difficult question will be presented the next day.

[1569] Additionally, the server provides a means to conduct regular level checks as needed. These checks also use the generative AI model to generate questions and present them to the user. The server then analyzes the user's answers again to assess their current skill level. Based on the results, it reconstructs the learning program and sends it to the device. This ensures that the user's learning plan is always up to date.

[1570] An example of a specific prompt might be, "The user wants to learn intermediate-level reading comprehension questions to improve their reading. Please generate appropriate questions." This prompt is input into a generative AI model, which then generates appropriate reading questions.

[1571] In this way, this system provides a high level of collaboration between the server, terminals, and users, enabling efficient and effective improvement of English skills according to the individual needs of each user.

[1572] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1573] System program processing flow

[1574] Step 1: User Login and Authentication

[1575] Input: The user enters their email address and password into the terminal and presses the login button.

[1576] Action: The device sends the entered authentication information to the server.

[1577] Processing: The server checks the authentication information against the database and authorizes the user to log in.

[1578] Output: Successful login information is sent to the terminal and displayed to the user.

[1579] Step 2: Generate questions for initial level check

[1580] Input: After the server verifies the user's login, it starts the initial level check.

[1581] How it works: The server inputs prompts into the generative AI model, which generates questions for reading, writing, listening, and speaking.

[1582] Processing: The generative AI model generates a question based on the prompt and returns the result to the server.

[1583] Output: The generated questions are sent to the terminal and displayed to the user.

[1584] Step 3: Conduct an initial level check

[1585] Input: The user answers the questions and enters the answer data into the terminal.

[1586] Operation: The device sends the response data to the server.

[1587] Processing: The server analyzes the response data and evaluates the user's initial level for each skill (reading, writing, listening, and speaking).

[1588] Output: Evaluation results and feedback are sent to the device.

[1589] Step 4: Set goals and create a learning program

[1590] Input: The user inputs the goal (e.g., "800 points on the TOEIC") and the deadline for achieving it into the terminal and submits it.

[1591] Operation: The device sends the goal and deadline information to the server.

[1592] Processing: The server generates a learning program based on the user's initial level and goals. In this process, the generative AI model is used again.

[1593] Output: The customized learning program is sent to the terminal and displayed to the user.

[1594] Step 5: Provide daily learning content

[1595] Input: The server generates today's learning content based on the learning program.

[1596] How it works: The generative AI model generates learning content based on the prompt.

[1597] Processing: The server sends the generated learning content to the terminal.

[1598] Output: Today's learning content will be displayed on the terminal.

[1599] Step 6: Input and analysis of training results

[1600] Input: The user works on the learning content and inputs the results into the terminal.

[1601] Operation: The device sends the learning results to the server.

[1602] Processing: The server analyzes the learning results, evaluates the learning outcomes of the day, and generates feedback to adjust the learning content for the next day.

[1603] Output: Analysis results and applied feedback are sent to the device.

[1604] Step 7: Regular level checks and program restructuring

[1605] Input: After a certain period of time has passed, the server generates questions for regular level checks.

[1606] How it works: The regenerative AI model generates a question based on the prompt.

[1607] Processing: The server sends the generated questions to the terminal, and the user answers them. The answer data is sent to the server and analyzed.

[1608] Output: The latest skill level evaluation results are sent to the terminal, and the learning program is reconstructed.

[1609] Through the above processing steps, the system of the present invention efficiently and effectively supports users in improving their English skills in accordance with their individual needs.

[1610] (Application example 1)

[1611] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1612] Improving listening and speaking skills in English learning is a difficult task for many learners. In particular, there is a demand for efficient use of time while riding in an autonomous vehicle to improve English skills. To solve this problem, an effective English learning support system using smart devices is required.

[1613] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1614] In this invention, the server includes a means for generating questions to assess the learner's level and creating a learning program based on the learner's answers, a means for generating daily learning content and providing it to the learner, and a means for analyzing the learner's answer data and reconstructing the learning program as needed, thereby enabling a means for receiving and analyzing voice input using a smart device, a means for dynamically generating questions and answers using a generative AI model, and a means for providing audio and video content to the learner using a smart device.

[1615] "Learner" refers to an individual who uses the System to improve their English language skills.

[1616] "Level assessment questions" are questions that are asked to understand the learner's current level of reading, writing, listening, and speaking skills.

[1617] A "learning program" is a plan that combines the practice and materials necessary for a learner to achieve their desired English skills.

[1618] A "generative AI model" is an algorithm that uses artificial intelligence technology to automatically generate optimal learning content and questions.

[1619] "Smart devices" refer to electronic devices that can connect to the Internet and have voice input and video display functions, and examples include smartphones and smart glasses.

[1620] "Means for receiving and analyzing voice input" refers to the function of collecting learners' speech through the microphone of a smart device and recognizing and analyzing its content.

[1621] "Means for reviewing and dynamically providing learning content" refers to a function that generates and delivers optimal practice questions and teaching materials in real time according to the learner's level and progress.

[1622] "Means for dynamically generating questions and answers" refers to a function that uses a generative AI model to generate appropriate questions and answers in real time according to the learner's level and learning progress.

[1623] "Means for providing audio and video content" refers to a function that supports learning by allowing learners to listen to audio or watch videos using smart devices.

[1624] "Regular level checks" refer to tests conducted at regular intervals to assess learners' progress and determine their current English skill level.

[1625] The English conversation learning support system using smart devices of the present invention is implemented as follows: This system is made up of a server, a terminal, and a user, and each element works in cooperation with each other.

[1626] 1. Initial setup and level check

[1627] The user accesses the system using their own terminal (smart device) and logs in.

[1628] The server verifies the user's credentials and approves the login.

[1629] The server uses a generative AI model to generate questions for an initial level check, which assess each of the four skills: reading, writing, listening, and speaking.

[1630] The user answers the questions and sends the answers from the terminal to the server.

[1631] The server analyzes the user's response data and evaluates the user's initial level in each skill, thereby understanding the learner's current level.

[1632] 2. Setting goals and creating a learning program

[1633] The user inputs the goal (e.g., "to acquire English conversation skills at a daily conversation level") and the date by which it will be achieved from the terminal and sends it to the server.

[1634] The server generates an optimal learning program based on the user's current level and set goals. This program includes problem sets and audio materials to improve listening and speaking skills.

[1635] 3. Providing daily learning content and analyzing learning results

[1636] The server generates daily learning content using a generative AI model and sends it to the device.

[1637] The user checks the learning content through the device and works on each task. For example, "Today's listening questions" and "Today's speaking tasks" are presented.

[1638] The user responds by voice input, and the voice data is sent from the terminal to the server.

[1639] The server analyzes the audio data and evaluates the learning outcomes for that day, and the learning content for the next day is adjusted based on this evaluation.

[1640] 4. Regular level checks and program restructuring

[1641] The server periodically generates level check questions and sends them to the terminal.

[1642] The user answers the questions and sends the answer data from the terminal to the server.

[1643] The server analyzes the response data and evaluates the user's current English level.

[1644] The server reconstructs the learning program based on the latest evaluation and sends it to the terminal, allowing the user to always receive a program that meets their latest learning needs.

[1645] Hardware and software used

[1646] Hardware: Smart devices (smartphones, smart glasses, etc.)

[1647] Software: Generative AI models, speech recognition libraries (e.g., speech_recognition)

[1648] The system uses a generative AI model to dynamically generate questions and collects and analyzes user speech data through speech recognition, allowing users to efficiently improve their English conversation skills.

[1649] Specific examples

[1650] When a user uses this system to learn English conversation while riding in an autonomous vehicle, the following prompt sentences are generated:

[1651] "Please generate a list of English conversation phrases suitable for a beginner level. These phrases should be commonly used in travel situations."

[1652] For example, a passenger repeats the following phrase presented as "Today's Listening Question."

[1653] "How can I get to the nearest train station?"

[1654] Once the passenger has spoken correctly, their speech is analyzed and the next phrase is presented.

[1655] "Thank you! Let's move on to the next phrase."

[1656] In this way, you can efficiently improve your English conversation skills while managing your progress.

[1657] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1658] Step 1:

[1659] Initial setup and level check

[1660] When a user accesses the system using their own device, the server verifies the user's authentication information and approves the login. Next, it uses the generative AI model to generate questions for an initial level check and sends them to the device. The user answers the questions and sends the answers from the device to the server. The server then analyzes the user's answer data and evaluates the user's initial level in each skill. In this process, the user's answers are the input data, the generative AI model generates questions, and the server analyzes the data to evaluate the learner's level.

[1661] Step 2:

[1662] Setting goals and creating learning programs

[1663] The user inputs their desired goal (e.g., "Acquire English conversation skills at a daily conversation level") and the date by which they will achieve this goal from their device and sends it to the server. The server generates an optimal learning program based on the user's current level and the set goal. Specifically, it uses a generative AI model to dynamically create problem sets and audio materials to improve listening and speaking skills. In this process, the user's goal and current level are input data, and the generative AI model is used to generate a learning program, which outputs an optimal learning plan.

[1664] Step 3:

[1665] Providing daily learning content and analyzing learning results

[1666] The server generates daily learning content using a generative AI model and sends it to the device. The user checks the learning content through the device and works on each assignment. The user answers through voice input, and the voice data is sent from the device to the server. The server analyzes the voice data and evaluates the learning outcome for that day. Based on this evaluation, the learning content for the next day is adjusted. In this process, the user's voice data is the input data, and the server analyzes and evaluates the learning outcome, adjusting the learning content for the next day to obtain the output.

[1667] Step 4:

[1668] Regular level checks and program restructuring

[1669] The server periodically generates level-check questions and sends them to the terminal. The user answers the questions and sends the answer data from the terminal to the server. The server analyzes the answer data and evaluates the user's latest English level. The server reconstructs a learning program based on the latest evaluation and sends it to the terminal. This allows the user to always receive a program that meets their latest learning needs. In this process, the user's answer data is the input data, and the server analyzes it to evaluate the user's latest English level, then generates and outputs a new learning program.

[1670] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1671] The self-contained English skill improvement learning support system of this invention achieves a higher level of personalization by combining a conventional learning support system with an emotion engine that recognizes the user's emotions. The system is composed of a server, terminals, and users, and each element works in cooperation with each other.

[1672] 1. Initial setup and level check

[1673] A user accesses the system using their terminal and logs in.

[1674] The server verifies the user's credentials and approves the login.

[1675] The server uses a generative AI model to generate questions for an initial level check, which automatically generate questions to assess the four skills of reading, writing, listening, and speaking.

[1676] The user answers the questions and sends the answers from the terminal to the server.

[1677] The server analyzes the user's response data and evaluates the user's initial level in each skill, thereby understanding the learner's current level.

[1678] 2. Setting goals and creating a learning program

[1679] The user inputs the goal they are aiming for (e.g., "Aiming for 800 points on the TOEIC") and the date by which they will achieve it on their device, and sends it to the server.

[1680] The server generates an optimal learning program based on the user's current level and set goals. This program includes practice content for each skill required to achieve the user's goal.

[1681] For example, it includes problem sets to strengthen reading skills and audio materials to improve listening skills.

[1682] 3. Providing daily learning content and analyzing learning results

[1683] The server generates daily learning content using a generative AI model and sends it to the device.

[1684] Users check the learning content through their devices and work on each assignment.

[1685] For example, "Today's listening questions" and "Today's writing assignments" are presented.

[1686] The user inputs the learning results and answers and sends them from the terminal to the server.

[1687] The server receives this and analyzes it.

[1688] 4. Implementation and Use of Emotion Engine

[1689] The server uses an emotion engine to recognize the user's emotions in real time, specifically assessing the user's current emotions (e.g., stress, concentration, fatigue, etc.) through facial recognition technology and voice analysis.

[1690] The server then adjusts learning content and break suggestions based on the recognized emotion data. For example, if the user feels tired, a pop-up will appear recommending a break.

[1691] The user accepts the sentiment-based suggestions and chooses whether to pause or continue learning.

[1692] 5. Regular level checks and program restructuring

[1693] The server periodically generates level check questions and sends them to the terminal.

[1694] The user answers the questions and sends the answer data from the terminal to the server.

[1695] The server analyzes the response data and evaluates the user's current English level.

[1696] The server reconstructs the learning program based on the latest evaluation and sends it to the terminal, allowing the user to always receive a program that meets their latest learning needs.

[1697] Specific examples

[1698] For example, consider a case where a user sets a goal of "aiming for 800 points on the TOEIC." In the initial level check, the user's reading ability is assessed as intermediate and their listening ability as beginner. In this case, the server provides intermediate-level reading comprehension questions to strengthen reading ability, while constructing a program that includes many beginner-level audio materials for listening ability.

[1699] During daily learning, the emotion engine detects the user's stress level, and if high stress is detected, the server reduces the learning content and switches to more relaxing content. If the user feels tired, a notification is displayed recommending a break.

[1700] Regular level checks evaluate progress since the last level check and check how much reading and listening skills have improved. Based on the results, a new learning program is set up so that users can continue to study appropriately toward their goals.

[1701] In this way, the system constantly monitors the user's progress and emotional state, dynamically optimizing learning content to improve English skills efficiently and effectively.

[1702] The processing flow will be explained below.

[1703] Step 1:

[1704] A user accesses the system using their terminal and logs in.

[1705] The server verifies the user's credentials and approves the login.

[1706] Step 2:

[1707] The server uses a generative AI model to generate questions for an initial level check.

[1708] Specifically, it automatically generates questions to assess the four skills of reading, writing, listening, and speaking.

[1709] Step 3:

[1710] The server sends the generated question to the terminal.

[1711] The user checks the questions on the terminal and enters the answers.

[1712] Step 4:

[1713] The user sends the answer data from the terminal to the server.

[1714] The server receives the user's response data and analyzes it.

[1715] Step 5:

[1716] The server evaluates the user's initial level of the four skills based on the response data.

[1717] Based on the evaluation results, the user's current skill level is recorded.

[1718] Step 6:

[1719] The user uses the terminal to input the goal and the date of achievement into the server and transmit it.

[1720] Specifically, set a goal such as "aiming for 800 points on the TOEIC."

[1721] Step 7:

[1722] The server generates an optimal learning program based on the user's current level and set goals.

[1723] The program includes practice tailored to the skills of reading, listening, writing and speaking.

[1724] Step 8:

[1725] The server transmits the generated learning program to the terminal.

[1726] The user checks and practices the learning program on the device.

[1727] Step 9:

[1728] The server generates daily learning content using a generative AI model and sends it to the device.

[1729] The user works on the day's learning content (for example, listening questions and writing assignments) on the terminal.

[1730] Step 10:

[1731] The user inputs the results of that day's learning into the terminal and sends them to the server.

[1732] The server receives this and analyzes it.

[1733] Step 11:

[1734] The server evaluates the user's progress based on the learning results.

[1735] Based on the evaluation results, the learning content for the next day is adjusted and provided to the user.

[1736] Step 12:

[1737] The server uses an emotion engine to recognize the user's emotions in real time.

[1738] Specifically, it assesses the user's current emotions (e.g., stress, concentration, fatigue, etc.) through facial recognition technology and voice analysis.

[1739] Step 13:

[1740] The server adjusts learning content and break suggestions based on the recognized emotion data.

[1741] For example, if stress levels are high, the system will provide less difficult questions and switch to content that has a relaxing effect.

[1742] Step 14:

[1743] The user chooses whether to accept sentiment-based suggestions.

[1744] If the user feels they need a break, they can pause their learning and refresh.

[1745] Step 15:

[1746] The server generates questions for regular level checks as needed and sends them to the terminal.

[1747] The user answers this and sends the results to the server.

[1748] Step 16:

[1749] The server analyzes the results of regular level checks and evaluates the user's latest English level.

[1750] The learning program is reconstructed based on the evaluation results and sent to the terminal.

[1751] Step 17:

[1752] The user continues learning with a new learning program.

[1753] When you log in again, the cycle repeats.

[1754] Through these processing steps, users are always provided with a learning plan that is optimized for their level, progress, and emotional state, allowing them to efficiently improve their English skills.

[1755] Example 2

[1756] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1757] Conventional learning support systems provide learning programs tailored to a learner's current level and goals, but they are unable to recognize the learner's emotional state and adjust the learning content based on that emotion. This creates the problem that appropriate support is not provided even when the learner feels fatigued or stressed, making efficient learning difficult. In addition, rebuilding learning programs and conducting regular level checks is cumbersome, creating a need for automatic dynamic adjustments.

[1758] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1759] In this invention, the server includes means for generating questions to assess the learner's level and creating a learning program based on the learner's answers, means for generating daily learning content and providing it to the learner, means for recognizing the learner's emotional state in real time and adjusting the learning content and break suggestions, means for analyzing the learner's answer data and reconstructing the learning program as appropriate, and means for conducting regular level checks as necessary and adjusting the learning program based on the evaluation, thereby enabling efficient and personalized learning support that takes the learner's emotional state into consideration.

[1760] The term "learner" refers to an individual who uses the learning system for the purpose of improving a particular skill (mainly language skill in the present invention).

[1761] A "server" refers to a computer system that provides various services to learners via a network, and in the present invention, it mainly generates and provides learning programs.

[1762] "Question" refers to a question or problem generated by the server to assess a learner's skill or level.

[1763] "Learning program" refers to a set of educational curriculum and assignments created by the server based on the learner's current level and goals.

[1764] "Learning content" refers to specific learning tasks and learning materials that are generated by the server daily and provided to learners.

[1765] "Response data" refers to the responses entered by learners to questions and assignments.

[1766] "Emotional state" refers to the learner's current psychological or emotional state (e.g., stress, concentration, fatigue).

[1767] An "emotion engine" refers to a system that uses facial recognition technology, voice analysis technology, and other techniques to recognize a learner's emotional state in real time.

[1768] "Dynamic adjustment" refers to changing and optimizing learning programs and content in real time based on the learner's progress and emotional state.

[1769] "Level check" refers to a test or assessment to assess a learner's current skill level.

[1770] This invention relates to a self-contained English skill improvement learning support system that assesses learners' levels and creates and provides learning programs. The system consists of a server, terminals, and learners, and each element works in conjunction with the others. The server uses a generative AI model to generate questions and learning content, and an emotion engine to recognize the learner's emotional state and dynamically adjust the learning content.

[1771] Initial setup and level check

[1772] The learner accesses the system using a device and logs in. The server verifies the learner's authentication information and approves the login. The server uses a generative AI model to generate questions to assess the four skills of reading, writing, listening, and speaking. The learner answers the questions and sends the data from their device to the server. The server analyzes the response data and evaluates the initial level of each skill to determine the learner's current skill level.

[1773] Setting goals and creating learning programs

[1774] Learners input their goal (e.g., "Aim for 800 points on the TOEIC") and the target date for achieving it on their device and send it to the server. The server then generates an optimal learning program based on the learner's current level and the set goal. This program might include, for example, a problem set to improve reading skills or audio materials to improve listening ability.

[1775] Providing daily learning content and analyzing learning results

[1776] The server uses a generative AI model to generate daily learning content and sends it to the device. The learner checks the learning content through the device and works on each task. For example, they are presented with a "today's listening question" or a "today's writing task." The learner inputs their learning results and answers and sends them from their device to the server. The server analyzes the received data and evaluates the learner's progress.

[1777] Implementing and utilizing an emotion engine

[1778] The server uses an emotion engine to recognize the learner's emotional state in real time. Specifically, it uses facial recognition technology and voice analysis to assess the learner's current emotions (e.g., stress, concentration, fatigue, etc.). The server adjusts learning content and break suggestions based on the emotion data. For example, if the learner feels fatigued, a pop-up recommending a break will be displayed. The learner can accept the emotion-based suggestion and choose whether to pause or continue learning.

[1779] Regular level checks and program restructuring

[1780] The server periodically generates level-check questions and sends them to the device. The learner answers the questions and sends the answer data from the device to the server. The server analyzes the data and evaluates the learner's current English level. Based on this evaluation, the server reconstructs the learning program and sends it to the device. This ensures that the learner always receives a program that meets their latest learning needs.

[1781] Specific examples and examples of prompts for the generative AI model

[1782] For example, consider a case where a learner sets a goal of "achieving 800 points on the TOEIC." Suppose that the initial level check assesses reading as intermediate and listening as beginner. In this case, the server provides intermediate-level reading comprehension questions to strengthen reading, while constructing a program that includes many beginner-level audio materials for listening. During daily learning, if the emotion engine detects high stress, the server reduces the learning content and switches to content with a relaxing effect. Furthermore, if the learner feels tired, a notification is displayed recommending a break.

[1783] Example prompts for generative AI models

[1784] "A student is aiming for 800 points on the TOEIC, and their current reading level is intermediate and their listening level is beginner. Please generate daily study content based on the student's progress."

[1785] In this way, the system constantly monitors learners' progress and emotional state, dynamically optimizing learning content to help them improve their English skills efficiently and effectively.

[1786] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1787] Step 1:

[1788] A user accesses the system using their terminal and logs in.

[1789] Input: User ID and password.

[1790] How it works: The user opens a browser or a dedicated app, accesses the login screen, and enters their user ID and password.

[1791] Output: The entered authentication information is sent to the server.

[1792] Step 2:

[1793] The server verifies the user's credentials and approves the login.

[1794] Input: User ID and password.

[1795] How it works: The server accesses the database and checks the entered credentials.

[1796] Output: If authentication is successful, the login is approved and the user is shown the dashboard.

[1797] Step 3:

[1798] The server uses a generative AI model to generate questions for an initial level check.

[1799] Input: User login information.

[1800] How it works: The server sends prompts to a generative AI model to generate questions to assess the four skills of reading, writing, listening, and speaking.

[1801] Output: The generated questions are sent to the user's terminal.

[1802] Step 4:

[1803] The user answers the questions and sends the answers from the terminal to the server.

[1804] Input: The generated question.

[1805] How it works: The user reviews the questions on the terminal and enters answers to each question.

[1806] Output: The answer data is sent from the device to the server.

[1807] Step 5:

[1808] The server analyzes the user's response data and evaluates the user's initial level in each skill.

[1809] Input: User response data.

[1810] How it works: The server uses natural language processing (NLP) technology to analyze the response data and assess the initial level of each skill (reading, writing, listening, and speaking).

[1811] Output: User's initial level assessment result.

[1812] Step 6:

[1813] The user inputs the goal and the date of achievement from the terminal and transmits it to the server.

[1814] Input: Your goal and the date you want to reach it.

[1815] How it works: The user enters the goal and target achievement date into a dedicated input form and presses the submit button.

[1816] Output: The goal and the target date are sent to the server.

[1817] Step 7:

[1818] The server generates an optimal learning program based on the user's current level and set goals.

[1819] Input: User's initial level assessment results and goal / target achievement date.

[1820] How it works: The server sends prompts to the generative AI model to generate a learning program appropriate for the user's current level and goals.

[1821] Output: The generated learning program is sent to the user's terminal.

[1822] Step 8:

[1823] The server generates daily learning content using a generative AI model and sends it to the device.

[1824] Input: The user's study program.

[1825] How it works: The server sends prompts to the generative AI model to generate learnings for the day.

[1826] Output: The generated learning content is sent to the user's device.

[1827] Step 9:

[1828] Users check the learning content through their devices and work on each assignment.

[1829] Input: What you learned that day.

[1830] Action: The user checks the learning content on the device and works on the assignments. For example, they play an audio file and answer listening questions.

[1831] Output: Learning results and answer data.

[1832] Step 10:

[1833] The user inputs the learning results and answers and sends them from the terminal to the server.

[1834] Input: Learning results and answer data.

[1835] Operation: The user enters the learning results and answers and presses the submit button.

[1836] Output: Results and response data are sent to the server.

[1837] Step 11:

[1838] The server uses an emotion engine to recognize the learner's emotional state in real time.

[1839] Input: Learner's video and audio data.

[1840] Operation: The server runs the emotion engine to assess the learner's current emotional state through facial recognition technology and voice analysis.

[1841] Output: Emotional state assessment results.

[1842] Step 12:

[1843] The server adjusts learning content and break suggestions based on the recognized emotion data.

[1844] Input: Emotional state assessment results.

[1845] How it works: The server adjusts the difficulty of the learning content and suggests breaks based on the user's emotional state. For example, if the user is highly fatigued, a pop-up will appear recommending a break.

[1846] Output: Tailored learning content and break suggestions.

[1847] Step 13:

[1848] The user chooses whether to accept sentiment-based suggestions.

[1849] Input: Study content and break suggestions.

[1850] Action: The user reviews the suggestions and selects "Take a break" or "Continue learning."

[1851] Output: Next action based on user selection.

[1852] Step 14:

[1853] The server periodically generates level check questions and sends them to the terminal.

[1854] Input: Learner progress data.

[1855] How it works: The server uses the generative AI model to generate questions for regular level checks.

[1856] Output: Generated level check questions.

[1857] Step 15:

[1858] The user answers the questions and sends the answer data from the terminal to the server.

[1859] Input: Questions for regular level checks.

[1860] Operation: The user answers the questions and sends the answer data from the terminal to the server.

[1861] Output: The answer data is sent to the server.

[1862] Step 16:

[1863] The server analyzes the response data and evaluates the user's current English level.

[1864] Input: Response data.

[1865] How it works: The server analyzes the response data and evaluates your current English level.

[1866] Output: Latest English level assessment results.

[1867] Step 17:

[1868] The server reconstructs the learning program based on the latest evaluation and sends it to the device.

[1869] Input: Your most recent English level assessment results.

[1870] How it works: The server uses the generative AI model to reconstruct a learning program based on the latest English level.

[1871] Output: A reconstructed learning program.

[1872] (Application example 2)

[1873] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1874] Conventional learning support systems provide learning content mechanically without considering the learner's emotional state, resulting in problems such as reduced learning efficiency due to learner fatigue or stress. While these systems dynamically adjust learning programs based on the learner's progress and ability, they do not monitor the learner's emotional state, resulting in an insufficient optimization of the individual learning experience. In particular, in work environments using robots, such as factories, a system is needed that can optimize work efficiency according to the worker's emotional state.

[1875] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: a means for generating questions to evaluate the learner's level and creating a learning program based on the learner's answers; a means for generating daily learning content and providing it to the learner; a means for analyzing the learner's answer data and reconstructing the learning program as needed; a means for conducting periodic level checks as needed and adjusting the learning program based on the evaluation; and a means for recognizing the learner's emotional state in real time and dynamically optimizing the learning content based on the learner's emotional state. This makes it possible to provide a learning program that takes the learner's emotional state into consideration, thereby reducing learner stress and fatigue and improving learning efficiency. Furthermore, by applying this to factory work sites, it is possible to optimize work schedules and robot operations according to the worker's emotional state.

[1876] "Assessing the learner's level" means measuring the level of knowledge and skills that the learner currently possesses and providing appropriate learning content and programs based on that.

[1877] "Question generation" means automatically creating questions or problems to assess a learner's ability.

[1878] "Creating a learning program" means planning and designing learning content and activities based on the learner's goals and current abilities.

[1879] "Generating daily learning content" means automatically creating and providing learning tasks and materials that learners should tackle every day.

[1880] "Providing to learners" means displaying and transmitting learning content and assignments to the devices and systems used by learners.

[1881] "Analyzing learner response data" means analyzing the responses and results submitted by learners and evaluating their learning progress and level of understanding based on the results.

[1882] "Reconstructing learning programs in a timely manner" means dynamically updating and redesigning learning content and programs according to learners' progress and assessment results.

[1883] "Conducting regular level checks" means administering tests and questions to assess learners' abilities on a regular basis.

[1884] "Recognizing the learner's emotional state in real time" means instantly assessing the learner's current emotional and psychological state using information such as their facial expressions and voice.

[1885] "Dynamic optimization of learning content based on emotional state" means flexibly adjusting the learning content and difficulty level by reflecting the learner's emotional and psychological state.

[1886] "Facial recognition technology" is a technology that detects an individual's face from camera footage and analyzes their facial expressions and characteristics.

[1887] "Voice analysis technology" is a technology that analyzes voice data and evaluates its content and the speaker's emotional state.

[1888] "Detecting fatigue and stress" means determining the level of fatigue and stress experienced by learners through facial recognition and voice analysis.

[1889] "Displaying a notification recommending a break" means that when the system detects that the learner is fatigued or stressed, a message urging the learner to take a break will be displayed on the device.

[1890] The present invention provides a learning support system that recognizes the emotional state of a learner in real time and dynamically optimizes the generation and provision of learning programs. The system is composed of a server, terminals, and users.

[1891] 1. Initial setup and level check

[1892] A user accesses the system using a terminal and logs in. The server verifies the user's authentication information and approves the login. The server then uses a generative AI model to generate questions for an initial level check. These questions are designed to assess the four skills of reading, writing, listening, and speaking. The user answers the questions and sends the answers from the terminal to the server. The server analyzes the user's response data and evaluates the initial level in each skill.

[1893] 2. Setting goals and creating a learning program

[1894] The user inputs their goal (for example, "Aiming for 800 points on the TOEIC") and the date they wish to achieve it, and sends it to the server. The server then generates an optimal learning program based on the user's current level and the goal they set. This program includes practice content for each skill the user needs to achieve their goal.

[1895] 3. Providing daily learning content and analyzing learning results

[1896] The server uses a generative AI model to generate daily learning content and sends it to the device. The user checks the learning content through the device and works on each task. The device then sends the learning results and answers to the server, which receives and analyzes them.

[1897] 4. Implementation and Use of Emotion Engine

[1898] The server uses an emotion engine to recognize the user's emotions in real time. The hardware used includes a camera and microphone. The software uses OpenCV for facial recognition technology and TensorFlow / Keras for emotion recognition models. Using these technologies, the server evaluates the user's current emotions (e.g., stress, concentration, fatigue, etc.). Based on the recognized emotion data, the server adjusts the learning content and suggests breaks. For example, if the server recognizes that the user is feeling fatigued, it will display a notification on the device recommending a break.

[1899] 5. Regular level checks and program restructuring

[1900] The server periodically generates level-check questions and sends them to the device. The user answers the questions and sends the answer data from the device to the server. The server analyzes the answer data and evaluates the user's latest English level. Based on the latest evaluation, the server reconstructs the learning program and sends it to the device. This ensures that the user always receives a program that meets their latest learning needs.

[1901] Specific examples

[1902] For example, consider a user who sets a goal of "achieving 800 points on the TOEIC." Suppose the user's reading and listening skills are assessed as intermediate and beginner levels in the initial level check. In this case, the server provides intermediate-level reading comprehension questions to strengthen reading, while constructing a program that includes many beginner-level audio materials for listening. During daily study, the emotion engine detects the user's stress level. If high stress is recognized, the server reduces the study content and switches to content with a relaxing effect. If the user feels tired, the server displays a notification recommending a break. Periodic level checks evaluate progress since the previous level check and confirm the degree of improvement in reading and listening skills. Based on the results, a new study program is created, allowing the user to continue studying appropriately toward their goal.

[1903] Prompt Sentence Examples

[1904] "Please explain how to recognize the emotions of factory workers in real time and optimize labor efficiency based on their emotions. I would like to build a system that sends emotional data to a server, which then adjusts optimal work schedules and robot operations. As an example, please explain in detail how you combine facial recognition technology with an emotion engine."

[1905] This allows learners' progress and emotional state to be constantly monitored, and learning content to be dynamically optimized, enabling them to improve their English skills efficiently and effectively.

[1906] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1907] Step 1:

[1908] A user accesses the system using a terminal and logs in. The user enters login information (user ID and password), which the terminal sends to the server. The server verifies the user's authentication information and approves the login. The input data is the user's authentication information, and the output data is the login success or failure status.

[1909] Step 2:

[1910] The server uses a generative AI model to generate questions for an initial level check. The server uses prompts to request questions to assess the skills of reading, writing, listening, and speaking from the generative AI model, and sends the generated questions to the terminal. The input data is the prompts, and the output data is the generated questions.

[1911] Step 3:

[1912] The user answers questions through the terminal and sends the answers to the server. The terminal acquires the user's input and sends the data to the server. The input data is the user's answer, and the output data is a notification that the answer data has been sent.

[1913] Step 4:

[1914] The server analyzes the user's response data and evaluates their initial level. The server then inputs the response data into an analysis algorithm to calculate a score for each skill. The input data is the user's response data, and the output data is the evaluation result of the initial level for each skill.

[1915] Step 5:

[1916] The user inputs the goal they are aiming for and the date by which they will reach it, and sends it to the server. The user inputs the goal they have set (for example, "Aim for 800 points on the TOEIC") and the date by which they plan to reach it from their terminal, and sends it to the server. The input data is the goal and the date by which it will be reached, and the output data is a notification that the goal has been set.

[1917] Step 6:

[1918] The server generates an optimal learning program based on the user's current level and set goals. Based on the initial level assessment results and goals, the server requests the AI ​​model to generate learning content suitable for the user, and creates that content. The input data is the initial level assessment results and the user's goals, and the output data is the generated learning program.

[1919] Step 7:

[1920] The server generates daily learning content using a generative AI model and sends it to the device. The server dynamically adjusts the daily learning content taking into account the user's progress and emotional state and sends it to the device. The input data is the user's progress and emotional state, and the output data is the generated daily learning program.

[1921] Step 8:

[1922] The user works on daily learning content through the terminal and inputs the results. The user performs the learning tasks displayed on the terminal and inputs the answers. The input data is the user's learning results, and the output data is a notification that the learning results have been sent.

[1923] Step 9:

[1924] The server receives and analyzes the learning result data. The server analyzes the received learning results and evaluates the progress. The input data is the learning result data, and the output data is the progress evaluation result.

[1925] Step 10:

[1926] The server uses an emotion engine to recognize the user's emotions in real time. The server then activates the emotion engine and analyzes the camera video and audio data to evaluate the user's emotional state. The input data are the camera video and audio data, and the output data are the emotion evaluation results.

[1927] Step 11:

[1928] The server adjusts the learning content and break suggestions based on the emotion evaluation results. The server dynamically adjusts the learning content based on the emotion evaluation results, and in some cases displays a notification on the device recommending a break. The input data is the emotion evaluation results, and the output data is the adjusted learning content and break suggestions.

[1929] Step 12:

[1930] The server periodically generates level check questions and sends them to the terminal. The server also generates questions to reassess the user's skills at regular intervals and sends them to the terminal. The input data is the prompt text, and the output data is the generated level check questions.

[1931] Step 13:

[1932] The user answers questions for the regular level check and sends the results to the server. The user answers questions and sends the data to the server. The input data is the answers to the regular level check, and the output data is a notification that the answer data has been sent.

[1933] Step 14:

[1934] The server analyzes the response data and evaluates the user's latest English level. The server analyzes the response data and evaluates the user's progress. The input data is the response data from the regular level check, and the output data is the latest English level evaluation result.

[1935] Step 15:

[1936] The server reconstructs the learning program based on the latest evaluation results and sends it to the terminal. The server generates a new learning program that reflects the latest evaluation results and sends it to the terminal. The input data is the latest evaluation results, and the output data is the reconstructed learning program.

[1937] This makes it possible to provide an optimal learning environment that takes into account the user's emotional state, maximizing learning efficiency.

[1938] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1939] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1940] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1941] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1942] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1943] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1944] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are express...

Claims

1. means for generating questions to assess the level of a learner and creating a learning program based on the learner's answers; A means of generating and delivering daily learning content to learners; A means for analyzing learner response data and reconstructing the learning program in a timely manner; A means to conduct regular level checks and adjust the study program based on the assessment, if necessary; A system including:

2. 2. The system of claim 1, wherein the generated questions and study programs cover at least four skills: reading, writing, listening, and speaking.

3. 10. The system of claim 1, wherein the learning program schedule is dynamically adjusted to meet the goals and achievement dates set by the learner.

Images