system

Abstract

Provide a system. [Solution] means for receiving English proficiency data from a user; means for assessing the user's skill level based on the received English proficiency data; A means for automatically selecting a training menu according to the skill level of the user; A means for obtaining schedule information of a user; A means for scheduling a training menu during free time in the acquired schedule information; means for sending a notification to a user terminal at the start time of a scheduled training menu; and means for automatically starting a learning application installed on a user terminal at a specified time.

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Previously, English learning systems struggled to flexibly provide training menus tailored to users' skill levels and schedules. In particular, they lacked mechanisms for providing appropriate learning content while effectively utilizing users' free time, making it difficult for users to develop consistent learning habits. Furthermore, there were limited means for tracking users' learning progress in real time and providing appropriate feedback. A system that can solve these problems and support effective English learning is needed. [Means for solving the problem]

[0005] The present invention provides a server that receives English proficiency data from a user and evaluates the user's skill level based on that data. It then automatically selects an optimal training menu based on the evaluation. It also includes a means for acquiring the user's schedule information and scheduling the training menu during available time. It also sends a notification to the user's terminal at the start time of the scheduled training menu and automatically launches a learning application at the specified time. It also acquires the user's progress data via the learning application and sends it to the server, providing feedback based on the user's learning progress. This allows the user to efficiently utilize their spare time to advance their English language study and develop consistent learning habits.

[0006] "User" refers to an individual who uses the System to study English.

[0007] "English Proficiency Data" refers to information such as scores and skill levels used to evaluate a user's English skills.

[0008] "Means for receiving" refers to the mechanism by which the server receives data sent by the user.

[0009] "Means for assessing skill level" refers to a process for determining a user's level of English language skill based on received English language proficiency data.

[0010] "Training menu" refers to the specific learning content and plan for the user to study English.

[0011] "Means for selection" refers to a mechanism for determining the optimal training menu based on the user's skill level.

[0012] "Schedule information" refers to information that indicates a user's daily schedule and free time.

[0013] "Means of acquisition" refers to the mechanism for acquiring the user's schedule information.

[0014] "Free time" refers to free time in a user's schedule that does not have any specific appointments.

[0015] "Scheduling means" refers to a mechanism for allocating training menus to specific times based on acquired schedule information.

[0016] "Means for sending notifications" refers to a system for notifying users of the start time of a scheduled training menu.

[0017] "User terminal" refers to an electronic device used by a User, such as a computer, smartphone, or tablet.

[0018] "Means for automatic startup" refers to a mechanism that automatically starts the learning application installed on the user's device at a specified time.

[0019] "Learning Application" means software that enables users to learn English.

[0020] "Progress Data" means data that indicates the progress and results of a User's learning using a Learning Application.

[0021] "Means for sending" refers to the mechanism for sending progress data from the user terminal to the server.

[0022] "Server" refers to a central computer that performs various processes based on data received from user terminals. [Brief explanation of the drawings]

[0023] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2]1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0024] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0025] First, the terms used in the following description will be explained.

[0026] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0027] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0028] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0029] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0030] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0031] [First embodiment]

[0032] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0033] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0034] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0035] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0036] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0037] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0038] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0039] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0040] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0041] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0042] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0043] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0044] ---

[0045] The present invention is a system for effectively supporting users' English learning, and aims to automatically provide optimal training menus based on English proficiency data entered by the user, and to efficiently manage learning time according to the user's schedule. This system is mainly composed of a server, a user terminal, and processing steps including the exchange of input data from the user.

[0046] Main functions and processes of the server

[0047] 1. User data reception function

[0048] When users initially set up the system, they input their TOEIC score and English skill level, and the server receives and records this data.

[0049] 2. Skill evaluation function

[0050] The server automatically evaluates the user's skill level based on the received English proficiency data. For example, if the TOEIC score is 650, the server will evaluate the user's skill level as "intermediate."

[0051] 3. Training menu selection function

[0052] The server automatically selects a training menu appropriate for the assessed skill level, for example, "intermediate reading training" or "intermediate listening training."

[0053] 4. Schedule management function

[0054] The server retrieves the user's schedule information, such as Google Calendar, and finds available time slots. It then schedules training sessions that fit the available time slots.

[0055] 5. Notification function

[0056] When the scheduled training session is about to begin, the server sends a notification to the user's device, with a message like "Your next training session will begin at 7:30."

[0057] Main functions and processing of the user terminal

[0058] 1. Notification reception function

[0059] The user device receives a notification sent from the server, which indicates the content and start time of the next learning session.

[0060] 2. App auto-start function

[0061] The device that receives the notification will automatically launch the English learning app at the specified time. For example, "Intermediate Reading Training" will start at 7:30 AM.

[0062] 3. Learning Progression Function

[0063] The user can then use the app to progress through the training, which will then record the progress of the training and the user's responses in real time.

[0064] 4. Progress data transmission function

[0065] At the end of a study session, the app sends the user's progress data to the server, such as "80% correct reading" or "75% correct listening."

[0066] User operations

[0067] 1. Data Entry

[0068] During initial setup, users enter their TOEIC score, English skill level, and Google Calendar connection information, which allows the server to suggest appropriate training menus and schedules.

[0069] 2. Notification confirmation

[0070] The user checks the notification sent from the server to find out when the next learning session will start.

[0071] 3. Study with apps

[0072] At the designated time, the user will use the automatically launched app to carry out the training, which will help develop a natural learning habit.

[0073] 4. Feedback confirmation

[0074] Once the training is complete, users can view their progress and feedback from the app.

[0075] Specific examples

[0076] For example, if a user with a TOEIC score of 650 provides input data to the system, the server will evaluate the user as "intermediate." The server then selects "intermediate reading training" and "intermediate listening training." It then checks Google Calendar to find available time between 7:30 and 8:00 a.m. on weekdays. The server schedules training for this time period and sends a notification to the user's device. At 7:30 a.m., the learning app installed on the user's device will automatically launch and the training will begin. When the user completes the training, progress data is sent to the server, which then provides feedback based on that data.

[0077] As described above, the present invention is a system that can provide flexible training according to the user's skill level and schedule, and can efficiently manage progress.

[0078] The processing flow will be explained below.

[0079] ---

[0080] Step 1:

[0081] The server receives the TOEIC score and English skill level data sent by the user. Specifically, the score and skill information entered by the user on the initial setting screen is sent to the server.

[0082] Step 2:

[0083] The server automatically evaluates the user's skill level based on the received English proficiency data. For example, if the TOEIC score is 650, the server evaluates the user's skill level as "intermediate."

[0084] Step 3:

[0085] The server automatically selects a training menu according to the user's skill level, such as "Intermediate Reading Training" or "Intermediate Listening Training."

[0086] Step 4:

[0087] The server requests permission to access the schedule information from the user to obtain the schedule information of the user's Google Calendar, etc. After the user grants permission, the server obtains the user's schedule data.

[0088] Step 5:

[0089] The server detects available times from the acquired schedule information. For example, it determines that available times are from 7:30 to 8:00 a.m. on weekdays.

[0090] Step 6:

[0091] The server schedules training sessions during the available times it finds, for example, allocating "Intermediate Reading Training" between 7:30 and 8:00 AM.

[0092] Step 7:

[0093] The server sends a notification to the user device 15 minutes before the scheduled training, with a message such as "The next learning session will start at 7:30."

[0094] Step 8:

[0095] The user terminal receives the notification from the server and displays a pop-up message to the user, who then confirms the start time.

[0096] Step 9:

[0097] At the specified time (7:30 AM in this case), the user's device automatically launches the English learning application, and the user begins training in the app.

[0098] Step 10:

[0099] The user progresses through the learning process by following the designated training menu, for example, answering questions in the "Intermediate Reading Training" section.

[0100] Step 11:

[0101] Once the learning session is over, the user's device records the training progress and results and sends them to the server.

[0102] Step 12:

[0103] The server analyzes the received progress data and provides feedback to the user, such as "Your accuracy rate for this reading training was 80%."

[0104] By following these steps, users can study English efficiently in their spare time and naturally develop a study habit. Real-time progress management and feedback also improve the effectiveness of users' learning.

[0105] Example 1

[0106] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0107] Conventional English learning systems have difficulty flexibly adapting to individual users' skill levels and schedules, preventing them from maximizing learning effectiveness. They also lacked a mechanism for recording users' progress in real time or the ability to provide feedback based on that progress. This prevented users from accurately understanding their own learning progress and results, hindering effective learning.

[0108] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0109] In this invention, the server includes means for receiving English proficiency data from a user, means for evaluating the user's skill level based on the received English proficiency data, means for automatically selecting a training menu according to the user's skill level, means for acquiring the user's schedule information, means for scheduling a training menu during free time in the acquired schedule information, means for sending a notification to the user terminal at the start time of the scheduled training menu, means for automatically starting a learning application installed on the user terminal at a specified time, means for the learning application to record the user's training progress in real time, means for sending user progress data to the server after training ends, and means for providing feedback based on the progress data to the user terminal. This enables efficient learning tailored to the user's schedule and skill level, and by accurately understanding the user's learning progress and receiving feedback, effective learning can be achieved.

[0110] "User" refers to an individual or organization that uses the English Learning System.

[0111] "English Proficiency Data" refers to data necessary for assessing a user's English proficiency, such as a user's TOEIC score or English skill level.

[0112] "Skill level" refers to a user's proficiency in understanding and practicing English as assessed based on English proficiency data.

[0113] "Training Menu" refers to a set of English learning sessions and assignments provided according to the user's skill level.

[0114] "Schedule information" refers to data such as calendar information that indicates the user's schedule and free time.

[0115] "Real-time" refers to the fact that progress and achievements are recorded instantly while a user is using a learning application.

[0116] "Progress data" refers to data that indicates the user's learning status, such as the results achieved by the user during a study session and the percentage of correct answers.

[0117] "Feedback" refers to information provided to the user based on progress data, such as indications of learning results and next goals.

[0118] "Server" refers to the central processing unit that receives and analyzes user data, selects training menus, and manages schedules.

[0119] "User terminal" refers to a computing device on which a user installs and uses a learning application.

[0120] "Learning Application" refers to software that a user uses to train their English language skills.

[0121] "Notification" refers to a message or alert sent from a server to a user device.

[0122] This invention is a system for effectively supporting users' English learning. This system provides an optimal training menu based on the user's English proficiency data and efficiently manages learning time according to the user's schedule. It is mainly composed of a server, a user terminal, and processing steps including the exchange of input data from the user.

[0123] Specific server operations

[0124] User data reception function

[0125] When users initially set up the system, they input their TOEIC score and English skill level, and the server receives and records this data.

[0126] Skill evaluation function

[0127] The server automatically evaluates the user's skill level based on the received English proficiency data. For example, if the TOEIC score is 650, the server will evaluate the user's skill level as "intermediate."

[0128] Training menu selection function

[0129] The server automatically selects a training menu according to the assessed skill level, for example, "intermediate reading training" or "intermediate listening training."

[0130] Schedule management function

[0131] The server retrieves the user's schedule information (e.g., Google Calendar), finds available time slots, and schedules training sessions that fit the available time slots.

[0132] Notification function

[0133] When the scheduled training session is about to begin, the server sends a notification to the user's device, for example, a message saying, "Your next training session will begin at 7:30."

[0134] Specific operation of the user terminal

[0135] Notification reception function

[0136] The user device receives a notification sent from the server, which indicates the content and start time of the next learning session.

[0137] App auto-start function

[0138] The device that receives the notification will automatically launch the English learning app at the specified time. For example, "Intermediate Reading Training" will start at 7:30 AM.

[0139] Learning Progression Function

[0140] The user can then use the app to progress through the training, which will then record the progress of the training and the user's responses in real time.

[0141] Progress data transmission function

[0142] At the end of a study session, the app sends the user's progress data to the server, such as "80% correct reading" or "75% correct listening."

[0143] Specific user operations

[0144] Data Entry

[0145] During initial setup, users enter their TOEIC score, English skill level, and Google Calendar connection information, which allows the server to suggest appropriate training menus and schedules.

[0146] Notification confirmation

[0147] The user checks the notification sent from the server to find out when the next learning session will start.

[0148] Learn with the app

[0149] At the designated time, the app will automatically start and the training will begin, making it easier to naturally develop a study habit.

[0150] Feedback Check

[0151] Once the training is complete, users can view their progress and feedback from the app.

[0152] Specific examples

[0153] For example, a user with a TOEIC score of 650 provides input data to the system. The server evaluates the user as "intermediate." The server then selects "intermediate reading training" and "intermediate listening training." After that, it checks Google Calendar to find available time slots between 7:30 and 8:00 a.m. on weekdays. The server schedules the training for this time period and sends a notification to the user's device. At 7:30 a.m., the learning app installed on the user's device automatically launches and the training begins. When the user completes the training, progress data is sent to the server, which then provides feedback based on that data.

[0154] As described above, the present invention is a system that can provide flexible training according to the user's skill level and schedule, and can efficiently manage progress.

[0155] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0156] Step 1:

[0157] Initial data entry by the user

[0158] The user opens the system's initial setup screen and enters their TOEIC score, English skill level, and Google Calendar connection information, which then sends the initial data to the server.

[0159] Input: TOEIC score, English skill level, Google Calendar link information

[0160] Output: Initial data sent to the server

[0161] Step 2:

[0162] Data reception and analysis by the server

[0163] The server receives the initial data sent by the user, checks the integrity of the received data, and saves each item in the database.

[0164] Input: Initial data

[0165] Output: User data stored in the database

[0166] Specific operation: The server checks whether the TOEIC score and English skill level are within a reasonable range.

[0167] Step 3:

[0168] Server-based skill evaluation

[0169] The server analyzes the received data and evaluates the user's skill level. For example, if a TOEIC score is 650, it will be evaluated as "intermediate."

[0170] Input: User data

[0171] Output: Assessed skill level

[0172] Specific operation: The server analyzes the score using the evaluation algorithm.

[0173] Step 4:

[0174] Training menu selection by server

[0175] The server will select the most appropriate training menu based on your assessed skill level, such as "Intermediate Reading Training."

[0176] Input: Skill Level

[0177] Output: Training menu

[0178] Specific operation: The server retrieves the appropriate training menu from the database.

[0179] Step 5:

[0180] Server-based schedule management

[0181] The server retrieves the user's Google Calendar schedule information, finds available time slots, and schedules training sessions for those available times.

[0182] Input: Google Calendar schedule information

[0183] Output: Scheduled training sessions

[0184] Specific operation: The server analyzes the schedule information and identifies free time.

[0185] Step 6:

[0186] Server-based schedule notification

[0187] When the server approaches the start time of a scheduled training session, it sends a notification to the user's device, for example, "The next learning session will start at 7:30."

[0188] Input: Scheduled training sessions

[0189] Output: Notification sent to the user's device

[0190] Specific operation: The server sends an email or push notification

[0191] Step 7:

[0192] Receiving notifications and launching apps on your device

[0193] The user device receives the notification from the server and automatically launches the learning app at the specified time.

[0194] Input: Notification from the server

[0195] Output: Learning app launched

[0196] Specific operation: The device launches the learning app at the specified time.

[0197] Step 8:

[0198] User-driven training

[0199] The user starts the learning app and progresses through the training. The app records the user's progress in real time.

[0200] Input: User interaction

[0201] Output: Real-time recorded progress

[0202] What it does: The user performs the workout and the app records their progress in real time.

[0203] Step 9:

[0204] Progress data transmission by device

[0205] At the end of a study session, the app sends the user's progress data to a server, such as "80% correct reading."

[0206] Input: Progress data

[0207] Output: Progress data sent to the server

[0208] Specific behavior: The app detects the end of the session and sends progress data to the server.

[0209] Step 10:

[0210] Server-provided feedback

[0211] The server analyzes the submitted progress data and provides feedback to the user, such as "You got 80% of your reading correct, so try practicing faster next time."

[0212] Input: Progress data

[0213] Output: Feedback provided to the user's device

[0214] Specific behavior: The server analyzes the progress data, generates feedback, and sends it to the user.

[0215] This allows for efficient learning tailored to the user's schedule and skill level, and by accurately understanding the user's learning progress and receiving feedback, effective learning can be achieved.

[0216] (Application example 1)

[0217] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0218] Conventional English learning support systems often lack the ability to provide optimal training menus tailored to the user's skill level or manage schedules, hindering efficient learning. Furthermore, progress data recording and feedback provision are performed manually, reducing the continuity and effectiveness of learning. Furthermore, it is difficult to efficiently manage learning time using the user's schedule information. There is a need to provide an effective and efficient English learning support system that can solve these issues.

[0219] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0220] In this invention, the server includes means for receiving English proficiency data from a user, means for evaluating the user's skill level based on the received English proficiency data, means for automatically selecting a training menu according to the user's skill level, means for scheduling a training menu during free time in the acquired schedule information, means for sending a notification to the user terminal at the start time of the scheduled training menu, means for automatically launching a learning application installed on the user terminal at a specified time, means for monitoring the ongoing training menu in real time and recording progress data, means for sending the recorded progress data to the server, means for providing feedback to the user based on the sent progress data, means for automatically generating an optimal training menu for the user using a generative model based on the English proficiency data and schedule information, means for generating prompt sentences to be given to the generative AI model, and means for automatically supporting the user's learning behavior and providing appropriate feedback in real time, thereby promoting the user's learning habits, improving learning effectiveness, and making efficient use of study time.

[0221] "User" refers to an individual person who uses the system and provides English proficiency data and schedule information.

[0222] "English proficiency data" is a collection of information indicating a user's English proficiency, such as a user's TOEIC score or English skill level.

[0223] "Skill level" refers to the level of a user's English ability that is evaluated based on specific standards, and is classified as beginner, intermediate, advanced, etc.

[0224] A "training menu" is a set of English learning tasks and exercises selected according to the user's skill level.

[0225] "Schedule information" is information that indicates the user's daily plans, and mainly includes calendar information.

[0226] A "notification" is a message sent to a user terminal at a specified time, informing the user of the start time of a training menu, etc.

[0227] "User terminal" refers to a device that a user operates directly to study English, including smartphones and tablets.

[0228] A "learning application" is software that is installed on a user's device and provides learning materials and training menus for English language learning.

[0229] "Progress data" refers to information about the learning progress and results recorded as the user performs the training menu.

[0230] "Server" is a computer that receives English proficiency data and progress data from users and provides feedback based on this data.

[0231] "Feedback" refers to information provided based on the user's progress data, including an evaluation of the learning results and advice for the next learning session.

[0232] A "generative model" is a machine learning model that automatically generates optimal training menus based on English proficiency data and schedule information.

[0233] A "prompt sentence" is a sentence input to a generative model, and is a specific instruction sentence created based on the user's learning content and purpose.

[0234] This invention is a system for effectively supporting users' English learning, and in particular, it supports the establishment of study habits and efficient learning through individualized study training according to the user's ability and effective schedule management. The invention utilizes the following hardware and software:

[0235] Hardware and software used

[0236] Server: Uses a web server (e.g., Apache (registered trademark)) and a database (e.g., MongoDB) to receive data, evaluate skill levels, select training menus, analyze progress data, and execute generative models.

[0237] User terminal: A smartphone or tablet (e.g., iOS or ANDROID (registered trademark)) is assumed to be a device that a user can directly operate to study English.

[0238] Google Calendar API: Used to retrieve user schedule information.

[0239] Outgoing Mail Server: Uses SMTP to send training session notifications to users.

[0240] Natural language explanation of the process

[0241] Main functions and processes of the server

[0242] 1. Receiving user data

[0243] The server receives the English proficiency data (e.g., TOEIC score, English skill level) provided by the user and records it in a database.

[0244] 2. Skill Level Assessment

[0245] The server automatically evaluates the user's skill level based on the received English proficiency data. For example, if the user's TOEIC score is 650, the server evaluates the user's skill level as "intermediate."

[0246] 3. Selecting a training menu

[0247] The server automatically selects a training menu appropriate for the assessed skill level, for example, "intermediate reading training" or "intermediate listening training."

[0248] 4. Schedule Management

[0249] The server uses the Google Calendar API to retrieve the user's schedule, find available time slots, and schedule suitable training sessions.

[0250] 5. Sending notifications

[0251] When the scheduled start time of the training approaches, the server sends a notification to the user terminal using the mail sending server.

[0252] 6. Automatic generation of training menus

[0253] The server uses the generative model to automatically generate the optimal training menu for the user based on English proficiency data and schedule information.

[0254] 7. Prompt generation for generative AI models

[0255] The server generates prompt sentences to be given to the generative AI model based on the user's learning content and goals.

[0256] Main functions and processing of the user terminal

[0257] 1. Receiving notifications

[0258] The user terminal receives the notification sent from the server and presents the content and start time of the next learning session.

[0259] 2. App auto-start

[0260] At the specified time, the learning application installed on the user's device will automatically start and the training will begin.

[0261] 3. Recording and sending progress data

[0262] It records your training progress in real time and sends the progress data to a server once your learning session is over.

[0263] Specific examples

[0264] For example, when a user with a TOEIC score of 650 initially configures the system, the server evaluates the user as "intermediate" and selects "intermediate reading training" and "intermediate listening training." The server then checks the user's Google Calendar to find available time between 7:30 and 8:00 a.m. on weekdays. Training is scheduled for this time period and a notification is sent to the user's device. At the designated time of 7:30 a.m., the learning application installed on the user's device automatically launches and the training begins. When the training session ends, progress data is sent to the server, and feedback is provided. The server also uses a generative model to automatically generate the next training menu, maximizing the user's learning effectiveness.

[0265] Prompt Sentence Examples

[0266] My TOEIC score is 650. Please suggest a training menu suitable for intermediate level and schedule study sessions during your free time. Please also send me a notification before the next session starts. The Google Calendar information is as follows: [Calendar Information].

[0267] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0268] Step 1:

[0269] Receiving user data

[0270] The server receives English proficiency data (e.g., TOEIC score or English skill level) entered by the user. User data is entered via a web form or similar and sent to the server. Input: TOEIC score, English skill level. Output: Recording of received data in a database. Specifically, the server receives an HTTP request and saves the data in a database (e.g., MongoDB).

[0271] Step 2:

[0272] Skill Level Assessment

[0273] The server evaluates the user's skill level based on the received English proficiency data. It analyzes the received data (e.g., TOEIC score) and determines the skill level. Input: TOEIC score. Output: Skill level (e.g., "Intermediate"). Specifically, the server applies predefined evaluation logic based on the data it receives to determine the skill level.

[0274] Step 3:

[0275] Selection of training menu

[0276] The server automatically selects a training menu according to skill level. It retrieves an appropriate training menu from the database and proposes it to the user. Input: Skill level. Output: Training menu. Specifically, the server selects candidates from a list of training menus pre-registered in the database based on the corresponding skill level.

[0277] Step 4:

[0278] Get schedule information

[0279] The server uses the Google Calendar API to obtain the user's schedule information. Input: User's calendar information. Output: Schedule information (such as a list of free time slots). Specifically, the server sends authentication information to the Google Calendar API and obtains events from the user's calendar.

[0280] Step 5:

[0281] Schedule a training session

[0282] The server detects available time slots based on the acquired schedule information and schedules training sessions. Input: Schedule information, training menu. Output: Scheduled training session. Specifically, the server analyzes the schedule information and associates the optimal time slot with the training menu.

[0283] Step 6:

[0284] Sending notifications

[0285] The server sends a notification to the user device when the scheduled training session is about to start. Input: Scheduled training session. Output: Notification message to the user device. Specifically, the server uses an email server or push notification service to send the specified message to the user.

[0286] Step 7:

[0287] App autostart

[0288] The user device will automatically launch the installed learning application at the specified time. Input: Notification message. Output: Launching of learning application. Specifically, the device will launch the learning application that is set as a trigger in response to the received notification message.

[0289] Step 8:

[0290] Recording and sending progress data

[0291] The user's device records the training progress in real time, and when the learning session ends, it sends the progress data to the server. Input: Operation data during training. Output: Sending progress data to the server. Specifically, the device records the user's answers and operations, and sends the data to the server via an HTTP request when the learning session ends.

[0292] Step 9:

[0293] Providing feedback

[0294] The server provides feedback to the user based on the submitted progress data. Input: Progress data. Output: Feedback message. Specifically, the server analyzes the progress data and generates and sends a message to the user based on predefined feedback logic.

[0295] Step 10:

[0296] Prompt generation for generative AI models

[0297] The server generates a prompt sentence to be given to the generative AI model based on English proficiency data and schedule information. Input: English proficiency data, schedule information. Output: Prompt sentence. Specifically, the server creates a prompt sentence appropriate for the generative AI model based on the specified data and provides it to the model.

[0298] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0299] ---

[0300] This invention is a system that effectively supports users' English learning, and in particular, is a system that can optimize the learning experience by recognizing the user's emotions. It provides training menus based on the user's input data and schedule information, and furthermore, uses emotion recognition functions to flexibly adjust the learning content and schedule, thereby achieving more effective learning.

[0301] Main functions and processes of the server

[0302] 1. User data reception function

[0303] The server receives data such as TOEIC scores and English skill levels sent by users, providing the initial information for creating a personalized learning plan for each user.

[0304] 2. Skill evaluation function

[0305] The server evaluates the user's skill level based on the received English proficiency data. For example, if the TOEIC score is 650, the user's skill level is evaluated as "intermediate."

[0306] 3. Emotion recognition function

[0307] The server analyzes facial expression and voice data sent from the user device and uses an emotion recognition engine to determine the user's emotional state, for example, to obtain information such as whether the user is tired or stressed.

[0308] 4. Training menu selection function

[0309] The server selects the most appropriate training menu based on the user's skill level and emotional state. For example, if the user is tired, a menu with a lower level of difficulty can be selected.

[0310] 5. Schedule management function

[0311] The server retrieves the user's schedule information, such as Google Calendar, and finds available time slots. It then schedules training sessions for those available times.

[0312] 6. Notification function

[0313] The server sends a notification to the user device at the start time of the scheduled workout, which includes a menu that is fine-tuned according to the user's emotional state.

[0314] Main functions and processing of the user terminal

[0315] 1. Notification reception function

[0316] The user terminal receives the notification sent from the server and notifies the user of the start time and training content.

[0317] 2. App auto-start function

[0318] Upon receiving the notification, at the specified time, the learning application installed on the user's device will automatically start.

[0319] 3. Learning Progression Function

[0320] The user device analyzes the user's facial expressions and voice data in real time and also records emotional changes during learning.

[0321] 4. Progress data transmission function

[0322] When the learning session ends, the user device transmits the training progress and emotional data to the server.

[0323] User operations

[0324] 1. Data Entry

[0325] When users first set up the system, they input their TOEIC score and English skill level, and also link the system with Google Calendar.

[0326] 2. Notification confirmation

[0327] The user checks the notification from the server to find out the start time and content of the next learning session.

[0328] 3. Study with apps

[0329] At the specified time, the user begins training using the automatically launched app.

[0330] 4. Feedback confirmation

[0331] After studying, users can check the feedback and progress data displayed to understand their learning status.

[0332] Specific examples

[0333] For example, suppose a user with a TOEIC score of 650 is registered in the system. In this case, the server evaluates the skill level as "intermediate." The server then analyzes the user's facial expression and voice data and determines that the user is feeling fatigued. The server selects a relatively easy task from the "intermediate reading training" and schedules it for an available time in the user's Google calendar. The server determines that this training session will take place from 7:30 to 8:00 AM on a weekday. A notification is sent to the user's device, and the learning application automatically launches at the specified time. The user performs the "intermediate reading training," and upon completion, progress data and emotional data are sent to the server. The server analyzes the data and provides the user with feedback, such as "80% accuracy in reading, high stress level."

[0334] As such, the present invention is a system that can provide flexible training tailored to the user's skill level and emotional state, and efficiently manage progress and provide feedback in real time, allowing users to efficiently utilize their spare time to study English and maximize the effectiveness of their learning.

[0335] The processing flow will be explained below.

[0336] ---

[0337] Step 1:

[0338] During the initial setup, users enter their TOEIC score, English skill level, and Google Calendar integration information into the device, which then sends this data to the server.

[0339] Step 2:

[0340] The server receives the TOEIC score and English skill level data sent by the user. For example, if a TOEIC score of 650 is used, the English skill level is set as "intermediate."

[0341] Step 3:

[0342] The server evaluates the user's skill level based on the received data. For example, a TOEIC score of 650 would be rated as "intermediate."

[0343] Step 4:

[0344] The server requests permission to retrieve the user's Google Calendar information. If the user grants access, the server retrieves the calendar information.

[0345] Step 5:

[0346] The server detects available time from the acquired schedule information. For example, it identifies available time from 7:30 to 8:00 AM on weekdays.

[0347] Step 6:

[0348] The user terminal collects the user's facial expression and voice data in real time and transmits them to the server, which then recognizes the user's emotional state.

[0349] Step 7:

[0350] The server uses an emotion recognition engine to determine the user's emotional state, for example, determining that the user is feeling tired.

[0351] Step 8:

[0352] The server selects a training menu based on the user's skill level and emotional state, for example, selecting "intermediate reading training" and adjusting the difficulty level.

[0353] Step 9:

[0354] The server sends a notification to the user device when the scheduled training session approaches, with a message such as "The next learning session will start at 7:30."

[0355] Step 10:

[0356] The user terminal receives the notification and displays it to the user as a pop-up message, and the user confirms the start time of the next learning session.

[0357] Step 11:

[0358] At the specified time (7:30 AM in this case), the user's device will automatically launch the English learning application, and the user will begin training in the app.

[0359] Step 12:

[0360] The user progresses through the learning process according to a designated training menu (e.g., "Intermediate Reading Training"). During the training, the user's facial expressions and voice data are collected again, and emotional changes are recorded.

[0361] Step 13:

[0362] When the learning session ends, the user's device records the training progress and emotional data and transmits it to the server.

[0363] Step 14:

[0364] The server analyzes the received progress data and emotion data and provides feedback to the user, such as "80% correct answer rate in reading, high stress level."

[0365] By taking these steps, users can efficiently utilize their spare time to study English and obtain an optimal learning experience that takes into account their emotional state, thereby improving learning effectiveness and helping to establish consistent learning habits.

[0366] Example 2

[0367] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0368] Conventional English learning support systems are limited in providing training tailored to the user's skill level, and have had the problem of difficulty in providing flexible learning content that responds to the user's emotional state and real-time progress.In addition, they lack the ability to automatically set a learning schedule that utilizes the user's free time, or to provide feedback based on emotional recognition during learning, making it impossible to provide an optimal learning experience for users.

[0369] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0370] In this invention, the server includes means for receiving language ability data from a user, means for evaluating the user's skill level based on the received language ability data, means for recognizing the user's skill level and emotional state and automatically selecting a training menu accordingly, means for acquiring the user's schedule information, means for scheduling a training menu during free time in the acquired schedule information and sending a notification to the user terminal at the training start time, means for automatically starting learning software installed on the user terminal at a specified time, means for acquiring the user's progress and emotional data in real time while learning, and means for analyzing the acquired progress and emotional data, generating feedback, and providing it to the user. This makes it possible to provide flexible and optimal learning content according to the user's emotional state and real-time progress, and to automatically set an efficient learning schedule that utilizes the user's free time and provide feedback in real time.

[0371] "Language Proficiency Data" refers to TOEIC scores and other data indicating English proficiency for assessing a user's English skills.

[0372] The "skill level" is a level indicating the user's English ability as a result of evaluating the user's English ability.

[0373] "Emotional state" refers to the user's psychological and emotional state, which is determined by analyzing the user's facial expressions and voice data.

[0374] A "training menu" is a set of specific exercises and tasks for learning English that are selected based on the user's skill level and emotional state.

[0375] "Schedule information" is information relating to a user's schedule and free time, and is managed, for example, by a calendar.

[0376] "Free time" is a time period in the user's schedule information in which no other plans are scheduled.

[0377] A "notification" is a message sent from the server to the user terminal, informing the user of the start time and content of the training menu.

[0378] "Learning software" refers to an application for learning English that is installed and used by users.

[0379] "Progress" is information that indicates how much of a task a user has completed while studying English.

[0380] "Real-time" means that the actions and states of the user at that moment are reflected immediately as they progress through their learning process.

[0381] "Feedback" refers to evaluations and advice provided by the server regarding the learning a user has done.

[0382] The present invention provides a system for effectively supporting a user's English learning, and in particular, a system for optimizing the learning experience by recognizing the user's emotions. Specific embodiments for carrying out the present invention are described below.

[0383] Basic system configuration

[0384] This system consists of a server and a user device. The server receives language ability data and schedule information sent by the user, and uses this information to select training menus and manage schedules. It also uses an emotion recognition engine to determine the user's emotional state and flexibly adjust the learning content. The user device receives notifications from the server, automatically launches the learning software, and acquires learning progress data in real time and sends it to the server.

[0385] Hardware and Software

[0386] The entire system is implemented using the following hardware and software:

[0387] Server: Uses a cloud server or on-premise server to perform data analysis, emotion recognition and scheduling.

[0388] User devices: Use devices that can install learning software, such as computers, smartphones, or tablets.

[0389] Learning software: An application for learning English that includes the ability to record the user's learning progress and emotional state.

[0390] Emotion recognition engine: For example, emotion recognition is performed using the Emotion API of Microsoft (registered trademark) Azure (registered trademark).

[0391] Data processing and calculation

[0392] The server performs the following steps to process and calculate the data:

[0393] 1. Data reception and processing: The server receives language ability data (e.g., TOEIC score) and schedule information sent by the user and analyzes them as initial data.

[0394] 2. Skill level assessment: Based on the received language proficiency data, a calculation process is performed to assess the user's skill level.

[0395] 3. Emotion Recognition: An emotion recognition engine is used to analyze the user's facial expressions and voice data to determine their emotional state.

[0396] 4. Training Menu Selection: Select an appropriate training menu based on assessed skill level and emotional state.

[0397] 5. Schedule Management: Detects free time from user schedule information and automatically schedules training sessions.

[0398] 6. Notifications and Feedback: Send notifications at the scheduled start time of training, and provide feedback after learning by analyzing progress and sentiment data obtained.

[0399] Specific examples

[0400] For example, suppose a user with a TOEIC score of 650 registers with the system. Based on the received score, the server evaluates the user's skill level as "intermediate." The server then analyzes the user's facial expression and voice data and determines that the user is tired.

[0401] The server selects a lower level of difficulty from the "Intermediate Reading Training" and schedules the training session during the user's free time (e.g., weekdays from 7:30 to 8:00 a.m.). At the designated time, the learning software installed on the user's device automatically starts up and the training begins.

[0402] After the user completes the training, the learning software sends progress and emotional data to the server, which analyzes the progress and emotional data and provides feedback to the user, such as "80% correct reading, high stress level."

[0403] Prompt Sentence Examples

[0404] "If a user with a TOEIC score of 650 is determined to be fatigued, generate a prompt from the AI ​​system that selects relatively easy intermediate reading training tasks and provides a way to schedule the training during the user's free time."

[0405] As such, the present invention is a system that provides flexible and optimal learning according to the user's skill level and emotional state, efficiently manages learning schedules by utilizing free time, and provides real-time feedback.

[0406] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0407] Step 1:

[0408] Entering and submitting user data

[0409] During initial setup, the user enters their TOEIC score and English skill level and sends them to the server. The system also links with Google Calendar. The input for this step is the user's language ability data and schedule information, such as "TOEIC score 650, skill level intermediate." The output is the user data sent to the server.

[0410] Step 2:

[0411] Data reception by the server and skill evaluation

[0412] The server receives the language ability data sent by the user and evaluates the user's skill level based on this. The input to this step is the language ability data sent by the user. The data is saved in an internal database and an evaluation process is performed. For example, if the user's TOEIC score is 650, the user's skill level is evaluated as "intermediate." The output is the data that evaluates the user's skill level.

[0413] Step 3:

[0414] Emotion recognition by server

[0415] The server analyzes the facial expression and voice data sent from the user device to determine the user's emotional state. The input for this step is the facial expression and voice data sent from the user device. An emotion recognition engine (e.g., Emotion API) is used to analyze the data and determine whether the user is tired. The output is data indicating the user's emotional state.

[0416] Step 4:

[0417] Selection of training menu

[0418] The server selects the optimal training menu based on the user's skill level and emotional state. The input for this step is the user's skill level data and emotional state data. A data analysis algorithm is used to identify a training menu suitable for the user. For example, if the user is tired, a less difficult "intermediate reading training" is selected. The output is the selected training menu.

[0419] Step 5:

[0420] Schedule registration by server

[0421] The server retrieves schedule information from the user's Google Calendar and detects available time slots. The input to this step is the user's schedule information. The server analyzes the available time slots and schedules training sessions for those times. For example, the server schedules sessions from 7:30 to 8:00 AM on weekdays. The output is the training sessions registered in the schedule.

[0422] Step 6:

[0423] Sending notifications

[0424] The server sends a notification to the user terminal at the scheduled training start time. The input for this step is the schedule information and the selected training menu. The notification system is used to send the specified message to the user terminal. The output is a notification displayed on the user terminal.

[0425] Step 7:

[0426] Receiving notifications on the user device and automatically launching the app

[0427] The user device receives the notification sent from the server and automatically launches the learning software at the specified time. The input to this step is the notification from the server. The software is launched at the specified time using the application auto-launch function. For example, at 7:30 AM, the app to start "Intermediate Reading Training" is launched. The output is the launched learning software.

[0428] Step 8:

[0429] User-led learning sessions

[0430] The user trains using the automatically launched learning software. The inputs of this step are the launched learning software and the training menu. The user progresses through the specified training. The outputs are the training progress and emotion data.

[0431] Step 9:

[0432] Progress data transmission by user terminal

[0433] After the training session is completed, the user device sends the progress and emotion data to the server. The input of this step is the progress and emotion data collected during training. The data transmission function is used to transmit this data to the server. The output is the progress and emotion data transmitted to the server.

[0434] Step 10:

[0435] Data analysis by server and feedback transmission

[0436] The server analyzes the received data and sends feedback to the user. The input of this step is the user's progress data and emotional data. Using a data analysis algorithm, the server identifies the user's learning achievements and areas for improvement and generates feedback. For example, it generates feedback such as "80% correct reading, high stress level." The output is the generated feedback.

[0437] (Application example 2)

[0438] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0439] Conventional English learning support systems focus on providing training menus based on the user's skill level, but do not adequately address optimization based on the user's emotional state or learning environment. As a result, even if the user feels tired or stressed, a fixed learning menu is provided, which can reduce learning efficiency. Furthermore, systems lack functionality to effectively support learning activities in physical stores, and it is necessary to provide an optimal learning experience tailored to the store environment.

[0440] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0441] In this invention, the server includes means for receiving English proficiency data from a user, means for evaluating the user's skill level based on the received English proficiency data, means for automatically selecting a training menu according to the user's skill level, means for acquiring the user's schedule information, means for scheduling a training menu during free time in the acquired schedule information, means for transmitting the scheduled training menu to a user device at a start time, means for automatically starting a learning program installed on the user device at a specified time, means for optimizing the learning content based on the store environment, and means for recognizing the user's emotional state and adjusting the learning content. This enables flexible training provision according to the user's emotional state and learning environment, and effective learning support within a physical store.

[0442] The "means for receiving English ability data from users" is a system that allows the server to receive ability data such as TOEIC scores and English skill levels provided by users.

[0443] The "means for assessing the user's skill level based on the received English proficiency data" is a system for analyzing the received data and assessing the user's current English proficiency based on specific criteria.

[0444] The "means for automatically selecting a training menu according to the user's skill level" is a system for automatically selecting the most suitable English training program based on the evaluated skill level information.

[0445] "Means for obtaining user schedule information" refers to a system that allows the server to obtain schedule information such as the user's Google Calendar.

[0446] The "means for scheduling a training menu during free time in the acquired schedule information" is a system for identifying free time in the acquired schedule and allocating an English training session to that time.

[0447] The "means for transmitting to the user device at the start time of the scheduled training menu" is a system for sending a notification to the user's device, such as a smartphone or tablet, at the set training start time.

[0448] "Means for automatically starting a learning program installed on a user device at a specified time" refers to a system for automatically starting learning software installed on a user's terminal at a specified time.

[0449] The "means for optimizing learning content based on the store environment" is a system that collects environmental data such as noise levels and congestion levels in physical stores and optimizes English training content accordingly.

[0450] The "means for recognizing the user's emotional state and adjusting the learning content" is a system that uses a camera and microphone to analyze the user's facial expressions and tone of voice, and adjusts the English training content according to the recognized emotions.

[0451] The present invention is a system for effectively supporting users' English learning in a brick-and-mortar store, and in particular, a system that can recognize users' emotions and optimize the learning experience. The system includes the following main elements:

[0452] Main functions and processes of the server

[0453] 1. User data reception function

[0454] The server receives data such as TOEIC scores and English skill levels sent by users, providing the initial information for creating a personalized learning plan for each user.

[0455] 2. Skill evaluation function

[0456] The server evaluates the user's skill level based on the received English proficiency data. For example, if the TOEIC score is 650, the user's skill level is evaluated as "intermediate."

[0457] 3. Emotion recognition function

[0458] The server analyzes facial expression and voice data sent from the user's device and uses an emotion recognition engine to determine the user's emotional state. For example, it acquires information such as whether the user is tired or stressed. The hardware used includes a camera and microphone, and the software used includes a facial expression recognition API and a voice recognition API.

[0459] 4. Training menu selection function

[0460] The server selects the most appropriate training menu based on the user's skill level and emotional state. For example, if the user is tired, a menu with a lower level of difficulty can be selected.

[0461] 5. Schedule management function

[0462] The server retrieves the user's calendar information, finds available time slots, and schedules training sessions within those times. The software used includes the Google Calendar API.

[0463] 6. Notification function

[0464] The server sends a notification to the user device at the start time of the scheduled workout, which includes a menu that is fine-tuned according to the user's emotional state.

[0465] Main functions and processing of the user terminal

[0466] 1. In-store detection function

[0467] The user device uses Wi-Fi or Bluetooth beacons to detect when the user is in a physical store.

[0468] 2. Environment recognition function

[0469] The user device recognizes the noise level and occupancy status inside the store and sends the information to the server. The hardware used includes a microphone, and the software used includes a noise-canceling algorithm.

[0470] 3. Notification reception function

[0471] The user terminal receives the notification sent from the server and notifies the user of the start time and training content.

[0472] 4. App auto-start function

[0473] Upon receiving the notification, the learning program installed on the user's terminal will automatically start at the specified time.

[0474] 5. Learning Progression Function

[0475] The user device analyzes the user's facial expressions and voice data in real time and also records emotional changes during learning.

[0476] 6. Progress data transmission function

[0477] When the learning session ends, the user device transmits the training progress and emotional data to the server.

[0478] Specific examples

[0479] For example, suppose a user with a TOEIC score of 650 is registered in the system. In this case, the server evaluates the skill level as "intermediate." Then, it analyzes the user's facial expression and voice data and determines that the user is feeling fatigued. The server selects a relatively easy task from "intermediate reading training" and schedules it for an available time in the user's calendar. It determines that this training session will take place, for example, from 7:30 to 8:00 AM on a weekday. A notification is sent to the user's device, and the learning program automatically starts at the specified time. The user performs the "intermediate reading training," and after it is completed, progress data and emotional data are sent to the server.

[0480] Prompt Sentence Examples

[0481] "I'd like to do some listening practice during the cafe time today. If the cafe is quiet, I'd like more difficult questions, and if it's noisy, I'd like easier questions."

[0482] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0483] Step 1: Receiving user data

[0484] The server receives data such as TOEIC scores and English skill levels sent by users. When the user enters this data in the application and presses the submit button, the server receives the data and stores it in a database.

[0485] Input: TOEIC score, English skill level data

[0486] Output: Save user data to database

[0487] Step 2: Skills Assessment

[0488] The server evaluates the user's skill level based on the received English proficiency data. For example, if the TOEIC score is 650, the server will evaluate it as "intermediate." An evaluation algorithm is applied to this data to determine the user's skill level.

[0489] Input: English proficiency data

[0490] Output: Assessed skill level (e.g., intermediate)

[0491] Step 3: In-store detection

[0492] The user device uses Wi-Fi or Bluetooth beacons to detect when the user is in a physical store, which then transmits specific store environment information to the server.

[0493] Input: Signals from Wi-Fi or Bluetooth beacons

[0494] Output: Confirmation that the user is in the store

[0495] Step 4: Environmental awareness

[0496] The user device recognizes the noise level and congestion status inside the store and sends the information to the server. Specifically, it uses a microphone to collect surrounding sounds and analyzes the noise level.

[0497] Input: Audio data from the microphone

[0498] Output: Noise level, congestion data

[0499] Step 5: Emotion Recognition

[0500] The server analyzes the facial expression and voice data sent from the user's device and uses an emotion recognition engine to determine the user's emotional state, for example, extracting information such as whether the user is tired or stressed.

[0501] Input: facial expression data, voice data

[0502] Output: Emotional state assessment (e.g., tired)

[0503] Step 6: Choose your training menu

[0504] The server selects the most appropriate training menu based on the user's skill level and emotional state: for example, if the user is tired, it selects a reading training with a lower level of difficulty.

[0505] Input: Skill level, emotional state

[0506] Output: Selected training menu

[0507] Step 7: Retrieve schedule information

[0508] The server retrieves the user's calendar information, using the Google Calendar API to read the user's schedule and identify available times.

[0509] Input: User's Google Calendar information

[0510] Output: Free / busy information

[0511] Step 8: Training Schedule

[0512] The server schedules the training menu for the available time slots in the acquired schedule information. For example, the server may set a training session for the available time slots between 7:30 and 8:00 AM on weekdays.

[0513] Input: Free / busy information, training menu

[0514] Output: Scheduled training sessions

[0515] Step 9: Send notification

[0516] The server sends a notification to the user terminal at the start time of the scheduled training menu, which also includes the training content.

[0517] Input: Schedule a training session

[0518] Output: Notification to user terminal

[0519] Step 10: App Auto-Launch

[0520] The user terminal automatically starts the installed learning program at the specified time. Upon receiving the notification, the user terminal starts the program and begins training.

[0521] Input: Notification content

[0522] Output: Automatic launch of learning program

[0523] Step 11: Record your learning progress and emotional changes

[0524] The user device analyzes the user's facial expressions and voice data in real time and records emotional changes during learning. Specifically, it collects and analyzes data using a camera and microphone.

[0525] Input: Real-time facial expression data, voice data

[0526] Output: Recorded data of emotional changes

[0527] Step 12: Send progress data

[0528] Once the training session is over, the user device transmits training progress and emotional data to the server, which stores this data in a database for further analysis.

[0529] Input: Progress data, emotion data

[0530] Output: Send data to the server

[0531] Examples of prompts:

[0532] "I'd like to do some listening practice during the cafe time today. If the cafe is quiet, I'd like more difficult questions, and if it's noisy, I'd like easier questions."

[0533] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0534] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0535] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0536] [Second embodiment]

[0537] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0538] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0539] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0540] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0541] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0542] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0543] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0544] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0545] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0546] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0547] In the smart glasses 214, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0548] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0549] ---

[0550] The present invention is a system for effectively supporting users' English learning, and aims to automatically provide optimal training menus based on English proficiency data entered by the user, and to efficiently manage learning time according to the user's schedule. This system is mainly composed of a server, a user terminal, and processing steps including the exchange of input data from the user.

[0551] Main functions and processes of the server

[0552] 1. User data reception function

[0553] When users initially set up the system, they input their TOEIC score and English skill level, and the server receives and records this data.

[0554] 2. Skill evaluation function

[0555] The server automatically evaluates the user's skill level based on the received English proficiency data. For example, if the TOEIC score is 650, the server will evaluate the user's skill level as "intermediate."

[0556] 3. Training menu selection function

[0557] The server automatically selects a training menu appropriate for the assessed skill level, for example, "intermediate reading training" or "intermediate listening training."

[0558] 4. Schedule management function

[0559] The server retrieves the user's schedule information, such as Google Calendar, and finds available time slots. It then schedules training sessions that fit the available time slots.

[0560] 5. Notification function

[0561] When the scheduled training session is about to begin, the server sends a notification to the user's device, with a message like "Your next training session will begin at 7:30."

[0562] Main functions and processing of the user terminal

[0563] 1. Notification reception function

[0564] The user device receives a notification sent from the server, which indicates the content and start time of the next learning session.

[0565] 2. App auto-start function

[0566] The device that receives the notification will automatically launch the English learning app at the specified time. For example, "Intermediate Reading Training" will start at 7:30 AM.

[0567] 3. Learning Progression Function

[0568] The user can then use the app to progress through the training, which will then record the progress of the training and the user's responses in real time.

[0569] 4. Progress data transmission function

[0570] At the end of a study session, the app sends the user's progress data to the server, such as "80% correct reading" or "75% correct listening."

[0571] User operations

[0572] 1. Data Entry

[0573] During initial setup, users enter their TOEIC score, English skill level, and Google Calendar connection information, which allows the server to suggest appropriate training menus and schedules.

[0574] 2. Notification confirmation

[0575] The user checks the notification sent from the server to find out when the next learning session will start.

[0576] 3. Study with apps

[0577] At the designated time, the user will use the automatically launched app to carry out the training, which will help develop a natural learning habit.

[0578] 4. Feedback confirmation

[0579] Once the training is complete, users can view their progress and feedback from the app.

[0580] Specific examples

[0581] For example, if a user with a TOEIC score of 650 provides input data to the system, the server will evaluate the user as "intermediate." The server then selects "intermediate reading training" and "intermediate listening training." It then checks Google Calendar to find available time between 7:30 and 8:00 a.m. on weekdays. The server schedules training for this time period and sends a notification to the user's device. At 7:30 a.m., the learning app installed on the user's device will automatically launch and the training will begin. When the user completes the training, progress data is sent to the server, which then provides feedback based on that data.

[0582] As described above, the present invention is a system that can provide flexible training according to the user's skill level and schedule, and can efficiently manage progress.

[0583] The processing flow will be explained below.

[0584] ---

[0585] Step 1:

[0586] The server receives the TOEIC score and English skill level data sent by the user. Specifically, the score and skill information entered by the user on the initial setting screen is sent to the server.

[0587] Step 2:

[0588] The server automatically evaluates the user's skill level based on the received English proficiency data. For example, if the TOEIC score is 650, the server evaluates the user's skill level as "intermediate."

[0589] Step 3:

[0590] The server automatically selects a training menu according to the user's skill level, such as "Intermediate Reading Training" or "Intermediate Listening Training."

[0591] Step 4:

[0592] The server requests permission to access the schedule information from the user to obtain the schedule information of the user's Google Calendar, etc. After the user grants permission, the server obtains the user's schedule data.

[0593] Step 5:

[0594] The server detects available times from the acquired schedule information. For example, it determines that available times are from 7:30 to 8:00 a.m. on weekdays.

[0595] Step 6:

[0596] The server schedules training sessions during the available times it finds, for example, allocating "Intermediate Reading Training" between 7:30 and 8:00 AM.

[0597] Step 7:

[0598] The server sends a notification to the user device 15 minutes before the scheduled training, with a message such as "The next learning session will start at 7:30."

[0599] Step 8:

[0600] The user terminal receives the notification from the server and displays a pop-up message to the user, who then confirms the start time.

[0601] Step 9:

[0602] At the specified time (7:30 AM in this case), the user's device automatically launches the English learning application, and the user begins training in the app.

[0603] Step 10:

[0604] The user progresses through the learning process by following the designated training menu, for example, answering questions in the "Intermediate Reading Training" section.

[0605] Step 11:

[0606] Once the learning session is over, the user's device records the training progress and results and sends them to the server.

[0607] Step 12:

[0608] The server analyzes the received progress data and provides feedback to the user, such as "Your accuracy rate for this reading training was 80%."

[0609] By following these steps, users can study English efficiently in their spare time and naturally develop a study habit. Real-time progress management and feedback also improve the effectiveness of users' learning.

[0610] Example 1

[0611] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0612] Conventional English learning systems have difficulty flexibly adapting to individual users' skill levels and schedules, preventing them from maximizing learning effectiveness. They also lacked a mechanism for recording users' progress in real time or the ability to provide feedback based on that progress. This prevented users from accurately understanding their own learning progress and results, hindering effective learning.

[0613] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0614] In this invention, the server includes means for receiving English proficiency data from a user, means for evaluating the user's skill level based on the received English proficiency data, means for automatically selecting a training menu according to the user's skill level, means for acquiring the user's schedule information, means for scheduling a training menu during free time in the acquired schedule information, means for sending a notification to the user terminal at the start time of the scheduled training menu, means for automatically starting a learning application installed on the user terminal at a specified time, means for the learning application to record the user's training progress in real time, means for sending user progress data to the server after training ends, and means for providing feedback based on the progress data to the user terminal. This enables efficient learning tailored to the user's schedule and skill level, and by accurately understanding the user's learning progress and receiving feedback, effective learning can be achieved.

[0615] "User" refers to an individual or organization that uses the English Learning System.

[0616] "English Proficiency Data" refers to data necessary for assessing a user's English proficiency, such as a user's TOEIC score or English skill level.

[0617] "Skill level" refers to a user's proficiency in understanding and practicing English as assessed based on English proficiency data.

[0618] "Training Menu" refers to a set of English learning sessions and assignments provided according to the user's skill level.

[0619] "Schedule information" refers to data such as calendar information that indicates the user's schedule and free time.

[0620] "Real-time" refers to the fact that progress and achievements are recorded instantly while a user is using a learning application.

[0621] "Progress data" refers to data that indicates the user's learning status, such as the results achieved by the user during a study session and the percentage of correct answers.

[0622] "Feedback" refers to information provided to the user based on progress data, such as indications of learning results and next goals.

[0623] "Server" refers to the central processing unit that receives and analyzes user data, selects training menus, and manages schedules.

[0624] "User terminal" refers to a computing device on which a user installs and uses a learning application.

[0625] "Learning Application" refers to software that a user uses to train their English language skills.

[0626] "Notification" refers to a message or alert sent from a server to a user device.

[0627] This invention is a system for effectively supporting users' English learning. This system provides an optimal training menu based on the user's English proficiency data and efficiently manages learning time according to the user's schedule. It is mainly composed of a server, a user terminal, and processing steps including the exchange of input data from the user.

[0628] Specific server operations

[0629] User data reception function

[0630] When users initially set up the system, they input their TOEIC score and English skill level, and the server receives and records this data.

[0631] Skill evaluation function

[0632] The server automatically evaluates the user's skill level based on the received English proficiency data. For example, if the TOEIC score is 650, the server will evaluate the user's skill level as "intermediate."

[0633] Training menu selection function

[0634] The server automatically selects a training menu according to the assessed skill level, for example, "intermediate reading training" or "intermediate listening training."

[0635] Schedule management function

[0636] The server retrieves the user's schedule information (e.g., Google Calendar), finds available time slots, and schedules training sessions that fit the available time slots.

[0637] Notification function

[0638] When the scheduled training session is about to begin, the server sends a notification to the user's device, for example, a message saying, "Your next training session will begin at 7:30."

[0639] Specific operation of the user terminal

[0640] Notification reception function

[0641] The user device receives a notification sent from the server, which indicates the content and start time of the next learning session.

[0642] App auto-start function

[0643] The device that receives the notification will automatically launch the English learning app at the specified time. For example, "Intermediate Reading Training" will start at 7:30 AM.

[0644] Learning Progression Function

[0645] The user can then use the app to progress through the training, which will then record the progress of the training and the user's responses in real time.

[0646] Progress data transmission function

[0647] At the end of a study session, the app sends the user's progress data to the server, such as "80% correct reading" or "75% correct listening."

[0648] Specific user operations

[0649] Data Entry

[0650] During initial setup, users enter their TOEIC score, English skill level, and Google Calendar connection information, which allows the server to suggest appropriate training menus and schedules.

[0651] Notification confirmation

[0652] The user checks the notification sent from the server to find out when the next learning session will start.

[0653] Learn with the app

[0654] At the designated time, the app will automatically start and the training will begin, making it easier to naturally develop a study habit.

[0655] Feedback Check

[0656] Once the training is complete, users can view their progress and feedback from the app.

[0657] Specific examples

[0658] For example, a user with a TOEIC score of 650 provides input data to the system. The server evaluates the user as "intermediate." The server then selects "intermediate reading training" and "intermediate listening training." After that, it checks Google Calendar to find available time slots between 7:30 and 8:00 a.m. on weekdays. The server schedules the training for this time period and sends a notification to the user's device. At 7:30 a.m., the learning app installed on the user's device automatically launches and the training begins. When the user completes the training, progress data is sent to the server, which then provides feedback based on that data.

[0659] As described above, the present invention is a system that can provide flexible training according to the user's skill level and schedule, and can efficiently manage progress.

[0660] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0661] Step 1:

[0662] Initial data entry by the user

[0663] The user opens the system's initial setup screen and enters their TOEIC score, English skill level, and Google Calendar connection information, which then sends the initial data to the server.

[0664] Input: TOEIC score, English skill level, Google Calendar link information

[0665] Output: Initial data sent to the server

[0666] Step 2:

[0667] Data reception and analysis by the server

[0668] The server receives the initial data sent by the user, checks the integrity of the received data, and saves each item in the database.

[0669] Input: Initial data

[0670] Output: User data stored in the database

[0671] Specific operation: The server checks whether the TOEIC score and English skill level are within a reasonable range.

[0672] Step 3:

[0673] Server-based skill evaluation

[0674] The server analyzes the received data and evaluates the user's skill level. For example, if a TOEIC score is 650, it will be evaluated as "intermediate."

[0675] Input: User data

[0676] Output: Assessed skill level

[0677] Specific operation: The server analyzes the score using the evaluation algorithm.

[0678] Step 4:

[0679] Training menu selection by server

[0680] The server will select the most appropriate training menu based on your assessed skill level, such as "Intermediate Reading Training."

[0681] Input: Skill Level

[0682] Output: Training menu

[0683] Specific operation: The server retrieves the appropriate training menu from the database.

[0684] Step 5:

[0685] Server-based schedule management

[0686] The server retrieves the user's Google Calendar schedule information, finds available time slots, and schedules training sessions for those available times.

[0687] Input: Google Calendar schedule information

[0688] Output: Scheduled training sessions

[0689] Specific operation: The server analyzes the schedule information and identifies free time.

[0690] Step 6:

[0691] Server-based schedule notification

[0692] When the server approaches the start time of a scheduled training session, it sends a notification to the user's device, for example, "The next learning session will start at 7:30."

[0693] Input: Scheduled training sessions

[0694] Output: Notification sent to the user's device

[0695] Specific operation: The server sends an email or push notification

[0696] Step 7:

[0697] Receiving notifications and launching apps on your device

[0698] The user device receives the notification from the server and automatically launches the learning app at the specified time.

[0699] Input: Notification from the server

[0700] Output: Learning app launched

[0701] Specific operation: The device launches the learning app at the specified time.

[0702] Step 8:

[0703] User-driven training

[0704] The user starts the learning app and progresses through the training. The app records the user's progress in real time.

[0705] Input: User interaction

[0706] Output: Real-time recorded progress

[0707] What it does: The user performs the workout and the app records their progress in real time.

[0708] Step 9:

[0709] Progress data transmission by device

[0710] At the end of a study session, the app sends the user's progress data to a server, such as "80% correct reading."

[0711] Input: Progress data

[0712] Output: Progress data sent to the server

[0713] Specific behavior: The app detects the end of the session and sends progress data to the server.

[0714] Step 10:

[0715] Server-provided feedback

[0716] The server analyzes the submitted progress data and provides feedback to the user, such as "You got 80% of your reading correct, so try practicing faster next time."

[0717] Input: Progress data

[0718] Output: Feedback provided to the user's device

[0719] Specific behavior: The server analyzes the progress data, generates feedback, and sends it to the user.

[0720] This allows for efficient learning tailored to the user's schedule and skill level, and by accurately understanding the user's learning progress and receiving feedback, effective learning can be achieved.

[0721] (Application example 1)

[0722] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0723] Conventional English learning support systems often lack the ability to provide optimal training menus tailored to the user's skill level or manage schedules, hindering efficient learning. Furthermore, progress data recording and feedback provision are performed manually, reducing the continuity and effectiveness of learning. Furthermore, it is difficult to efficiently manage learning time using the user's schedule information. There is a need to provide an effective and efficient English learning support system that can solve these issues.

[0724] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0725] In this invention, the server includes means for receiving English proficiency data from a user, means for evaluating the user's skill level based on the received English proficiency data, means for automatically selecting a training menu according to the user's skill level, means for scheduling a training menu during free time in the acquired schedule information, means for sending a notification to the user terminal at the start time of the scheduled training menu, means for automatically launching a learning application installed on the user terminal at a specified time, means for monitoring the ongoing training menu in real time and recording progress data, means for sending the recorded progress data to the server, means for providing feedback to the user based on the sent progress data, means for automatically generating an optimal training menu for the user using a generative model based on the English proficiency data and schedule information, means for generating prompt sentences to be given to the generative AI model, and means for automatically supporting the user's learning behavior and providing appropriate feedback in real time, thereby promoting the user's learning habits, improving learning effectiveness, and making efficient use of study time.

[0726] "User" refers to an individual person who uses the system and provides English proficiency data and schedule information.

[0727] "English proficiency data" is a collection of information indicating a user's English proficiency, such as a user's TOEIC score or English skill level.

[0728] "Skill level" refers to the level of a user's English ability that is evaluated based on specific standards, and is classified as beginner, intermediate, advanced, etc.

[0729] A "training menu" is a set of English learning tasks and exercises selected according to the user's skill level.

[0730] "Schedule information" is information that indicates the user's daily plans, and mainly includes calendar information.

[0731] A "notification" is a message sent to a user terminal at a specified time, informing the user of the start time of a training menu, etc.

[0732] "User terminal" refers to a device that a user operates directly to study English, including smartphones and tablets.

[0733] A "learning application" is software that is installed on a user's device and provides learning materials and training menus for English language learning.

[0734] "Progress data" refers to information about the learning progress and results recorded as the user performs the training menu.

[0735] "Server" is a computer that receives English proficiency data and progress data from users and provides feedback based on this data.

[0736] "Feedback" refers to information provided based on the user's progress data, including an evaluation of the learning results and advice for the next learning session.

[0737] A "generative model" is a machine learning model that automatically generates optimal training menus based on English proficiency data and schedule information.

[0738] A "prompt sentence" is a sentence input to a generative model, and is a specific instruction sentence created based on the user's learning content and purpose.

[0739] This invention is a system for effectively supporting users' English learning, and in particular, it supports the establishment of study habits and efficient learning through individualized study training according to the user's ability and effective schedule management. The invention utilizes the following hardware and software:

[0740] Hardware and software used

[0741] Server: A web server (e.g., Apache) and a database (e.g., MongoDB) are used to receive data, assess skill levels, select training menus, analyze progress data, and run generative models.

[0742] User device: A smartphone or tablet (e.g., iOS or Android) is assumed to be the device that users will operate directly to study English.

[0743] Google Calendar API: Used to retrieve user schedule information.

[0744] Outgoing Mail Server: Uses SMTP to send training session notifications to users.

[0745] Natural language explanation of the process

[0746] Main functions and processes of the server

[0747] 1. Receiving user data

[0748] The server receives the English proficiency data (e.g., TOEIC score, English skill level) provided by the user and records it in a database.

[0749] 2. Skill Level Assessment

[0750] The server automatically evaluates the user's skill level based on the received English proficiency data. For example, if the user's TOEIC score is 650, the server evaluates the user's skill level as "intermediate."

[0751] 3. Selecting a training menu

[0752] The server automatically selects a training menu appropriate for the assessed skill level, for example, "intermediate reading training" or "intermediate listening training."

[0753] 4. Schedule Management

[0754] The server uses the Google Calendar API to retrieve the user's schedule, find available time slots, and schedule suitable training sessions.

[0755] 5. Sending notifications

[0756] When the scheduled start time of the training approaches, the server sends a notification to the user terminal using the mail sending server.

[0757] 6. Automatic generation of training menus

[0758] The server uses the generative model to automatically generate the optimal training menu for the user based on English proficiency data and schedule information.

[0759] 7. Prompt generation for generative AI models

[0760] The server generates prompt sentences to be given to the generative AI model based on the user's learning content and goals.

[0761] Main functions and processing of the user terminal

[0762] 1. Receiving notifications

[0763] The user terminal receives the notification sent from the server and presents the content and start time of the next learning session.

[0764] 2. App auto-start

[0765] At the specified time, the learning application installed on the user's device will automatically start and the training will begin.

[0766] 3. Recording and sending progress data

[0767] It records your training progress in real time and sends the progress data to a server once your learning session is over.

[0768] Specific examples

[0769] For example, when a user with a TOEIC score of 650 initially configures the system, the server evaluates the user as "intermediate" and selects "intermediate reading training" and "intermediate listening training." The server then checks the user's Google Calendar to find available time between 7:30 and 8:00 a.m. on weekdays. Training is scheduled for this time period and a notification is sent to the user's device. At the designated time of 7:30 a.m., the learning application installed on the user's device automatically launches and the training begins. When the training session ends, progress data is sent to the server, and feedback is provided. The server also uses a generative model to automatically generate the next training menu, maximizing the user's learning effectiveness.

[0770] Prompt Sentence Examples

[0771] My TOEIC score is 650. Please suggest a training menu suitable for intermediate level and schedule study sessions during your free time. Please also send me a notification before the next session starts. The Google Calendar information is as follows: [Calendar Information].

[0772] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0773] Step 1:

[0774] Receiving user data

[0775] The server receives English proficiency data (e.g., TOEIC score or English skill level) entered by the user. User data is entered via a web form or similar and sent to the server. Input: TOEIC score, English skill level. Output: Recording of received data in a database. Specifically, the server receives an HTTP request and saves the data in a database (e.g., MongoDB).

[0776] Step 2:

[0777] Skill Level Assessment

[0778] The server evaluates the user's skill level based on the received English proficiency data. It analyzes the received data (e.g., TOEIC score) and determines the skill level. Input: TOEIC score. Output: Skill level (e.g., "Intermediate"). Specifically, the server applies predefined evaluation logic based on the data it receives to determine the skill level.

[0779] Step 3:

[0780] Selection of training menu

[0781] The server automatically selects a training menu according to skill level. It retrieves an appropriate training menu from the database and proposes it to the user. Input: Skill level. Output: Training menu. Specifically, the server selects candidates from a list of training menus pre-registered in the database based on the corresponding skill level.

[0782] Step 4:

[0783] Get schedule information

[0784] The server uses the Google Calendar API to obtain the user's schedule information. Input: User's calendar information. Output: Schedule information (such as a list of free time slots). Specifically, the server sends authentication information to the Google Calendar API and obtains events from the user's calendar.

[0785] Step 5:

[0786] Schedule a training session

[0787] The server detects available time slots based on the acquired schedule information and schedules training sessions. Input: Schedule information, training menu. Output: Scheduled training session. Specifically, the server analyzes the schedule information and associates the optimal time slot with the training menu.

[0788] Step 6:

[0789] Sending notifications

[0790] The server sends a notification to the user device when the scheduled training session is about to start. Input: Scheduled training session. Output: Notification message to the user device. Specifically, the server uses an email server or push notification service to send the specified message to the user.

[0791] Step 7:

[0792] App autostart

[0793] The user device will automatically launch the installed learning application at the specified time. Input: Notification message. Output: Launching of learning application. Specifically, the device will launch the learning application that is set as a trigger in response to the received notification message.

[0794] Step 8:

[0795] Recording and sending progress data

[0796] The user's device records the training progress in real time, and when the learning session ends, it sends the progress data to the server. Input: Operation data during training. Output: Sending progress data to the server. Specifically, the device records the user's answers and operations, and sends the data to the server via an HTTP request when the learning session ends.

[0797] Step 9:

[0798] Providing feedback

[0799] The server provides feedback to the user based on the submitted progress data. Input: Progress data. Output: Feedback message. Specifically, the server analyzes the progress data and generates and sends a message to the user based on predefined feedback logic.

[0800] Step 10:

[0801] Prompt generation for generative AI models

[0802] The server generates a prompt sentence to be given to the generative AI model based on English proficiency data and schedule information. Input: English proficiency data, schedule information. Output: Prompt sentence. Specifically, the server creates a prompt sentence appropriate for the generative AI model based on the specified data and provides it to the model.

[0803] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0804] ---

[0805] This invention is a system that effectively supports users' English learning, and in particular, is a system that can optimize the learning experience by recognizing the user's emotions. It provides training menus based on the user's input data and schedule information, and furthermore, uses emotion recognition functions to flexibly adjust the learning content and schedule, thereby achieving more effective learning.

[0806] Main functions and processes of the server

[0807] 1. User data reception function

[0808] The server receives data such as TOEIC scores and English skill levels sent by users, providing the initial information for creating a personalized learning plan for each user.

[0809] 2. Skill evaluation function

[0810] The server evaluates the user's skill level based on the received English proficiency data. For example, if the TOEIC score is 650, the user's skill level is evaluated as "intermediate."

[0811] 3. Emotion recognition function

[0812] The server analyzes facial expression and voice data sent from the user device and uses an emotion recognition engine to determine the user's emotional state, for example, to obtain information such as whether the user is tired or stressed.

[0813] 4. Training menu selection function

[0814] The server selects the most appropriate training menu based on the user's skill level and emotional state. For example, if the user is tired, a menu with a lower level of difficulty can be selected.

[0815] 5. Schedule management function

[0816] The server retrieves the user's schedule information, such as Google Calendar, and finds available time slots. It then schedules training sessions for those available times.

[0817] 6. Notification function

[0818] The server sends a notification to the user device at the start time of the scheduled workout, which includes a menu that is fine-tuned according to the user's emotional state.

[0819] Main functions and processing of the user terminal

[0820] 1. Notification reception function

[0821] The user terminal receives the notification sent from the server and notifies the user of the start time and training content.

[0822] 2. App auto-start function

[0823] Upon receiving the notification, at the specified time, the learning application installed on the user's device will automatically start.

[0824] 3. Learning Progression Function

[0825] The user device analyzes the user's facial expressions and voice data in real time and also records emotional changes during learning.

[0826] 4. Progress data transmission function

[0827] When the learning session ends, the user device transmits the training progress and emotional data to the server.

[0828] User operations

[0829] 1. Data Entry

[0830] When users first set up the system, they input their TOEIC score and English skill level, and also link the system with Google Calendar.

[0831] 2. Notification confirmation

[0832] The user checks the notification from the server to find out the start time and content of the next learning session.

[0833] 3. Study with apps

[0834] At the specified time, the user begins training using the automatically launched app.

[0835] 4. Feedback confirmation

[0836] After studying, users can check the feedback and progress data displayed to understand their learning status.

[0837] Specific examples

[0838] For example, suppose a user with a TOEIC score of 650 is registered in the system. In this case, the server evaluates the skill level as "intermediate." The server then analyzes the user's facial expression and voice data and determines that the user is feeling fatigued. The server selects a relatively easy task from the "intermediate reading training" and schedules it for an available time in the user's Google calendar. The server determines that this training session will take place from 7:30 to 8:00 AM on a weekday. A notification is sent to the user's device, and the learning application automatically launches at the specified time. The user performs the "intermediate reading training," and upon completion, progress data and emotional data are sent to the server. The server analyzes the data and provides the user with feedback, such as "80% accuracy in reading, high stress level."

[0839] As such, the present invention is a system that can provide flexible training tailored to the user's skill level and emotional state, and efficiently manage progress and provide feedback in real time, allowing users to efficiently utilize their spare time to study English and maximize the effectiveness of their learning.

[0840] The processing flow will be explained below.

[0841] ---

[0842] Step 1:

[0843] During the initial setup, users enter their TOEIC score, English skill level, and Google Calendar integration information into the device, which then sends this data to the server.

[0844] Step 2:

[0845] The server receives the TOEIC score and English skill level data sent by the user. For example, if a TOEIC score of 650 is used, the English skill level is set as "intermediate."

[0846] Step 3:

[0847] The server evaluates the user's skill level based on the received data. For example, a TOEIC score of 650 would be rated as "intermediate."

[0848] Step 4:

[0849] The server requests permission to retrieve the user's Google Calendar information. If the user grants access, the server retrieves the calendar information.

[0850] Step 5:

[0851] The server detects available time from the acquired schedule information. For example, it identifies available time from 7:30 to 8:00 AM on weekdays.

[0852] Step 6:

[0853] The user terminal collects the user's facial expression and voice data in real time and transmits them to the server, which then recognizes the user's emotional state.

[0854] Step 7:

[0855] The server uses an emotion recognition engine to determine the user's emotional state, for example, determining that the user is feeling tired.

[0856] Step 8:

[0857] The server selects a training menu based on the user's skill level and emotional state, for example, selecting "intermediate reading training" and adjusting the difficulty level.

[0858] Step 9:

[0859] The server sends a notification to the user device when the scheduled training session approaches, with a message such as "The next learning session will start at 7:30."

[0860] Step 10:

[0861] The user terminal receives the notification and displays it to the user as a pop-up message, and the user confirms the start time of the next learning session.

[0862] Step 11:

[0863] At the specified time (7:30 AM in this case), the user's device will automatically launch the English learning application, and the user will begin training in the app.

[0864] Step 12:

[0865] The user progresses through the learning process according to a designated training menu (e.g., "Intermediate Reading Training"). During the training, the user's facial expressions and voice data are collected again, and emotional changes are recorded.

[0866] Step 13:

[0867] When the learning session ends, the user's device records the training progress and emotional data and transmits it to the server.

[0868] Step 14:

[0869] The server analyzes the received progress data and emotion data and provides feedback to the user, such as "80% correct answer rate in reading, high stress level."

[0870] By taking these steps, users can efficiently utilize their spare time to study English and obtain an optimal learning experience that takes into account their emotional state, thereby improving learning effectiveness and helping to establish consistent learning habits.

[0871] Example 2

[0872] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0873] Conventional English learning support systems are limited in providing training tailored to the user's skill level, and have had the problem of difficulty in providing flexible learning content that responds to the user's emotional state and real-time progress.In addition, they lack the ability to automatically set a learning schedule that utilizes the user's free time, or to provide feedback based on emotional recognition during learning, making it impossible to provide an optimal learning experience for users.

[0874] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0875] In this invention, the server includes means for receiving language ability data from a user, means for evaluating the user's skill level based on the received language ability data, means for recognizing the user's skill level and emotional state and automatically selecting a training menu accordingly, means for acquiring the user's schedule information, means for scheduling a training menu during free time in the acquired schedule information and sending a notification to the user terminal at the training start time, means for automatically starting learning software installed on the user terminal at a specified time, means for acquiring the user's progress and emotional data in real time while learning, and means for analyzing the acquired progress and emotional data, generating feedback, and providing it to the user. This makes it possible to provide flexible and optimal learning content according to the user's emotional state and real-time progress, and to automatically set an efficient learning schedule that utilizes the user's free time and provide feedback in real time.

[0876] "Language Proficiency Data" refers to TOEIC scores and other data indicating English proficiency for assessing a user's English skills.

[0877] The "skill level" is a level indicating the user's English ability as a result of evaluating the user's English ability.

[0878] "Emotional state" refers to the user's psychological and emotional state, which is determined by analyzing the user's facial expressions and voice data.

[0879] A "training menu" is a set of specific exercises and tasks for learning English that are selected based on the user's skill level and emotional state.

[0880] "Schedule information" is information relating to a user's schedule and free time, and is managed, for example, by a calendar.

[0881] "Free time" is a time period in the user's schedule information in which no other plans are scheduled.

[0882] A "notification" is a message sent from the server to the user terminal, informing the user of the start time and content of the training menu.

[0883] "Learning software" refers to an application for learning English that is installed and used by users.

[0884] "Progress" is information that indicates how much of a task a user has completed while studying English.

[0885] "Real-time" means that the actions and states of the user at that moment are reflected immediately as they progress through their learning process.

[0886] "Feedback" refers to evaluations and advice provided by the server regarding the learning a user has done.

[0887] The present invention provides a system for effectively supporting a user's English learning, and in particular, a system for optimizing the learning experience by recognizing the user's emotions. Specific embodiments for carrying out the present invention are described below.

[0888] Basic system configuration

[0889] This system consists of a server and a user device. The server receives language ability data and schedule information sent by the user, and uses this information to select training menus and manage schedules. It also uses an emotion recognition engine to determine the user's emotional state and flexibly adjust the learning content. The user device receives notifications from the server, automatically launches the learning software, and acquires learning progress data in real time and sends it to the server.

[0890] Hardware and Software

[0891] The entire system is implemented using the following hardware and software:

[0892] Server: Uses a cloud server or on-premise server to perform data analysis, emotion recognition and scheduling.

[0893] User devices: Use devices that can install learning software, such as computers, smartphones, or tablets.

[0894] Learning software: An application for learning English that includes the ability to record the user's learning progress and emotional state.

[0895] Emotion recognition engine: For example, emotion recognition is performed using Microsoft Azure's Emotion API.

[0896] Data processing and calculation

[0897] The server performs the following steps to process and calculate the data:

[0898] 1. Data reception and processing: The server receives language ability data (e.g., TOEIC score) and schedule information sent by the user and analyzes them as initial data.

[0899] 2. Skill level assessment: Based on the received language proficiency data, a calculation process is performed to assess the user's skill level.

[0900] 3. Emotion Recognition: An emotion recognition engine is used to analyze the user's facial expressions and voice data to determine their emotional state.

[0901] 4. Training Menu Selection: Select an appropriate training menu based on assessed skill level and emotional state.

[0902] 5. Schedule Management: Detects free time from user schedule information and automatically schedules training sessions.

[0903] 6. Notifications and Feedback: Send notifications at the scheduled start time of training, and provide feedback after learning by analyzing progress and sentiment data obtained.

[0904] Specific examples

[0905] For example, suppose a user with a TOEIC score of 650 registers with the system. Based on the received score, the server evaluates the user's skill level as "intermediate." The server then analyzes the user's facial expression and voice data and determines that the user is tired.

[0906] The server selects a lower level of difficulty from the "Intermediate Reading Training" and schedules the training session during the user's free time (e.g., weekdays from 7:30 to 8:00 a.m.). At the designated time, the learning software installed on the user's device automatically starts up and the training begins.

[0907] After the user completes the training, the learning software sends progress and emotional data to the server, which analyzes the progress and emotional data and provides feedback to the user, such as "80% correct reading, high stress level."

[0908] Prompt Sentence Examples

[0909] "If a user with a TOEIC score of 650 is determined to be fatigued, generate a prompt from the AI ​​system that selects relatively easy intermediate reading training tasks and provides a way to schedule the training during the user's free time."

[0910] As such, the present invention is a system that provides flexible and optimal learning according to the user's skill level and emotional state, efficiently manages learning schedules by utilizing free time, and provides real-time feedback.

[0911] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0912] Step 1:

[0913] Entering and submitting user data

[0914] During initial setup, the user enters their TOEIC score and English skill level and sends them to the server. The system also links with Google Calendar. The input for this step is the user's language ability data and schedule information, such as "TOEIC score 650, skill level intermediate." The output is the user data sent to the server.

[0915] Step 2:

[0916] Data reception by the server and skill evaluation

[0917] The server receives the language ability data sent by the user and evaluates the user's skill level based on this. The input to this step is the language ability data sent by the user. The data is saved in an internal database and an evaluation process is performed. For example, if the user's TOEIC score is 650, the user's skill level is evaluated as "intermediate." The output is the data that evaluates the user's skill level.

[0918] Step 3:

[0919] Emotion recognition by server

[0920] The server analyzes the facial expression and voice data sent from the user device to determine the user's emotional state. The input for this step is the facial expression and voice data sent from the user device. An emotion recognition engine (e.g., Emotion API) is used to analyze the data and determine whether the user is tired. The output is data indicating the user's emotional state.

[0921] Step 4:

[0922] Selection of training menu

[0923] The server selects the optimal training menu based on the user's skill level and emotional state. The input for this step is the user's skill level data and emotional state data. A data analysis algorithm is used to identify a training menu suitable for the user. For example, if the user is tired, a less difficult "intermediate reading training" is selected. The output is the selected training menu.

[0924] Step 5:

[0925] Schedule registration by server

[0926] The server retrieves schedule information from the user's Google Calendar and detects available time slots. The input to this step is the user's schedule information. The server analyzes the available time slots and schedules training sessions for those times. For example, the server schedules sessions from 7:30 to 8:00 AM on weekdays. The output is the training sessions registered in the schedule.

[0927] Step 6:

[0928] Sending notifications

[0929] The server sends a notification to the user terminal at the scheduled training start time. The input for this step is the schedule information and the selected training menu. The notification system is used to send the specified message to the user terminal. The output is a notification displayed on the user terminal.

[0930] Step 7:

[0931] Receiving notifications on the user device and automatically launching the app

[0932] The user device receives the notification sent from the server and automatically launches the learning software at the specified time. The input to this step is the notification from the server. The software is launched at the specified time using the application auto-launch function. For example, at 7:30 AM, the app to start "Intermediate Reading Training" is launched. The output is the launched learning software.

[0933] Step 8:

[0934] User-led learning sessions

[0935] The user trains using the automatically launched learning software. The inputs of this step are the launched learning software and the training menu. The user progresses through the specified training. The outputs are the training progress and emotion data.

[0936] Step 9:

[0937] Progress data transmission by user terminal

[0938] After the training session is completed, the user device sends the progress and emotion data to the server. The input of this step is the progress and emotion data collected during training. The data transmission function is used to transmit this data to the server. The output is the progress and emotion data transmitted to the server.

[0939] Step 10:

[0940] Data analysis by server and feedback transmission

[0941] The server analyzes the received data and sends feedback to the user. The input of this step is the user's progress data and emotional data. Using a data analysis algorithm, the server identifies the user's learning achievements and areas for improvement and generates feedback. For example, it generates feedback such as "80% correct reading, high stress level." The output is the generated feedback.

[0942] (Application example 2)

[0943] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0944] Conventional English learning support systems focus on providing training menus based on the user's skill level, but do not adequately address optimization based on the user's emotional state or learning environment. As a result, even if the user feels tired or stressed, a fixed learning menu is provided, which can reduce learning efficiency. Furthermore, systems lack functionality to effectively support learning activities in physical stores, and it is necessary to provide an optimal learning experience tailored to the store environment.

[0945] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0946] In this invention, the server includes means for receiving English proficiency data from a user, means for evaluating the user's skill level based on the received English proficiency data, means for automatically selecting a training menu according to the user's skill level, means for acquiring the user's schedule information, means for scheduling a training menu during free time in the acquired schedule information, means for transmitting the scheduled training menu to a user device at a start time, means for automatically starting a learning program installed on the user device at a specified time, means for optimizing the learning content based on the store environment, and means for recognizing the user's emotional state and adjusting the learning content. This enables flexible training provision according to the user's emotional state and learning environment, and effective learning support within a physical store.

[0947] The "means for receiving English ability data from users" is a system that allows the server to receive ability data such as TOEIC scores and English skill levels provided by users.

[0948] The "means for assessing the user's skill level based on the received English proficiency data" is a system for analyzing the received data and assessing the user's current English proficiency based on specific criteria.

[0949] The "means for automatically selecting a training menu according to the user's skill level" is a system for automatically selecting the most suitable English training program based on the evaluated skill level information.

[0950] "Means for obtaining user schedule information" refers to a system that allows the server to obtain schedule information such as the user's Google Calendar.

[0951] The "means for scheduling a training menu during free time in the acquired schedule information" is a system for identifying free time in the acquired schedule and allocating an English training session to that time.

[0952] The "means for transmitting to the user device at the start time of the scheduled training menu" is a system for sending a notification to the user's device, such as a smartphone or tablet, at the set training start time.

[0953] "Means for automatically starting a learning program installed on a user device at a specified time" refers to a system for automatically starting learning software installed on a user's terminal at a specified time.

[0954] The "means for optimizing learning content based on the store environment" is a system that collects environmental data such as noise levels and congestion levels in physical stores and optimizes English training content accordingly.

[0955] The "means for recognizing the user's emotional state and adjusting the learning content" is a system that uses a camera and microphone to analyze the user's facial expressions and tone of voice, and adjusts the English training content according to the recognized emotions.

[0956] The present invention is a system for effectively supporting users' English learning in a brick-and-mortar store, and in particular, a system that can recognize users' emotions and optimize the learning experience. The system includes the following main elements:

[0957] Main functions and processes of the server

[0958] 1. User data reception function

[0959] The server receives data such as TOEIC scores and English skill levels sent by users, providing the initial information for creating a personalized learning plan for each user.

[0960] 2. Skill evaluation function

[0961] The server evaluates the user's skill level based on the received English proficiency data. For example, if the TOEIC score is 650, the user's skill level is evaluated as "intermediate."

[0962] 3. Emotion recognition function

[0963] The server analyzes facial expression and voice data sent from the user's device and uses an emotion recognition engine to determine the user's emotional state. For example, it acquires information such as whether the user is tired or stressed. The hardware used includes a camera and microphone, and the software used includes a facial expression recognition API and a voice recognition API.

[0964] 4. Training menu selection function

[0965] The server selects the most appropriate training menu based on the user's skill level and emotional state. For example, if the user is tired, a menu with a lower level of difficulty can be selected.

[0966] 5. Schedule management function

[0967] The server retrieves the user's calendar information, finds available time slots, and schedules training sessions within those times. The software used includes the Google Calendar API.

[0968] 6. Notification function

[0969] The server sends a notification to the user device at the start time of the scheduled workout, which includes a menu that is fine-tuned according to the user's emotional state.

[0970] Main functions and processing of the user terminal

[0971] 1. In-store detection function

[0972] The user device uses Wi-Fi or Bluetooth beacons to detect when the user is in a physical store.

[0973] 2. Environment recognition function

[0974] The user device recognizes the noise level and occupancy status inside the store and sends the information to the server. The hardware used includes a microphone, and the software used includes a noise-canceling algorithm.

[0975] 3. Notification reception function

[0976] The user terminal receives the notification sent from the server and notifies the user of the start time and training content.

[0977] 4. App auto-start function

[0978] Upon receiving the notification, the learning program installed on the user's terminal will automatically start at the specified time.

[0979] 5. Learning Progression Function

[0980] The user device analyzes the user's facial expressions and voice data in real time and also records emotional changes during learning.

[0981] 6. Progress data transmission function

[0982] When the learning session ends, the user device transmits the training progress and emotional data to the server.

[0983] Specific examples

[0984] For example, suppose a user with a TOEIC score of 650 is registered in the system. In this case, the server evaluates the skill level as "intermediate." Then, it analyzes the user's facial expression and voice data and determines that the user is feeling fatigued. The server selects a relatively easy task from "intermediate reading training" and schedules it for an available time in the user's calendar. It determines that this training session will take place, for example, from 7:30 to 8:00 AM on a weekday. A notification is sent to the user's device, and the learning program automatically starts at the specified time. The user performs the "intermediate reading training," and after it is completed, progress data and emotional data are sent to the server.

[0985] Prompt Sentence Examples

[0986] "I'd like to do some listening practice during the cafe time today. If the cafe is quiet, I'd like more difficult questions, and if it's noisy, I'd like easier questions."

[0987] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0988] Step 1: Receiving user data

[0989] The server receives data such as TOEIC scores and English skill levels sent by users. When the user enters this data in the application and presses the submit button, the server receives the data and stores it in a database.

[0990] Input: TOEIC score, English skill level data

[0991] Output: Save user data to database

[0992] Step 2: Skills Assessment

[0993] The server evaluates the user's skill level based on the received English proficiency data. For example, if the TOEIC score is 650, the server will evaluate it as "intermediate." An evaluation algorithm is applied to this data to determine the user's skill level.

[0994] Input: English proficiency data

[0995] Output: Assessed skill level (e.g., intermediate)

[0996] Step 3: In-store detection

[0997] The user device uses Wi-Fi or Bluetooth beacons to detect when the user is in a physical store, which then transmits specific store environment information to the server.

[0998] Input: Signals from Wi-Fi or Bluetooth beacons

[0999] Output: Confirmation that the user is in the store

[1000] Step 4: Environmental awareness

[1001] The user device recognizes the noise level and congestion status inside the store and sends the information to the server. Specifically, it uses a microphone to collect surrounding sounds and analyzes the noise level.

[1002] Input: Audio data from the microphone

[1003] Output: Noise level, congestion data

[1004] Step 5: Emotion Recognition

[1005] The server analyzes the facial expression and voice data sent from the user's device and uses an emotion recognition engine to determine the user's emotional state, for example, extracting information such as whether the user is tired or stressed.

[1006] Input: facial expression data, voice data

[1007] Output: Emotional state assessment (e.g., tired)

[1008] Step 6: Choose your training menu

[1009] The server selects the most appropriate training menu based on the user's skill level and emotional state: for example, if the user is tired, it selects a reading training with a lower level of difficulty.

[1010] Input: Skill level, emotional state

[1011] Output: Selected training menu

[1012] Step 7: Retrieve schedule information

[1013] The server retrieves the user's calendar information, using the Google Calendar API to read the user's schedule and identify available times.

[1014] Input: User's Google Calendar information

[1015] Output: Free / busy information

[1016] Step 8: Training Schedule

[1017] The server schedules the training menu for the available time slots in the acquired schedule information. For example, the server may set a training session for the available time slots between 7:30 and 8:00 AM on weekdays.

[1018] Input: Free / busy information, training menu

[1019] Output: Scheduled training sessions

[1020] Step 9: Send notification

[1021] The server sends a notification to the user terminal at the start time of the scheduled training menu, which also includes the training content.

[1022] Input: Schedule a training session

[1023] Output: Notification to user terminal

[1024] Step 10: App Auto-Launch

[1025] The user terminal automatically starts the installed learning program at the specified time. Upon receiving the notification, the user terminal starts the program and begins training.

[1026] Input: Notification content

[1027] Output: Automatic launch of learning program

[1028] Step 11: Record your learning progress and emotional changes

[1029] The user device analyzes the user's facial expressions and voice data in real time and records emotional changes during learning. Specifically, it collects and analyzes data using a camera and microphone.

[1030] Input: Real-time facial expression data, voice data

[1031] Output: Recorded data of emotional changes

[1032] Step 12: Send progress data

[1033] Once the training session is over, the user device transmits training progress and emotional data to the server, which stores this data in a database for further analysis.

[1034] Input: Progress data, emotion data

[1035] Output: Send data to the server

[1036] Examples of prompts:

[1037] "I'd like to do some listening practice during the cafe time today. If the cafe is quiet, I'd like more difficult questions, and if it's noisy, I'd like easier questions."

[1038] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1039] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1040] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1041] [Third embodiment]

[1042] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1043] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1044] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1045] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1046] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1047] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1048] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1049] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1050] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1051] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1052] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1053] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1054] ---

[1055] The present invention is a system for effectively supporting users' English learning, and aims to automatically provide optimal training menus based on English proficiency data entered by the user, and to efficiently manage learning time according to the user's schedule. This system is mainly composed of a server, a user terminal, and processing steps including the exchange of input data from the user.

[1056] Main functions and processes of the server

[1057] 1. User data reception function

[1058] When users initially set up the system, they input their TOEIC score and English skill level, and the server receives and records this data.

[1059] 2. Skill evaluation function

[1060] The server automatically evaluates the user's skill level based on the received English proficiency data. For example, if the TOEIC score is 650, the server will evaluate the user's skill level as "intermediate."

[1061] 3. Training menu selection function

[1062] The server automatically selects a training menu appropriate for the assessed skill level, for example, "intermediate reading training" or "intermediate listening training."

[1063] 4. Schedule management function

[1064] The server retrieves the user's schedule information, such as Google Calendar, and finds available time slots. It then schedules training sessions that fit the available time slots.

[1065] 5. Notification function

[1066] When the scheduled training session is about to begin, the server sends a notification to the user's device, with a message like "Your next training session will begin at 7:30."

[1067] Main functions and processing of the user terminal

[1068] 1. Notification reception function

[1069] The user device receives a notification sent from the server, which indicates the content and start time of the next learning session.

[1070] 2. App auto-start function

[1071] The device that receives the notification will automatically launch the English learning app at the specified time. For example, "Intermediate Reading Training" will start at 7:30 AM.

[1072] 3. Learning Progression Function

[1073] The user can then use the app to progress through the training, which will then record the progress of the training and the user's responses in real time.

[1074] 4. Progress data transmission function

[1075] At the end of a study session, the app sends the user's progress data to the server, such as "80% correct reading" or "75% correct listening."

[1076] User operations

[1077] 1. Data Entry

[1078] During initial setup, users enter their TOEIC score, English skill level, and Google Calendar connection information, which allows the server to suggest appropriate training menus and schedules.

[1079] 2. Notification confirmation

[1080] The user checks the notification sent from the server to find out when the next learning session will start.

[1081] 3. Study with apps

[1082] At the designated time, the user will use the automatically launched app to carry out the training, which will help develop a natural learning habit.

[1083] 4. Feedback confirmation

[1084] Once the training is complete, users can view their progress and feedback from the app.

[1085] Specific examples

[1086] For example, if a user with a TOEIC score of 650 provides input data to the system, the server will evaluate the user as "intermediate." The server then selects "intermediate reading training" and "intermediate listening training." It then checks Google Calendar to find available time between 7:30 and 8:00 a.m. on weekdays. The server schedules training for this time period and sends a notification to the user's device. At 7:30 a.m., the learning app installed on the user's device will automatically launch and the training will begin. When the user completes the training, progress data is sent to the server, which then provides feedback based on that data.

[1087] As described above, the present invention is a system that can provide flexible training according to the user's skill level and schedule, and can efficiently manage progress.

[1088] The processing flow will be explained below.

[1089] ---

[1090] Step 1:

[1091] The server receives the TOEIC score and English skill level data sent by the user. Specifically, the score and skill information entered by the user on the initial setting screen is sent to the server.

[1092] Step 2:

[1093] The server automatically evaluates the user's skill level based on the received English proficiency data. For example, if the TOEIC score is 650, the server evaluates the user's skill level as "intermediate."

[1094] Step 3:

[1095] The server automatically selects a training menu according to the user's skill level, such as "Intermediate Reading Training" or "Intermediate Listening Training."

[1096] Step 4:

[1097] The server requests permission to access the schedule information from the user to obtain the schedule information of the user's Google Calendar, etc. After the user grants permission, the server obtains the user's schedule data.

[1098] Step 5:

[1099] The server detects available times from the acquired schedule information. For example, it determines that available times are from 7:30 to 8:00 a.m. on weekdays.

[1100] Step 6:

[1101] The server schedules training sessions during the available times it finds, for example, allocating "Intermediate Reading Training" between 7:30 and 8:00 AM.

[1102] Step 7:

[1103] The server sends a notification to the user device 15 minutes before the scheduled training, with a message such as "The next learning session will start at 7:30."

[1104] Step 8:

[1105] The user terminal receives the notification from the server and displays a pop-up message to the user, who then confirms the start time.

[1106] Step 9:

[1107] At the specified time (7:30 AM in this case), the user's device automatically launches the English learning application, and the user begins training in the app.

[1108] Step 10:

[1109] The user progresses through the learning process by following the designated training menu, for example, answering questions in the "Intermediate Reading Training" section.

[1110] Step 11:

[1111] Once the learning session is over, the user's device records the training progress and results and sends them to the server.

[1112] Step 12:

[1113] The server analyzes the received progress data and provides feedback to the user, such as "Your accuracy rate for this reading training was 80%."

[1114] By following these steps, users can study English efficiently in their spare time and naturally develop a study habit. Real-time progress management and feedback also improve the effectiveness of users' learning.

[1115] Example 1

[1116] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1117] Conventional English learning systems have difficulty flexibly adapting to individual users' skill levels and schedules, preventing them from maximizing learning effectiveness. They also lacked a mechanism for recording users' progress in real time or the ability to provide feedback based on that progress. This prevented users from accurately understanding their own learning progress and results, hindering effective learning.

[1118] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1119] In this invention, the server includes means for receiving English proficiency data from a user, means for evaluating the user's skill level based on the received English proficiency data, means for automatically selecting a training menu according to the user's skill level, means for acquiring the user's schedule information, means for scheduling a training menu during free time in the acquired schedule information, means for sending a notification to the user terminal at the start time of the scheduled training menu, means for automatically starting a learning application installed on the user terminal at a specified time, means for the learning application to record the user's training progress in real time, means for sending user progress data to the server after training ends, and means for providing feedback based on the progress data to the user terminal. This enables efficient learning tailored to the user's schedule and skill level, and by accurately understanding the user's learning progress and receiving feedback, effective learning can be achieved.

[1120] "User" refers to an individual or organization that uses the English Learning System.

[1121] "English Proficiency Data" refers to data necessary for assessing a user's English proficiency, such as a user's TOEIC score or English skill level.

[1122] "Skill level" refers to a user's proficiency in understanding and practicing English as assessed based on English proficiency data.

[1123] "Training Menu" refers to a set of English learning sessions and assignments provided according to the user's skill level.

[1124] "Schedule information" refers to data such as calendar information that indicates the user's schedule and free time.

[1125] "Real-time" refers to the fact that progress and achievements are recorded instantly while a user is using a learning application.

[1126] "Progress data" refers to data that indicates the user's learning status, such as the results achieved by the user during a study session and the percentage of correct answers.

[1127] "Feedback" refers to information provided to the user based on progress data, such as indications of learning results and next goals.

[1128] "Server" refers to the central processing unit that receives and analyzes user data, selects training menus, and manages schedules.

[1129] "User terminal" refers to a computing device on which a user installs and uses a learning application.

[1130] "Learning Application" refers to software that a user uses to train their English language skills.

[1131] "Notification" refers to a message or alert sent from a server to a user device.

[1132] This invention is a system for effectively supporting users' English learning. This system provides an optimal training menu based on the user's English proficiency data and efficiently manages learning time according to the user's schedule. It is mainly composed of a server, a user terminal, and processing steps including the exchange of input data from the user.

[1133] Specific server operations

[1134] User data reception function

[1135] When users initially set up the system, they input their TOEIC score and English skill level, and the server receives and records this data.

[1136] Skill evaluation function

[1137] The server automatically evaluates the user's skill level based on the received English proficiency data. For example, if the TOEIC score is 650, the server will evaluate the user's skill level as "intermediate."

[1138] Training menu selection function

[1139] The server automatically selects a training menu according to the assessed skill level, for example, "intermediate reading training" or "intermediate listening training."

[1140] Schedule management function

[1141] The server retrieves the user's schedule information (e.g., Google Calendar), finds available time slots, and schedules training sessions that fit the available time slots.

[1142] Notification function

[1143] When the scheduled training session is about to begin, the server sends a notification to the user's device, for example, a message saying, "Your next training session will begin at 7:30."

[1144] Specific operation of the user terminal

[1145] Notification reception function

[1146] The user device receives a notification sent from the server, which indicates the content and start time of the next learning session.

[1147] App auto-start function

[1148] The device that receives the notification will automatically launch the English learning app at the specified time. For example, "Intermediate Reading Training" will start at 7:30 AM.

[1149] Learning Progression Function

[1150] The user can then use the app to progress through the training, which will then record the progress of the training and the user's responses in real time.

[1151] Progress data transmission function

[1152] At the end of a study session, the app sends the user's progress data to the server, such as "80% correct reading" or "75% correct listening."

[1153] Specific user operations

[1154] Data Entry

[1155] During initial setup, users enter their TOEIC score, English skill level, and Google Calendar connection information, which allows the server to suggest appropriate training menus and schedules.

[1156] Notification confirmation

[1157] The user checks the notification sent from the server to find out when the next learning session will start.

[1158] Learn with the app

[1159] At the designated time, the app will automatically start and the training will begin, making it easier to naturally develop a study habit.

[1160] Feedback Check

[1161] Once the training is complete, users can view their progress and feedback from the app.

[1162] Specific examples

[1163] For example, a user with a TOEIC score of 650 provides input data to the system. The server evaluates the user as "intermediate." The server then selects "intermediate reading training" and "intermediate listening training." After that, it checks Google Calendar to find available time slots between 7:30 and 8:00 a.m. on weekdays. The server schedules the training for this time period and sends a notification to the user's device. At 7:30 a.m., the learning app installed on the user's device automatically launches and the training begins. When the user completes the training, progress data is sent to the server, which then provides feedback based on that data.

[1164] As described above, the present invention is a system that can provide flexible training according to the user's skill level and schedule, and can efficiently manage progress.

[1165] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1166] Step 1:

[1167] Initial data entry by the user

[1168] The user opens the system's initial setup screen and enters their TOEIC score, English skill level, and Google Calendar connection information, which then sends the initial data to the server.

[1169] Input: TOEIC score, English skill level, Google Calendar link information

[1170] Output: Initial data sent to the server

[1171] Step 2:

[1172] Data reception and analysis by the server

[1173] The server receives the initial data sent by the user, checks the integrity of the received data, and saves each item in the database.

[1174] Input: Initial data

[1175] Output: User data stored in the database

[1176] Specific operation: The server checks whether the TOEIC score and English skill level are within a reasonable range.

[1177] Step 3:

[1178] Server-based skill evaluation

[1179] The server analyzes the received data and evaluates the user's skill level. For example, if a TOEIC score is 650, it will be evaluated as "intermediate."

[1180] Input: User data

[1181] Output: Assessed skill level

[1182] Specific operation: The server analyzes the score using the evaluation algorithm.

[1183] Step 4:

[1184] Training menu selection by server

[1185] The server will select the most appropriate training menu based on your assessed skill level, such as "Intermediate Reading Training."

[1186] Input: Skill Level

[1187] Output: Training menu

[1188] Specific operation: The server retrieves the appropriate training menu from the database.

[1189] Step 5:

[1190] Server-based schedule management

[1191] The server retrieves the user's Google Calendar schedule information, finds available time slots, and schedules training sessions for those available times.

[1192] Input: Google Calendar schedule information

[1193] Output: Scheduled training sessions

[1194] Specific operation: The server analyzes the schedule information and identifies free time.

[1195] Step 6:

[1196] Server-based schedule notification

[1197] When the server approaches the start time of a scheduled training session, it sends a notification to the user's device, for example, "The next learning session will start at 7:30."

[1198] Input: Scheduled training sessions

[1199] Output: Notification sent to the user's device

[1200] Specific operation: The server sends an email or push notification

[1201] Step 7:

[1202] Receiving notifications and launching apps on your device

[1203] The user device receives the notification from the server and automatically launches the learning app at the specified time.

[1204] Input: Notification from the server

[1205] Output: Learning app launched

[1206] Specific operation: The device launches the learning app at the specified time.

[1207] Step 8:

[1208] User-driven training

[1209] The user starts the learning app and progresses through the training. The app records the user's progress in real time.

[1210] Input: User interaction

[1211] Output: Real-time recorded progress

[1212] What it does: The user performs the workout and the app records their progress in real time.

[1213] Step 9:

[1214] Progress data transmission by device

[1215] At the end of a study session, the app sends the user's progress data to a server, such as "80% correct reading."

[1216] Input: Progress data

[1217] Output: Progress data sent to the server

[1218] Specific behavior: The app detects the end of the session and sends progress data to the server.

[1219] Step 10:

[1220] Server-provided feedback

[1221] The server analyzes the submitted progress data and provides feedback to the user, such as "You got 80% of your reading correct, so try practicing faster next time."

[1222] Input: Progress data

[1223] Output: Feedback provided to the user's device

[1224] Specific behavior: The server analyzes the progress data, generates feedback, and sends it to the user.

[1225] This allows for efficient learning tailored to the user's schedule and skill level, and by accurately understanding the user's learning progress and receiving feedback, effective learning can be achieved.

[1226] (Application example 1)

[1227] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1228] Conventional English learning support systems often lack the ability to provide optimal training menus tailored to the user's skill level or manage schedules, hindering efficient learning. Furthermore, progress data recording and feedback provision are performed manually, reducing the continuity and effectiveness of learning. Furthermore, it is difficult to efficiently manage learning time using the user's schedule information. There is a need to provide an effective and efficient English learning support system that can solve these issues.

[1229] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1230] In this invention, the server includes means for receiving English proficiency data from a user, means for evaluating the user's skill level based on the received English proficiency data, means for automatically selecting a training menu according to the user's skill level, means for scheduling a training menu during free time in the acquired schedule information, means for sending a notification to the user terminal at the start time of the scheduled training menu, means for automatically launching a learning application installed on the user terminal at a specified time, means for monitoring the ongoing training menu in real time and recording progress data, means for sending the recorded progress data to the server, means for providing feedback to the user based on the sent progress data, means for automatically generating an optimal training menu for the user using a generative model based on the English proficiency data and schedule information, means for generating prompt sentences to be given to the generative AI model, and means for automatically supporting the user's learning behavior and providing appropriate feedback in real time, thereby promoting the user's learning habits, improving learning effectiveness, and making efficient use of study time.

[1231] "User" refers to an individual person who uses the system and provides English proficiency data and schedule information.

[1232] "English proficiency data" is a collection of information indicating a user's English proficiency, such as a user's TOEIC score or English skill level.

[1233] "Skill level" refers to the level of a user's English ability that is evaluated based on specific standards, and is classified as beginner, intermediate, advanced, etc.

[1234] A "training menu" is a set of English learning tasks and exercises selected according to the user's skill level.

[1235] "Schedule information" is information that indicates the user's daily plans, and mainly includes calendar information.

[1236] A "notification" is a message sent to a user terminal at a specified time, informing the user of the start time of a training menu, etc.

[1237] "User terminal" refers to a device that a user operates directly to study English, including smartphones and tablets.

[1238] A "learning application" is software that is installed on a user's device and provides learning materials and training menus for English language learning.

[1239] "Progress data" refers to information about the learning progress and results recorded as the user performs the training menu.

[1240] "Server" is a computer that receives English proficiency data and progress data from users and provides feedback based on this data.

[1241] "Feedback" refers to information provided based on the user's progress data, including an evaluation of the learning results and advice for the next learning session.

[1242] A "generative model" is a machine learning model that automatically generates optimal training menus based on English proficiency data and schedule information.

[1243] A "prompt sentence" is a sentence input to a generative model, and is a specific instruction sentence created based on the user's learning content and purpose.

[1244] This invention is a system for effectively supporting users' English learning, and in particular, it supports the establishment of study habits and efficient learning through individualized study training according to the user's ability and effective schedule management. The invention utilizes the following hardware and software:

[1245] Hardware and software used

[1246] Server: A web server (e.g., Apache) and a database (e.g., MongoDB) are used to receive data, assess skill levels, select training menus, analyze progress data, and run generative models.

[1247] User device: A smartphone or tablet (e.g., iOS or Android) is assumed to be the device that users will operate directly to study English.

[1248] Google Calendar API: Used to retrieve user schedule information.

[1249] Outgoing Mail Server: Uses SMTP to send training session notifications to users.

[1250] Natural language explanation of the process

[1251] Main functions and processes of the server

[1252] 1. Receiving user data

[1253] The server receives the English proficiency data (e.g., TOEIC score, English skill level) provided by the user and records it in a database.

[1254] 2. Skill Level Assessment

[1255] The server automatically evaluates the user's skill level based on the received English proficiency data. For example, if the user's TOEIC score is 650, the server evaluates the user's skill level as "intermediate."

[1256] 3. Selecting a training menu

[1257] The server automatically selects a training menu appropriate for the assessed skill level, for example, "intermediate reading training" or "intermediate listening training."

[1258] 4. Schedule Management

[1259] The server uses the Google Calendar API to retrieve the user's schedule, find available time slots, and schedule suitable training sessions.

[1260] 5. Sending notifications

[1261] When the scheduled start time of the training approaches, the server sends a notification to the user terminal using the mail sending server.

[1262] 6. Automatic generation of training menus

[1263] The server uses the generative model to automatically generate the optimal training menu for the user based on English proficiency data and schedule information.

[1264] 7. Prompt generation for generative AI models

[1265] The server generates prompt sentences to be given to the generative AI model based on the user's learning content and goals.

[1266] Main functions and processing of the user terminal

[1267] 1. Receiving notifications

[1268] The user terminal receives the notification sent from the server and presents the content and start time of the next learning session.

[1269] 2. App auto-start

[1270] At the specified time, the learning application installed on the user's device will automatically start and the training will begin.

[1271] 3. Recording and sending progress data

[1272] It records your training progress in real time and sends the progress data to a server once your learning session is over.

[1273] Specific examples

[1274] For example, when a user with a TOEIC score of 650 initially configures the system, the server evaluates the user as "intermediate" and selects "intermediate reading training" and "intermediate listening training." The server then checks the user's Google Calendar to find available time between 7:30 and 8:00 a.m. on weekdays. Training is scheduled for this time period and a notification is sent to the user's device. At the designated time of 7:30 a.m., the learning application installed on the user's device automatically launches and the training begins. When the training session ends, progress data is sent to the server, and feedback is provided. The server also uses a generative model to automatically generate the next training menu, maximizing the user's learning effectiveness.

[1275] Prompt Sentence Examples

[1276] My TOEIC score is 650. Please suggest a training menu suitable for intermediate level and schedule study sessions during your free time. Please also send me a notification before the next session starts. The Google Calendar information is as follows: [Calendar Information].

[1277] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1278] Step 1:

[1279] Receiving user data

[1280] The server receives English proficiency data (e.g., TOEIC score or English skill level) entered by the user. User data is entered via a web form or similar and sent to the server. Input: TOEIC score, English skill level. Output: Recording of received data in a database. Specifically, the server receives an HTTP request and saves the data in a database (e.g., MongoDB).

[1281] Step 2:

[1282] Skill Level Assessment

[1283] The server evaluates the user's skill level based on the received English proficiency data. It analyzes the received data (e.g., TOEIC score) and determines the skill level. Input: TOEIC score. Output: Skill level (e.g., "Intermediate"). Specifically, the server applies predefined evaluation logic based on the data it receives to determine the skill level.

[1284] Step 3:

[1285] Selection of training menu

[1286] The server automatically selects a training menu according to skill level. It retrieves an appropriate training menu from the database and proposes it to the user. Input: Skill level. Output: Training menu. Specifically, the server selects candidates from a list of training menus pre-registered in the database based on the corresponding skill level.

[1287] Step 4:

[1288] Get schedule information

[1289] The server uses the Google Calendar API to obtain the user's schedule information. Input: User's calendar information. Output: Schedule information (such as a list of free time slots). Specifically, the server sends authentication information to the Google Calendar API and obtains events from the user's calendar.

[1290] Step 5:

[1291] Schedule a training session

[1292] The server detects available time slots based on the acquired schedule information and schedules training sessions. Input: Schedule information, training menu. Output: Scheduled training session. Specifically, the server analyzes the schedule information and associates the optimal time slot with the training menu.

[1293] Step 6:

[1294] Sending notifications

[1295] The server sends a notification to the user device when the scheduled training session is about to start. Input: Scheduled training session. Output: Notification message to the user device. Specifically, the server uses an email server or push notification service to send the specified message to the user.

[1296] Step 7:

[1297] App autostart

[1298] The user device will automatically launch the installed learning application at the specified time. Input: Notification message. Output: Launching of learning application. Specifically, the device will launch the learning application that is set as a trigger in response to the received notification message.

[1299] Step 8:

[1300] Recording and sending progress data

[1301] The user's device records the training progress in real time, and when the learning session ends, it sends the progress data to the server. Input: Operation data during training. Output: Sending progress data to the server. Specifically, the device records the user's answers and operations, and sends the data to the server via an HTTP request when the learning session ends.

[1302] Step 9:

[1303] Providing feedback

[1304] The server provides feedback to the user based on the submitted progress data. Input: Progress data. Output: Feedback message. Specifically, the server analyzes the progress data and generates and sends a message to the user based on predefined feedback logic.

[1305] Step 10:

[1306] Prompt generation for generative AI models

[1307] The server generates a prompt sentence to be given to the generative AI model based on English proficiency data and schedule information. Input: English proficiency data, schedule information. Output: Prompt sentence. Specifically, the server creates a prompt sentence appropriate for the generative AI model based on the specified data and provides it to the model.

[1308] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1309] ---

[1310] This invention is a system that effectively supports users' English learning, and in particular, is a system that can optimize the learning experience by recognizing the user's emotions. It provides training menus based on the user's input data and schedule information, and furthermore, uses emotion recognition functions to flexibly adjust the learning content and schedule, thereby achieving more effective learning.

[1311] Main functions and processes of the server

[1312] 1. User data reception function

[1313] The server receives data such as TOEIC scores and English skill levels sent by users, providing the initial information for creating a personalized learning plan for each user.

[1314] 2. Skill evaluation function

[1315] The server evaluates the user's skill level based on the received English proficiency data. For example, if the TOEIC score is 650, the user's skill level is evaluated as "intermediate."

[1316] 3. Emotion recognition function

[1317] The server analyzes facial expression and voice data sent from the user device and uses an emotion recognition engine to determine the user's emotional state, for example, to obtain information such as whether the user is tired or stressed.

[1318] 4. Training menu selection function

[1319] The server selects the most appropriate training menu based on the user's skill level and emotional state. For example, if the user is tired, a menu with a lower level of difficulty can be selected.

[1320] 5. Schedule management function

[1321] The server retrieves the user's schedule information, such as Google Calendar, and finds available time slots. It then schedules training sessions for those available times.

[1322] 6. Notification function

[1323] The server sends a notification to the user device at the start time of the scheduled workout, which includes a menu that is fine-tuned according to the user's emotional state.

[1324] Main functions and processing of the user terminal

[1325] 1. Notification reception function

[1326] The user terminal receives the notification sent from the server and notifies the user of the start time and training content.

[1327] 2. App auto-start function

[1328] Upon receiving the notification, at the specified time, the learning application installed on the user's device will automatically start.

[1329] 3. Learning Progression Function

[1330] The user device analyzes the user's facial expressions and voice data in real time and also records emotional changes during learning.

[1331] 4. Progress data transmission function

[1332] When the learning session ends, the user device transmits the training progress and emotional data to the server.

[1333] User operations

[1334] 1. Data Entry

[1335] When users first set up the system, they input their TOEIC score and English skill level, and also link the system with Google Calendar.

[1336] 2. Notification confirmation

[1337] The user checks the notification from the server to find out the start time and content of the next learning session.

[1338] 3. Study with apps

[1339] At the specified time, the user begins training using the automatically launched app.

[1340] 4. Feedback confirmation

[1341] After studying, users can check the feedback and progress data displayed to understand their learning status.

[1342] Specific examples

[1343] For example, suppose a user with a TOEIC score of 650 is registered in the system. In this case, the server evaluates the skill level as "intermediate." The server then analyzes the user's facial expression and voice data and determines that the user is feeling fatigued. The server selects a relatively easy task from the "intermediate reading training" and schedules it for an available time in the user's Google calendar. The server determines that this training session will take place from 7:30 to 8:00 AM on a weekday. A notification is sent to the user's device, and the learning application automatically launches at the specified time. The user performs the "intermediate reading training," and upon completion, progress data and emotional data are sent to the server. The server analyzes the data and provides the user with feedback, such as "80% accuracy in reading, high stress level."

[1344] As such, the present invention is a system that can provide flexible training tailored to the user's skill level and emotional state, and efficiently manage progress and provide feedback in real time, allowing users to efficiently utilize their spare time to study English and maximize the effectiveness of their learning.

[1345] The processing flow will be explained below.

[1346] ---

[1347] Step 1:

[1348] During the initial setup, users enter their TOEIC score, English skill level, and Google Calendar integration information into the device, which then sends this data to the server.

[1349] Step 2:

[1350] The server receives the TOEIC score and English skill level data sent by the user. For example, if a TOEIC score of 650 is used, the English skill level is set as "intermediate."

[1351] Step 3:

[1352] The server evaluates the user's skill level based on the received data. For example, a TOEIC score of 650 would be rated as "intermediate."

[1353] Step 4:

[1354] The server requests permission to retrieve the user's Google Calendar information. If the user grants access, the server retrieves the calendar information.

[1355] Step 5:

[1356] The server detects available time from the acquired schedule information. For example, it identifies available time from 7:30 to 8:00 AM on weekdays.

[1357] Step 6:

[1358] The user terminal collects the user's facial expression and voice data in real time and transmits them to the server, which then recognizes the user's emotional state.

[1359] Step 7:

[1360] The server uses an emotion recognition engine to determine the user's emotional state, for example, determining that the user is feeling tired.

[1361] Step 8:

[1362] The server selects a training menu based on the user's skill level and emotional state, for example, selecting "intermediate reading training" and adjusting the difficulty level.

[1363] Step 9:

[1364] The server sends a notification to the user device when the scheduled training session approaches, with a message such as "The next learning session will start at 7:30."

[1365] Step 10:

[1366] The user terminal receives the notification and displays it to the user as a pop-up message, and the user confirms the start time of the next learning session.

[1367] Step 11:

[1368] At the specified time (7:30 AM in this case), the user's device will automatically launch the English learning application, and the user will begin training in the app.

[1369] Step 12:

[1370] The user progresses through the learning process according to a designated training menu (e.g., "Intermediate Reading Training"). During the training, the user's facial expressions and voice data are collected again, and emotional changes are recorded.

[1371] Step 13:

[1372] When the learning session ends, the user's device records the training progress and emotional data and transmits it to the server.

[1373] Step 14:

[1374] The server analyzes the received progress data and emotion data and provides feedback to the user, such as "80% correct answer rate in reading, high stress level."

[1375] By taking these steps, users can efficiently utilize their spare time to study English and obtain an optimal learning experience that takes into account their emotional state, thereby improving learning effectiveness and helping to establish consistent learning habits.

[1376] Example 2

[1377] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1378] Conventional English learning support systems are limited in providing training tailored to the user's skill level, and have had the problem of difficulty in providing flexible learning content that responds to the user's emotional state and real-time progress.In addition, they lack the ability to automatically set a learning schedule that utilizes the user's free time, or to provide feedback based on emotional recognition during learning, making it impossible to provide an optimal learning experience for users.

[1379] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1380] In this invention, the server includes means for receiving language ability data from a user, means for evaluating the user's skill level based on the received language ability data, means for recognizing the user's skill level and emotional state and automatically selecting a training menu accordingly, means for acquiring the user's schedule information, means for scheduling a training menu during free time in the acquired schedule information and sending a notification to the user terminal at the training start time, means for automatically starting learning software installed on the user terminal at a specified time, means for acquiring the user's progress and emotional data in real time while learning, and means for analyzing the acquired progress and emotional data, generating feedback, and providing it to the user. This makes it possible to provide flexible and optimal learning content according to the user's emotional state and real-time progress, and to automatically set an efficient learning schedule that utilizes the user's free time and provide feedback in real time.

[1381] "Language Proficiency Data" refers to TOEIC scores and other data indicating English proficiency for assessing a user's English skills.

[1382] The "skill level" is a level indicating the user's English ability as a result of evaluating the user's English ability.

[1383] "Emotional state" refers to the user's psychological and emotional state, which is determined by analyzing the user's facial expressions and voice data.

[1384] A "training menu" is a set of specific exercises and tasks for learning English that are selected based on the user's skill level and emotional state.

[1385] "Schedule information" is information relating to a user's schedule and free time, and is managed, for example, by a calendar.

[1386] "Free time" is a time period in the user's schedule information in which no other plans are scheduled.

[1387] A "notification" is a message sent from the server to the user terminal, informing the user of the start time and content of the training menu.

[1388] "Learning software" refers to an application for learning English that is installed and used by users.

[1389] "Progress" is information that indicates how much of a task a user has completed while studying English.

[1390] "Real-time" means that the actions and states of the user at that moment are reflected immediately as they progress through their learning process.

[1391] "Feedback" refers to evaluations and advice provided by the server regarding the learning a user has done.

[1392] The present invention provides a system for effectively supporting a user's English learning, and in particular, a system for optimizing the learning experience by recognizing the user's emotions. Specific embodiments for carrying out the present invention are described below.

[1393] Basic system configuration

[1394] This system consists of a server and a user device. The server receives language ability data and schedule information sent by the user, and uses this information to select training menus and manage schedules. It also uses an emotion recognition engine to determine the user's emotional state and flexibly adjust the learning content. The user device receives notifications from the server, automatically launches the learning software, and acquires learning progress data in real time and sends it to the server.

[1395] Hardware and Software

[1396] The entire system is implemented using the following hardware and software:

[1397] Server: Uses a cloud server or on-premise server to perform data analysis, emotion recognition and scheduling.

[1398] User devices: Use devices that can install learning software, such as computers, smartphones, or tablets.

[1399] Learning software: An application for learning English that includes the ability to record the user's learning progress and emotional state.

[1400] Emotion recognition engine: For example, emotion recognition is performed using Microsoft Azure's Emotion API.

[1401] Data processing and calculation

[1402] The server performs the following steps to process and calculate the data:

[1403] 1. Data reception and processing: The server receives language ability data (e.g., TOEIC score) and schedule information sent by the user and analyzes them as initial data.

[1404] 2. Skill level assessment: Based on the received language proficiency data, a calculation process is performed to assess the user's skill level.

[1405] 3. Emotion Recognition: An emotion recognition engine is used to analyze the user's facial expressions and voice data to determine their emotional state.

[1406] 4. Training Menu Selection: Select an appropriate training menu based on assessed skill level and emotional state.

[1407] 5. Schedule Management: Detects free time from user schedule information and automatically schedules training sessions.

[1408] 6. Notifications and Feedback: Send notifications at the scheduled start time of training, and provide feedback after learning by analyzing progress and sentiment data obtained.

[1409] Specific examples

[1410] For example, suppose a user with a TOEIC score of 650 registers with the system. Based on the received score, the server evaluates the user's skill level as "intermediate." The server then analyzes the user's facial expression and voice data and determines that the user is tired.

[1411] The server selects a lower level of difficulty from the "Intermediate Reading Training" and schedules the training session during the user's free time (e.g., weekdays from 7:30 to 8:00 a.m.). At the designated time, the learning software installed on the user's device automatically starts up and the training begins.

[1412] After the user completes the training, the learning software sends progress and emotional data to the server, which analyzes the progress and emotional data and provides feedback to the user, such as "80% correct reading, high stress level."

[1413] Prompt Sentence Examples

[1414] "If a user with a TOEIC score of 650 is determined to be fatigued, generate a prompt from the AI ​​system that selects relatively easy intermediate reading training tasks and provides a way to schedule the training during the user's free time."

[1415] As such, the present invention is a system that provides flexible and optimal learning according to the user's skill level and emotional state, efficiently manages learning schedules by utilizing free time, and provides real-time feedback.

[1416] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1417] Step 1:

[1418] Entering and submitting user data

[1419] During initial setup, the user enters their TOEIC score and English skill level and sends them to the server. The system also links with Google Calendar. The input for this step is the user's language ability data and schedule information, such as "TOEIC score 650, skill level intermediate." The output is the user data sent to the server.

[1420] Step 2:

[1421] Data reception by the server and skill evaluation

[1422] The server receives the language ability data sent by the user and evaluates the user's skill level based on this. The input to this step is the language ability data sent by the user. The data is saved in an internal database and an evaluation process is performed. For example, if the user's TOEIC score is 650, the user's skill level is evaluated as "intermediate." The output is the data that evaluates the user's skill level.

[1423] Step 3:

[1424] Emotion recognition by server

[1425] The server analyzes the facial expression and voice data sent from the user device to determine the user's emotional state. The input for this step is the facial expression and voice data sent from the user device. An emotion recognition engine (e.g., Emotion API) is used to analyze the data and determine whether the user is tired. The output is data indicating the user's emotional state.

[1426] Step 4:

[1427] Selection of training menu

[1428] The server selects the optimal training menu based on the user's skill level and emotional state. The input for this step is the user's skill level data and emotional state data. A data analysis algorithm is used to identify a training menu suitable for the user. For example, if the user is tired, a less difficult "intermediate reading training" is selected. The output is the selected training menu.

[1429] Step 5:

[1430] Schedule registration by server

[1431] The server retrieves schedule information from the user's Google Calendar and detects available time slots. The input to this step is the user's schedule information. The server analyzes the available time slots and schedules training sessions for those times. For example, the server schedules sessions from 7:30 to 8:00 AM on weekdays. The output is the training sessions registered in the schedule.

[1432] Step 6:

[1433] Sending notifications

[1434] The server sends a notification to the user terminal at the scheduled training start time. The input for this step is the schedule information and the selected training menu. The notification system is used to send the specified message to the user terminal. The output is a notification displayed on the user terminal.

[1435] Step 7:

[1436] Receiving notifications on the user device and automatically launching the app

[1437] The user device receives the notification sent from the server and automatically launches the learning software at the specified time. The input to this step is the notification from the server. The software is launched at the specified time using the application auto-launch function. For example, at 7:30 AM, the app to start "Intermediate Reading Training" is launched. The output is the launched learning software.

[1438] Step 8:

[1439] User-led learning sessions

[1440] The user trains using the automatically launched learning software. The inputs of this step are the launched learning software and the training menu. The user progresses through the specified training. The outputs are the training progress and emotion data.

[1441] Step 9:

[1442] Progress data transmission by user terminal

[1443] After the training session is completed, the user device sends the progress and emotion data to the server. The input of this step is the progress and emotion data collected during training. The data transmission function is used to transmit this data to the server. The output is the progress and emotion data transmitted to the server.

[1444] Step 10:

[1445] Data analysis by server and feedback transmission

[1446] The server analyzes the received data and sends feedback to the user. The input of this step is the user's progress data and emotional data. Using a data analysis algorithm, the server identifies the user's learning achievements and areas for improvement and generates feedback. For example, it generates feedback such as "80% correct reading, high stress level." The output is the generated feedback.

[1447] (Application example 2)

[1448] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1449] Conventional English learning support systems focus on providing training menus based on the user's skill level, but do not adequately address optimization based on the user's emotional state or learning environment. As a result, even if the user feels tired or stressed, a fixed learning menu is provided, which can reduce learning efficiency. Furthermore, systems lack functionality to effectively support learning activities in physical stores, and it is necessary to provide an optimal learning experience tailored to the store environment.

[1450] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1451] In this invention, the server includes means for receiving English proficiency data from a user, means for evaluating the user's skill level based on the received English proficiency data, means for automatically selecting a training menu according to the user's skill level, means for acquiring the user's schedule information, means for scheduling a training menu during free time in the acquired schedule information, means for transmitting the scheduled training menu to a user device at a start time, means for automatically starting a learning program installed on the user device at a specified time, means for optimizing the learning content based on the store environment, and means for recognizing the user's emotional state and adjusting the learning content. This enables flexible training provision according to the user's emotional state and learning environment, and effective learning support within a physical store.

[1452] The "means for receiving English ability data from users" is a system that allows the server to receive ability data such as TOEIC scores and English skill levels provided by users.

[1453] The "means for assessing the user's skill level based on the received English proficiency data" is a system for analyzing the received data and assessing the user's current English proficiency based on specific criteria.

[1454] The "means for automatically selecting a training menu according to the user's skill level" is a system for automatically selecting the most suitable English training program based on the evaluated skill level information.

[1455] "Means for obtaining user schedule information" refers to a system that allows the server to obtain schedule information such as the user's Google Calendar.

[1456] The "means for scheduling a training menu during free time in the acquired schedule information" is a system for identifying free time in the acquired schedule and allocating an English training session to that time.

[1457] The "means for transmitting to the user device at the start time of the scheduled training menu" is a system for sending a notification to the user's device, such as a smartphone or tablet, at the set training start time.

[1458] "Means for automatically starting a learning program installed on a user device at a specified time" refers to a system for automatically starting learning software installed on a user's terminal at a specified time.

[1459] The "means for optimizing learning content based on the store environment" is a system that collects environmental data such as noise levels and congestion levels in physical stores and optimizes English training content accordingly.

[1460] The "means for recognizing the user's emotional state and adjusting the learning content" is a system that uses a camera and microphone to analyze the user's facial expressions and tone of voice, and adjusts the English training content according to the recognized emotions.

[1461] The present invention is a system for effectively supporting users' English learning in a brick-and-mortar store, and in particular, a system that can recognize users' emotions and optimize the learning experience. The system includes the following main elements:

[1462] Main functions and processes of the server

[1463] 1. User data reception function

[1464] The server receives data such as TOEIC scores and English skill levels sent by users, providing the initial information for creating a personalized learning plan for each user.

[1465] 2. Skill evaluation function

[1466] The server evaluates the user's skill level based on the received English proficiency data. For example, if the TOEIC score is 650, the user's skill level is evaluated as "intermediate."

[1467] 3. Emotion recognition function

[1468] The server analyzes facial expression and voice data sent from the user's device and uses an emotion recognition engine to determine the user's emotional state. For example, it acquires information such as whether the user is tired or stressed. The hardware used includes a camera and microphone, and the software used includes a facial expression recognition API and a voice recognition API.

[1469] 4. Training menu selection function

[1470] The server selects the most appropriate training menu based on the user's skill level and emotional state. For example, if the user is tired, a menu with a lower level of difficulty can be selected.

[1471] 5. Schedule management function

[1472] The server retrieves the user's calendar information, finds available time slots, and schedules training sessions within those times. The software used includes the Google Calendar API.

[1473] 6. Notification function

[1474] The server sends a notification to the user device at the start time of the scheduled workout, which includes a menu that is fine-tuned according to the user's emotional state.

[1475] Main functions and processing of the user terminal

[1476] 1. In-store detection function

[1477] The user device uses Wi-Fi or Bluetooth beacons to detect when the user is in a physical store.

[1478] 2. Environment recognition function

[1479] The user device recognizes the noise level and occupancy status inside the store and sends the information to the server. The hardware used includes a microphone, and the software used includes a noise-canceling algorithm.

[1480] 3. Notification reception function

[1481] The user terminal receives the notification sent from the server and notifies the user of the start time and training content.

[1482] 4. App auto-start function

[1483] Upon receiving the notification, the learning program installed on the user's terminal will automatically start at the specified time.

[1484] 5. Learning Progression Function

[1485] The user device analyzes the user's facial expressions and voice data in real time and also records emotional changes during learning.

[1486] 6. Progress data transmission function

[1487] When the learning session ends, the user device transmits the training progress and emotional data to the server.

[1488] Specific examples

[1489] For example, suppose a user with a TOEIC score of 650 is registered in the system. In this case, the server evaluates the skill level as "intermediate." Then, it analyzes the user's facial expression and voice data and determines that the user is feeling fatigued. The server selects a relatively easy task from "intermediate reading training" and schedules it for an available time in the user's calendar. It determines that this training session will take place, for example, from 7:30 to 8:00 AM on a weekday. A notification is sent to the user's device, and the learning program automatically starts at the specified time. The user performs the "intermediate reading training," and after it is completed, progress data and emotional data are sent to the server.

[1490] Prompt Sentence Examples

[1491] "I'd like to do some listening practice during the cafe time today. If the cafe is quiet, I'd like more difficult questions, and if it's noisy, I'd like easier questions."

[1492] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1493] Step 1: Receiving user data

[1494] The server receives data such as TOEIC scores and English skill levels sent by users. When the user enters this data in the application and presses the submit button, the server receives the data and stores it in a database.

[1495] Input: TOEIC score, English skill level data

[1496] Output: Save user data to database

[1497] Step 2: Skills Assessment

[1498] The server evaluates the user's skill level based on the received English proficiency data. For example, if the TOEIC score is 650, the server will evaluate it as "intermediate." An evaluation algorithm is applied to this data to determine the user's skill level.

[1499] Input: English proficiency data

[1500] Output: Assessed skill level (e.g., intermediate)

[1501] Step 3: In-store detection

[1502] The user device uses Wi-Fi or Bluetooth beacons to detect when the user is in a physical store, which then transmits specific store environment information to the server.

[1503] Input: Signals from Wi-Fi or Bluetooth beacons

[1504] Output: Confirmation that the user is in the store

[1505] Step 4: Environmental awareness

[1506] The user device recognizes the noise level and congestion status inside the store and sends the information to the server. Specifically, it uses a microphone to collect surrounding sounds and analyzes the noise level.

[1507] Input: Audio data from the microphone

[1508] Output: Noise level, congestion data

[1509] Step 5: Emotion Recognition

[1510] The server analyzes the facial expression and voice data sent from the user's device and uses an emotion recognition engine to determine the user's emotional state, for example, extracting information such as whether the user is tired or stressed.

[1511] Input: facial expression data, voice data

[1512] Output: Emotional state assessment (e.g., tired)

[1513] Step 6: Choose your training menu

[1514] The server selects the most appropriate training menu based on the user's skill level and emotional state: for example, if the user is tired, it selects a reading training with a lower level of difficulty.

[1515] Input: Skill level, emotional state

[1516] Output: Selected training menu

[1517] Step 7: Retrieve schedule information

[1518] The server retrieves the user's calendar information, using the Google Calendar API to read the user's schedule and identify available times.

[1519] Input: User's Google Calendar information

[1520] Output: Free / busy information

[1521] Step 8: Training Schedule

[1522] The server schedules the training menu for the available time slots in the acquired schedule information. For example, the server may set a training session for the available time slots between 7:30 and 8:00 AM on weekdays.

[1523] Input: Free / busy information, training menu

[1524] Output: Scheduled training sessions

[1525] Step 9: Send notification

[1526] The server sends a notification to the user terminal at the start time of the scheduled training menu, which also includes the training content.

[1527] Input: Schedule a training session

[1528] Output: Notification to user terminal

[1529] Step 10: App Auto-Launch

[1530] The user terminal automatically starts the installed learning program at the specified time. Upon receiving the notification, the user terminal starts the program and begins training.

[1531] Input: Notification content

[1532] Output: Automatic launch of learning program

[1533] Step 11: Record your learning progress and emotional changes

[1534] The user device analyzes the user's facial expressions and voice data in real time and records emotional changes during learning. Specifically, it collects and analyzes data using a camera and microphone.

[1535] Input: Real-time facial expression data, voice data

[1536] Output: Recorded data of emotional changes

[1537] Step 12: Send progress data

[1538] Once the training session is over, the user device transmits training progress and emotional data to the server, which stores this data in a database for further analysis.

[1539] Input: Progress data, emotion data

[1540] Output: Send data to the server

[1541] Examples of prompts:

[1542] "I'd like to do some listening practice during the cafe time today. If the cafe is quiet, I'd like more difficult questions, and if it's noisy, I'd like easier questions."

[1543] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1544] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1545] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1546] [Fourth embodiment]

[1547] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1548] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1549] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1550] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1551] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1552] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1553] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1554] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1555] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1556] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1557] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1558] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1559] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1560] ---

[1561] The present invention is a system for effectively supporting users' English learning, and aims to automatically provide optimal training menus based on English proficiency data entered by the user, and to efficiently manage learning time according to the user's schedule. This system is mainly composed of a server, a user terminal, and processing steps including the exchange of input data from the user.

[1562] Main functions and processes of the server

[1563] 1. User data reception function

[1564] When users initially set up the system, they input their TOEIC score and English skill level, and the server receives and records this data.

[1565] 2. Skill evaluation function

[1566] The server automatically evaluates the user's skill level based on the received English proficiency data. For example, if the TOEIC score is 650, the server will evaluate the user's skill level as "intermediate."

[1567] 3. Training menu selection function

[1568] The server automatically selects a training menu appropriate for the assessed skill level, for example, "intermediate reading training" or "intermediate listening training."

[1569] 4. Schedule management function

[1570] The server retrieves the user's schedule information, such as Google Calendar, and finds available time slots. It then schedules training sessions that fit the available time slots.

[1571] 5. Notification function

[1572] When the scheduled training session is about to begin, the server sends a notification to the user's device, with a message like "Your next training session will begin at 7:30."

[1573] Main functions and processing of the user terminal

[1574] 1. Notification reception function

[1575] The user device receives a notification sent from the server, which indicates the content and start time of the next learning session.

[1576] 2. App auto-start function

[1577] The device that receives the notification will automatically launch the English learning app at the specified time. For example, "Intermediate Reading Training" will start at 7:30 AM.

[1578] 3. Learning Progression Function

[1579] The user can then use the app to progress through the training, which will then record the progress of the training and the user's responses in real time.

[1580] 4. Progress data transmission function

[1581] At the end of a study session, the app sends the user's progress data to the server, such as "80% correct reading" or "75% correct listening."

[1582] User operations

[1583] 1. Data Entry

[1584] During initial setup, users enter their TOEIC score, English skill level, and Google Calendar connection information, which allows the server to suggest appropriate training menus and schedules.

[1585] 2. Notification confirmation

[1586] The user checks the notification sent from the server to find out when the next learning session will start.

[1587] 3. Study with apps

[1588] At the designated time, the user will use the automatically launched app to carry out the training, which will help develop a natural learning habit.

[1589] 4. Feedback confirmation

[1590] Once the training is complete, users can view their progress and feedback from the app.

[1591] Specific examples

[1592] For example, if a user with a TOEIC score of 650 provides input data to the system, the server will evaluate the user as "intermediate." The server then selects "intermediate reading training" and "intermediate listening training." It then checks Google Calendar to find available time between 7:30 and 8:00 a.m. on weekdays. The server schedules training for this time period and sends a notification to the user's device. At 7:30 a.m., the learning app installed on the user's device will automatically launch and the training will begin. When the user completes the training, progress data is sent to the server, which then provides feedback based on that data.

[1593] As described above, the present invention is a system that can provide flexible training according to the user's skill level and schedule, and can efficiently manage progress.

[1594] The processing flow will be explained below.

[1595] ---

[1596] Step 1:

[1597] The server receives the TOEIC score and English skill level data sent by the user. Specifically, the score and skill information entered by the user on the initial setting screen is sent to the server.

[1598] Step 2:

[1599] The server automatically evaluates the user's skill level based on the received English proficiency data. For example, if the TOEIC score is 650, the server evaluates the user's skill level as "intermediate."

[1600] Step 3:

[1601] The server automatically selects a training menu according to the user's skill level, such as "Intermediate Reading Training" or "Intermediate Listening Training."

[1602] Step 4:

[1603] The server requests permission to access the schedule information from the user to obtain the schedule information of the user's Google Calendar, etc. After the user grants permission, the server obtains the user's schedule data.

[1604] Step 5:

[1605] The server detects available times from the acquired schedule information. For example, it determines that available times are from 7:30 to 8:00 a.m. on weekdays.

[1606] Step 6:

[1607] The server schedules training sessions during the available times it finds, for example, allocating "Intermediate Reading Training" between 7:30 and 8:00 AM.

[1608] Step 7:

[1609] The server sends a notification to the user device 15 minutes before the scheduled training, with a message such as "The next learning session will start at 7:30."

[1610] Step 8:

[1611] The user terminal receives the notification from the server and displays a pop-up message to the user, who then confirms the start time.

[1612] Step 9:

[1613] At the specified time (7:30 AM in this case), the user's device automatically launches the English learning application, and the user begins training in the app.

[1614] Step 10:

[1615] The user progresses through the learning process by following the designated training menu, for example, answering questions in the "Intermediate Reading Training" section.

[1616] Step 11:

[1617] Once the learning session is over, the user's device records the training progress and results and sends them to the server.

[1618] Step 12:

[1619] The server analyzes the received progress data and provides feedback to the user, such as "Your accuracy rate for this reading training was 80%."

[1620] By following these steps, users can study English efficiently in their spare time and naturally develop a study habit. Real-time progress management and feedback also improve the effectiveness of users' learning.

[1621] Example 1

[1622] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1623] Conventional English learning systems have difficulty flexibly adapting to individual users' skill levels and schedules, preventing them from maximizing learning effectiveness. They also lacked a mechanism for recording users' progress in real time or the ability to provide feedback based on that progress. This prevented users from accurately understanding their own learning progress and results, hindering effective learning.

[1624] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1625] In this invention, the server includes means for receiving English proficiency data from a user, means for evaluating the user's skill level based on the received English proficiency data, means for automatically selecting a training menu according to the user's skill level, means for acquiring the user's schedule information, means for scheduling a training menu during free time in the acquired schedule information, means for sending a notification to the user terminal at the start time of the scheduled training menu, means for automatically starting a learning application installed on the user terminal at a specified time, means for the learning application to record the user's training progress in real time, means for sending user progress data to the server after training ends, and means for providing feedback based on the progress data to the user terminal. This enables efficient learning tailored to the user's schedule and skill level, and by accurately understanding the user's learning progress and receiving feedback, effective learning can be achieved.

[1626] "User" refers to an individual or organization that uses the English Learning System.

[1627] "English Proficiency Data" refers to data necessary for assessing a user's English proficiency, such as a user's TOEIC score or English skill level.

[1628] "Skill level" refers to a user's proficiency in understanding and practicing English as assessed based on English proficiency data.

[1629] "Training Menu" refers to a set of English learning sessions and assignments provided according to the user's skill level.

[1630] "Schedule information" refers to data such as calendar information that indicates the user's schedule and free time.

[1631] "Real-time" refers to the fact that progress and achievements are recorded instantly while a user is using a learning application.

[1632] "Progress data" refers to data that indicates the user's learning status, such as the results achieved by the user during a study session and the percentage of correct answers.

[1633] "Feedback" refers to information provided to the user based on progress data, such as indications of learning results and next goals.

[1634] "Server" refers to the central processing unit that receives and analyzes user data, selects training menus, and manages schedules.

[1635] "User terminal" refers to a computing device on which a user installs and uses a learning application.

[1636] "Learning Application" refers to software that a user uses to train their English language skills.

[1637] "Notification" refers to a message or alert sent from a server to a user device.

[1638] This invention is a system for effectively supporting users' English learning. This system provides an optimal training menu based on the user's English proficiency data and efficiently manages learning time according to the user's schedule. It is mainly composed of a server, a user terminal, and processing steps including the exchange of input data from the user.

[1639] Specific server operations

[1640] User data reception function

[1641] When users initially set up the system, they input their TOEIC score and English skill level, and the server receives and records this data.

[1642] Skill evaluation function

[1643] The server automatically evaluates the user's skill level based on the received English proficiency data. For example, if the TOEIC score is 650, the server will evaluate the user's skill level as "intermediate."

[1644] Training menu selection function

[1645] The server automatically selects a training menu according to the assessed skill level, for example, "intermediate reading training" or "intermediate listening training."

[1646] Schedule management function

[1647] The server retrieves the user's schedule information (e.g., Google Calendar), finds available time slots, and schedules training sessions that fit the available time slots.

[1648] Notification function

[1649] When the scheduled training session is about to begin, the server sends a notification to the user's device, for example, a message saying, "Your next training session will begin at 7:30."

[1650] Specific operation of the user terminal

[1651] Notification reception function

[1652] The user device receives a notification sent from the server, which indicates the content and start time of the next learning session.

[1653] App auto-start function

[1654] The device that receives the notification will automatically launch the English learning app at the specified time. For example, "Intermediate Reading Training" will start at 7:30 AM.

[1655] Learning Progression Function

[1656] The user can then use the app to progress through the training, which will then record the progress of the training and the user's responses in real time.

[1657] Progress data transmission function

[1658] At the end of a study session, the app sends the user's progress data to the server, such as "80% correct reading" or "75% correct listening."

[1659] Specific user operations

[1660] Data Entry

[1661] During initial setup, users enter their TOEIC score, English skill level, and Google Calendar connection information, which allows the server to suggest appropriate training menus and schedules.

[1662] Notification confirmation

[1663] The user checks the notification sent from the server to find out when the next learning session will start.

[1664] Learn with the app

[1665] At the designated time, the app will automatically start and the training will begin, making it easier to naturally develop a study habit.

[1666] Feedback Check

[1667] Once the training is complete, users can view their progress and feedback from the app.

[1668] Specific examples

[1669] For example, a user with a TOEIC score of 650 provides input data to the system. The server evaluates the user as "intermediate." The server then selects "intermediate reading training" and "intermediate listening training." After that, it checks Google Calendar to find available time slots between 7:30 and 8:00 a.m. on weekdays. The server schedules the training for this time period and sends a notification to the user's device. At 7:30 a.m., the learning app installed on the user's device automatically launches and the training begins. When the user completes the training, progress data is sent to the server, which then provides feedback based on that data.

[1670] As described above, the present invention is a system that can provide flexible training according to the user's skill level and schedule, and can efficiently manage progress.

[1671] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1672] Step 1:

[1673] Initial data entry by the user

[1674] The user opens the system's initial setup screen and enters their TOEIC score, English skill level, and Google Calendar connection information, which then sends the initial data to the server.

[1675] Input: TOEIC score, English skill level, Google Calendar link information

[1676] Output: Initial data sent to the server

[1677] Step 2:

[1678] Data reception and analysis by the server

[1679] The server receives the initial data sent by the user, checks the integrity of the received data, and saves each item in the database.

[1680] Input: Initial data

[1681] Output: User data stored in the database

[1682] Specific operation: The server checks whether the TOEIC score and English skill level are within a reasonable range.

[1683] Step 3:

[1684] Server-based skill evaluation

[1685] The server analyzes the received data and evaluates the user's skill level. For example, if a TOEIC score is 650, it will be evaluated as "intermediate."

[1686] Input: User data

[1687] Output: Assessed skill level

[1688] Specific operation: The server analyzes the score using the evaluation algorithm.

[1689] Step 4:

[1690] Training menu selection by server

[1691] The server will select the most appropriate training menu based on your assessed skill level, such as "Intermediate Reading Training."

[1692] Input: Skill Level

[1693] Output: Training menu

[1694] Specific operation: The server retrieves the appropriate training menu from the database.

[1695] Step 5:

[1696] Server-based schedule management

[1697] The server retrieves the user's Google Calendar schedule information, finds available time slots, and schedules training sessions for those available times.

[1698] Input: Google Calendar schedule information

[1699] Output: Scheduled training sessions

[1700] Specific operation: The server analyzes the schedule information and identifies free time.

[1701] Step 6:

[1702] Server-based schedule notification

[1703] When the server approaches the start time of a scheduled training session, it sends a notification to the user's device, for example, "The next learning session will start at 7:30."

[1704] Input: Scheduled training sessions

[1705] Output: Notification sent to the user's device

[1706] Specific operation: The server sends an email or push notification

[1707] Step 7:

[1708] Receiving notifications and launching apps on your device

[1709] The user device receives the notification from the server and automatically launches the learning app at the specified time.

[1710] Input: Notification from the server

[1711] Output: Learning app launched

[1712] Specific operation: The device launches the learning app at the specified time.

[1713] Step 8:

[1714] User-driven training

[1715] The user starts the learning app and progresses through the training. The app records the user's progress in real time.

[1716] Input: User interaction

[1717] Output: Real-time recorded progress

[1718] What it does: The user performs the workout and the app records their progress in real time.

[1719] Step 9:

[1720] Progress data transmission by device

[1721] At the end of a study session, the app sends the user's progress data to a server, such as "80% correct reading."

[1722] Input: Progress data

[1723] Output: Progress data sent to the server

[1724] Specific behavior: The app detects the end of the session and sends progress data to the server.

[1725] Step 10:

[1726] Server-provided feedback

[1727] The server analyzes the submitted progress data and provides feedback to the user, such as "You got 80% of your reading correct, so try practicing faster next time."

[1728] Input: Progress data

[1729] Output: Feedback provided to the user's device

[1730] Specific behavior: The server analyzes the progress data, generates feedback, and sends it to the user.

[1731] This allows for efficient learning tailored to the user's schedule and skill level, and by accurately understanding the user's learning progress and receiving feedback, effective learning can be achieved.

[1732] (Application example 1)

[1733] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1734] Conventional English learning support systems often lack the ability to provide optimal training menus tailored to the user's skill level or manage schedules, hindering efficient learning. Furthermore, progress data recording and feedback provision are performed manually, reducing the continuity and effectiveness of learning. Furthermore, it is difficult to efficiently manage learning time using the user's schedule information. There is a need to provide an effective and efficient English learning support system that can solve these issues.

[1735] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1736] In this invention, the server includes means for receiving English proficiency data from a user, means for evaluating the user's skill level based on the received English proficiency data, means for automatically selecting a training menu according to the user's skill level, means for scheduling a training menu during free time in the acquired schedule information, means for sending a notification to the user terminal at the start time of the scheduled training menu, means for automatically launching a learning application installed on the user terminal at a specified time, means for monitoring the ongoing training menu in real time and recording progress data, means for sending the recorded progress data to the server, means for providing feedback to the user based on the sent progress data, means for automatically generating an optimal training menu for the user using a generative model based on the English proficiency data and schedule information, means for generating prompt sentences to be given to the generative AI model, and means for automatically supporting the user's learning behavior and providing appropriate feedback in real time, thereby promoting the user's learning habits, improving learning effectiveness, and making efficient use of study time.

[1737] "User" refers to an individual person who uses the system and provides English proficiency data and schedule information.

[1738] "English proficiency data" is a collection of information indicating a user's English proficiency, such as a user's TOEIC score or English skill level.

[1739] "Skill level" refers to the level of a user's English ability that is evaluated based on specific standards, and is classified as beginner, intermediate, advanced, etc.

[1740] A "training menu" is a set of English learning tasks and exercises selected according to the user's skill level.

[1741] "Schedule information" is information that indicates the user's daily plans, and mainly includes calendar information.

[1742] A "notification" is a message sent to a user terminal at a specified time, informing the user of the start time of a training menu, etc.

[1743] "User terminal" refers to a device that a user operates directly to study English, including smartphones and tablets.

[1744] A "learning application" is software that is installed on a user's device and provides learning materials and training menus for English language learning.

[1745] "Progress data" refers to information about the learning progress and results recorded as the user performs the training menu.

[1746] "Server" is a computer that receives English proficiency data and progress data from users and provides feedback based on this data.

[1747] "Feedback" refers to information provided based on the user's progress data, including an evaluation of the learning results and advice for the next learning session.

[1748] A "generative model" is a machine learning model that automatically generates optimal training menus based on English proficiency data and schedule information.

[1749] A "prompt sentence" is a sentence input to a generative model, and is a specific instruction sentence created based on the user's learning content and purpose.

[1750] This invention is a system for effectively supporting users' English learning, and in particular, it supports the establishment of study habits and efficient learning through individualized study training according to the user's ability and effective schedule management. The invention utilizes the following hardware and software:

[1751] Hardware and software used

[1752] Server: A web server (e.g., Apache) and a database (e.g., MongoDB) are used to receive data, assess skill levels, select training menus, analyze progress data, and run generative models.

[1753] User device: A smartphone or tablet (e.g., iOS or Android) is assumed to be the device that users will operate directly to study English.

[1754] Google Calendar API: Used to retrieve user schedule information.

[1755] Outgoing Mail Server: Uses SMTP to send training session notifications to users.

[1756] Natural language explanation of the process

[1757] Main functions and processes of the server

[1758] 1. Receiving user data

[1759] The server receives the English proficiency data (e.g., TOEIC score, English skill level) provided by the user and records it in a database.

[1760] 2. Skill Level Assessment

[1761] The server automatically evaluates the user's skill level based on the received English proficiency data. For example, if the user's TOEIC score is 650, the server evaluates the user's skill level as "intermediate."

[1762] 3. Selecting a training menu

[1763] The server automatically selects a training menu appropriate for the assessed skill level, for example, "intermediate reading training" or "intermediate listening training."

[1764] 4. Schedule Management

[1765] The server uses the Google Calendar API to retrieve the user's schedule, find available time slots, and schedule suitable training sessions.

[1766] 5. Sending notifications

[1767] When the scheduled start time of the training approaches, the server sends a notification to the user terminal using the mail sending server.

[1768] 6. Automatic generation of training menus

[1769] The server uses the generative model to automatically generate the optimal training menu for the user based on English proficiency data and schedule information.

[1770] 7. Prompt generation for generative AI models

[1771] The server generates prompt sentences to be given to the generative AI model based on the user's learning content and goals.

[1772] Main functions and processing of the user terminal

[1773] 1. Receiving notifications

[1774] The user terminal receives the notification sent from the server and presents the content and start time of the next learning session.

[1775] 2. App auto-start

[1776] At the specified time, the learning application installed on the user's device will automatically start and the training will begin.

[1777] 3. Recording and sending progress data

[1778] It records your training progress in real time and sends the progress data to a server once your learning session is over.

[1779] Specific examples

[1780] For example, when a user with a TOEIC score of 650 initially configures the system, the server evaluates the user as "intermediate" and selects "intermediate reading training" and "intermediate listening training." The server then checks the user's Google Calendar to find available time between 7:30 and 8:00 a.m. on weekdays. Training is scheduled for this time period and a notification is sent to the user's device. At the designated time of 7:30 a.m., the learning application installed on the user's device automatically launches and the training begins. When the training session ends, progress data is sent to the server, and feedback is provided. The server also uses a generative model to automatically generate the next training menu, maximizing the user's learning effectiveness.

[1781] Prompt Sentence Examples

[1782] My TOEIC score is 650. Please suggest a training menu suitable for intermediate level and schedule study sessions during your free time. Please also send me a notification before the next session starts. The Google Calendar information is as follows: [Calendar Information].

[1783] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1784] Step 1:

[1785] Receiving user data

[1786] The server receives English proficiency data (e.g., TOEIC score or English skill level) entered by the user. User data is entered via a web form or similar and sent to the server. Input: TOEIC score, English skill level. Output: Recording of received data in a database. Specifically, the server receives an HTTP request and saves the data in a database (e.g., MongoDB).

[1787] Step 2:

[1788] Skill Level Assessment

[1789] The server evaluates the user's skill level based on the received English proficiency data. It analyzes the received data (e.g., TOEIC score) and determines the skill level. Input: TOEIC score. Output: Skill level (e.g., "Intermediate"). Specifically, the server applies predefined evaluation logic based on the data it receives to determine the skill level.

[1790] Step 3:

[1791] Selection of training menu

[1792] The server automatically selects a training menu according to skill level. It retrieves an appropriate training menu from the database and proposes it to the user. Input: Skill level. Output: Training menu. Specifically, the server selects candidates from a list of training menus pre-registered in the database based on the corresponding skill level.

[1793] Step 4:

[1794] Get schedule information

[1795] The server uses the Google Calendar API to obtain the user's schedule information. Input: User's calendar information. Output: Schedule information (such as a list of free time slots). Specifically, the server sends authentication information to the Google Calendar API and obtains events from the user's calendar.

[1796] Step 5:

[1797] Schedule a training session

[1798] The server detects available time slots based on the acquired schedule information and schedules training sessions. Input: Schedule information, training menu. Output: Scheduled training session. Specifically, the server analyzes the schedule information and associates the optimal time slot with the training menu.

[1799] Step 6:

[1800] Sending notifications

[1801] The server sends a notification to the user device when the scheduled training session is about to start. Input: Scheduled training session. Output: Notification message to the user device. Specifically, the server uses an email server or push notification service to send the specified message to the user.

[1802] Step 7:

[1803] App autostart

[1804] The user device will automatically launch the installed learning application at the specified time. Input: Notification message. Output: Launching of learning application. Specifically, the device will launch the learning application that is set as a trigger in response to the received notification message.

[1805] Step 8:

[1806] Recording and sending progress data

[1807] The user's device records the training progress in real time, and when the learning session ends, it sends the progress data to the server. Input: Operation data during training. Output: Sending progress data to the server. Specifically, the device records the user's answers and operations, and sends the data to the server via an HTTP request when the learning session ends.

[1808] Step 9:

[1809] Providing feedback

[1810] The server provides feedback to the user based on the submitted progress data. Input: Progress data. Output: Feedback message. Specifically, the server analyzes the progress data and generates and sends a message to the user based on predefined feedback logic.

[1811] Step 10:

[1812] Prompt generation for generative AI models

[1813] The server generates a prompt sentence to be given to the generative AI model based on English proficiency data and schedule information. Input: English proficiency data, schedule information. Output: Prompt sentence. Specifically, the server creates a prompt sentence appropriate for the generative AI model based on the specified data and provides it to the model.

[1814] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1815] ---

[1816] This invention is a system that effectively supports users' English learning, and in particular, is a system that can optimize the learning experience by recognizing the user's emotions. It provides training menus based on the user's input data and schedule information, and furthermore, uses emotion recognition functions to flexibly adjust the learning content and schedule, thereby achieving more effective learning.

[1817] Main functions and processes of the server

[1818] 1. User data reception function

[1819] The server receives data such as TOEIC scores and English skill levels sent by users, providing the initial information for creating a personalized learning plan for each user.

[1820] 2. Skill evaluation function

[1821] The server evaluates the user's skill level based on the received English proficiency data. For example, if the TOEIC score is 650, the user's skill level is evaluated as "intermediate."

[1822] 3. Emotion recognition function

[1823] The server analyzes facial expression and voice data sent from the user device and uses an emotion recognition engine to determine the user's emotional state, for example, to obtain information such as whether the user is tired or stressed.

[1824] 4. Training menu selection function

[1825] The server selects the most appropriate training menu based on the user's skill level and emotional state. For example, if the user is tired, a menu with a lower level of difficulty can be selected.

[1826] 5. Schedule management function

[1827] The server retrieves the user's schedule information, such as Google Calendar, and finds available time slots. It then schedules training sessions for those available times.

[1828] 6. Notification function

[1829] The server sends a notification to the user device at the start time of the scheduled workout, which includes a menu that is fine-tuned according to the user's emotional state.

[1830] Main functions and processing of the user terminal

[1831] 1. Notification reception function

[1832] The user terminal receives the notification sent from the server and notifies the user of the start time and training content.

[1833] 2. App auto-start function

[1834] Upon receiving the notification, at the specified time, the learning application installed on the user's device will automatically start.

[1835] 3. Learning Progression Function

[1836] The user device analyzes the user's facial expressions and voice data in real time and also records emotional changes during learning.

[1837] 4. Progress data transmission function

[1838] When the learning session ends, the user device transmits the training progress and emotional data to the server.

[1839] User operations

[1840] 1. Data Entry

[1841] When users first set up the system, they input their TOEIC score and English skill level, and also link the system with Google Calendar.

[1842] 2. Notification confirmation

[1843] The user checks the notification from the server to find out the start time and content of the next learning session.

[1844] 3. Study with apps

[1845] At the specified time, the user begins training using the automatically launched app.

[1846] 4. Feedback confirmation

[1847] After studying, users can check the feedback and progress data displayed to understand their learning status.

[1848] Specific examples

[1849] For example, suppose a user with a TOEIC score of 650 is registered in the system. In this case, the server evaluates the skill level as "intermediate." The server then analyzes the user's facial expression and voice data and determines that the user is feeling fatigued. The server selects a relatively easy task from the "intermediate reading training" and schedules it for an available time in the user's Google calendar. The server determines that this training session will take place from 7:30 to 8:00 AM on a weekday. A notification is sent to the user's device, and the learning application automatically launches at the specified time. The user performs the "intermediate reading training," and upon completion, progress data and emotional data are sent to the server. The server analyzes the data and provides the user with feedback, such as "80% accuracy in reading, high stress level."

[1850] As such, the present invention is a system that can provide flexible training tailored to the user's skill level and emotional state, and efficiently manage progress and provide feedback in real time, allowing users to efficiently utilize their spare time to study English and maximize the effectiveness of their learning.

[1851] The processing flow will be explained below.

[1852] ---

[1853] Step 1:

[1854] During the initial setup, users enter their TOEIC score, English skill level, and Google Calendar integration information into the device, which then sends this data to the server.

[1855] Step 2:

[1856] The server receives the TOEIC score and English skill level data sent by the user. For example, if a TOEIC score of 650 is used, the English skill level is set as "intermediate."

[1857] Step 3:

[1858] The server evaluates the user's skill level based on the received data. For example, a TOEIC score of 650 would be rated as "intermediate."

[1859] Step 4:

[1860] The server requests permission to retrieve the user's Google Calendar information. If the user grants access, the server retrieves the calendar information.

[1861] Step 5:

[1862] The server detects available time from the acquired schedule information. For example, it identifies available time from 7:30 to 8:00 AM on weekdays.

[1863] Step 6:

[1864] The user terminal collects the user's facial expression and voice data in real time and transmits them to the server, which then recognizes the user's emotional state.

[1865] Step 7:

[1866] The server uses an emotion recognition engine to determine the user's emotional state, for example, determining that the user is feeling tired.

[1867] Step 8:

[1868] The server selects a training menu based on the user's skill level and emotional state, for example, selecting "intermediate reading training" and adjusting the difficulty level.

[1869] Step 9:

[1870] The server sends a notification to the user device when the scheduled training session approaches, with a message such as "The next learning session will start at 7:30."

[1871] Step 10:

[1872] The user terminal receives the notification and displays it to the user as a pop-up message, and the user confirms the start time of the next learning session.

[1873] Step 11:

[1874] At the specified time (7:30 AM in this case), the user's device will automatically launch the English learning application, and the user will begin training in the app.

[1875] Step 12:

[1876] The user progresses through the learning process according to a designated training menu (e.g., "Intermediate Reading Training"). During the training, the user's facial expressions and voice data are collected again, and emotional changes are recorded.

[1877] Step 13:

[1878] When the learning session ends, the user's device records the training progress and emotional data and transmits it to the server.

[1879] Step 14:

[1880] The server analyzes the received progress data and emotion data and provides feedback to the user, such as "80% correct answer rate in reading, high stress level."

[1881] By taking these steps, users can efficiently utilize their spare time to study English and obtain an optimal learning experience that takes into account their emotional state, thereby improving learning effectiveness and helping to establish consistent learning habits.

[1882] Example 2

[1883] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1884] Conventional English learning support systems are limited in providing training tailored to the user's skill level, and have had the problem of difficulty in providing flexible learning content that responds to the user's emotional state and real-time progress.In addition, they lack the ability to automatically set a learning schedule that utilizes the user's free time, or to provide feedback based on emotional recognition during learning, making it impossible to provide an optimal learning experience for users.

[1885] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1886] In this invention, the server includes means for receiving language ability data from a user, means for evaluating the user's skill level based on the received language ability data, means for recognizing the user's skill level and emotional state and automatically selecting a training menu accordingly, means for acquiring the user's schedule information, means for scheduling a training menu during free time in the acquired schedule information and sending a notification to the user terminal at the training start time, means for automatically starting learning software installed on the user terminal at a specified time, means for acquiring the user's progress and emotional data in real time while learning, and means for analyzing the acquired progress and emotional data, generating feedback, and providing it to the user. This makes it possible to provide flexible and optimal learning content according to the user's emotional state and real-time progress, and to automatically set an efficient learning schedule that utilizes the user's free time and provide feedback in real time.

[1887] "Language Proficiency Data" refers to TOEIC scores and other data indicating English proficiency for assessing a user's English skills.

[1888] The "skill level" is a level indicating the user's English ability as a result of evaluating the user's English ability.

[1889] "Emotional state" refers to the user's psychological and emotional state, which is determined by analyzing the user's facial expressions and voice data.

[1890] A "training menu" is a set of specific exercises and tasks for learning English that are selected based on the user's skill level and emotional state.

[1891] "Schedule information" is information relating to a user's schedule and free time, and is managed, for example, by a calendar.

[1892] "Free time" is a time period in the user's schedule information in which no other plans are scheduled.

[1893] A "notification" is a message sent from the server to the user terminal, informing the user of the start time and content of the training menu.

[1894] "Learning software" refers to an application for learning English that is installed and used by users.

[1895] "Progress" is information that indicates how much of a task a user has completed while studying English.

[1896] "Real-time" means that the actions and states of the user at that moment are reflected immediately as they progress through their learning process.

[1897] "Feedback" refers to evaluations and advice provided by the server regarding the learning a user has done.

[1898] The present invention provides a system for effectively supporting a user's English learning, and in particular, a system for optimizing the learning experience by recognizing the user's emotions. Specific embodiments for implementing the present invention are described below.

[1899] Basic system configuration

[1900] This system consists of a server and a user device. The server receives language ability data and schedule information sent by the user, and uses this information to select training menus and manage schedules. It also uses an emotion recognition engine to determine the user's emotional state and flexibly adjust the learning content. The user device receives notifications from the server, automatically launches the learning software, and acquires learning progress data in real time and sends it to the server.

[1901] Hardware and Software

[1902] The entire system is implemented using the following hardware and software:

[1903] Server: Uses a cloud server or on-premise server to perform data analysis, emotion recognition and scheduling.

[1904] User devices: Use devices that can install learning software, such as computers, smartphones, or tablets.

[1905] Learning software: An application for learning English that includes the ability to record the user's learning progress and emotional state.

[1906] Emotion recognition engine: For example, emotion recognition is performed using Microsoft Azure's Emotion API.

[1907] Data processing and calculation

[1908] The server performs the following steps to process and calculate the data:

[1909] 1. Data reception and processing: The server receives language ability data (e.g., TOEIC score) and schedule information sent by the user and analyzes them as initial data.

[1910] 2. Skill level assessment: Based on the received language proficiency data, a calculation process is performed to assess the user's skill level.

[1911] 3. Emotion Recognition: An emotion recognition engine is used to analyze the user's facial expressions and voice data to determine their emotional state.

[1912] 4. Training Menu Selection: Select an appropriate training menu based on assessed skill level and emotional state.

[1913] 5. Schedule Management: Detects free time from user schedule information and automatically schedules training sessions.

[1914] 6. Notifications and Feedback: Send notifications at the scheduled start time of training, and provide feedback after learning by analyzing progress and sentiment data obtained.

[1915] Specific examples

[1916] For example, suppose a user with a TOEIC score of 650 registers with the system. Based on the received score, the server evaluates the user's skill level as "intermediate." The server then analyzes the user's facial expression and voice data and determines that the user is tired.

[1917] The server selects a lower level of difficulty from the "Intermediate Reading Training" and schedules the training session during the user's free time (e.g., weekdays from 7:30 to 8:00 a.m.). At the designated time, the learning software installed on the user's device automatically starts up and the training begins.

[1918] After the user completes the training, the learning software sends progress and emotional data to the server, which analyzes the progress and emotional data and provides feedback to the user, such as "80% correct reading, high stress level."

[1919] Prompt Sentence Examples

[1920] "If a user with a TOEIC score of 650 is determined to be fatigued, generate a prompt from the AI ​​system that selects relatively easy intermediate reading training tasks and provides a way to schedule the training during the user's free time."

[1921] As such, the present invention is a system that provides flexible and optimal learning according to the user's skill level and emotional state, efficiently manages learning schedules by utilizing free time, and provides real-time feedback.

[1922] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1923] Step 1:

[1924] Entering and submitting user data

[1925] During initial setup, the user enters their TOEIC score and English skill level and sends them to the server. The system also links with Google Calendar. The input for this step is the user's language ability data and schedule information, such as "TOEIC score 650, skill level intermediate." The output is the user data sent to the server.

[1926] Step 2:

[1927] Data reception by the server and skill evaluation

[1928] The server receives the language ability data sent by the user and evaluates the user's skill level based on this. The input to this step is the language ability data sent by the user. The data is saved in an internal database and an evaluation process is performed. For example, if the user's TOEIC score is 650, the user's skill level is evaluated as "intermediate." The output is the data that evaluates the user's skill level.

[1929] Step 3:

[1930] Emotion recognition by server

[1931] The server analyzes the facial expression and voice data sent from the user device to determine the user's emotional state. The input for this step is the facial expression and voice data sent from the user device. An emotion recognition engine (e.g., Emotion API) is used to analyze the data and determine whether the user is tired. The output is data indicating the user's emotional state.

[1932] Step 4:

[1933] Selection of training menu

[1934] The server selects the optimal training menu based on the user's skill level and emotional state. The input for this step is the user's skill level data and emotional state data. A data analysis algorithm is used to identify a training menu suitable for the user. For example, if the user is tired, a less difficult "intermediate reading training" is selected. The output is the selected training menu.

[1935] Step 5:

[1936] Schedule registration by server

[1937] The server retrieves schedule information from the user's Google Calendar and detects available time slots. The input to this step is the user's schedule information. The server analyzes the available time slots and schedules training sessions for those times. For example, the server schedules sessions from 7:30 to 8:00 AM on weekdays. The output is the training sessions registered in the schedule.

[1938] Step 6:

[1939] Sending notifications

[1940] The server sends a notification to the user terminal at the scheduled training start time. The input for this step is the schedule information and the selected training menu. The notification system is used to send the specified message to the user terminal. The output is a notification displayed on the user terminal.

[1941] Step 7:

[1942] Receiving notifications on the user device and automatically launching the app

[1943] The user device receives the notification sent from the server and automatically launches the learning software at the specified time. The input to this step is the notification from the server. The software is launched at the specified time using the application auto-launch function. For example, at 7:30 AM, the app to start "Intermediate Reading Training" is launched. The output is the launched learning software.

[1944] Step 8:

[1945] User-led learning sessions

[1946] The user trains using the automatically launched learning software. The inputs of this step are the launched learning software and the training menu. The user progresses through the specified training. The outputs are the training progress and emotion data.

[1947] Step 9:

[1948] Progress data transmission by user terminal

[1949] After the training session is completed, the user device sends the progress and emotion data to the server. The input of this step is the progress and emotion data collected during training. The data transmission function is used to transmit this data to the server. The output is the progress and emotion data transmitted to the server.

[1950] Step 10:

[1951] Data analysis by server and feedback transmission

[1952] The server analyzes the received data and sends feedback to the user. The input of this step is the user's progress data and emotional data. Using a data analysis algorithm, the server identifies the user's learning achievements and areas for improvement and generates feedback. For example, it generates feedback such as "80% correct reading, high stress level." The output is the generated feedback.

[1953] (Application example 2)

[1954] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1955] Conventional English learning support systems focus on providing training menus based on the user's skill level, but do not adequately address optimization based on the user's emotional state or learning environment. As a result, even if the user feels tired or stressed, a fixed learning menu is provided, which can reduce learning efficiency. Furthermore, systems lack functionality to effectively support learning activities in physical stores, and it is necessary to provide an optimal learning experience tailored to the store environment.

[1956] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1957] In this invention, the server includes means for receiving English proficiency data from a user, means for evaluating the user's skill level based on the received English proficiency data, means for automatically selecting a training menu according to the user's skill level, means for acquiring the user's schedule information, means for scheduling a training menu during free time in the acquired schedule information, means for transmitting the scheduled training menu to a user device at a start time, means for automatically starting a learning program installed on the user device at a specified time, means for optimizing the learning content based on the store environment, and means for recognizing the user's emotional state and adjusting the learning content. This enables flexible training provision according to the user's emotional state and learning environment, and effective learning support within a physical store.

[1958] The "means for receiving English ability data from users" refers to a system by which the server receives ability data such as TOEIC scores and English skill levels provided by users.

[1959] The "means for assessing the user's skill level based on the received English proficiency data" is a system for analyzing the received data and assessing the user's current English proficiency based on specific criteria.

[1960] The "means for automatically selecting a training menu according to the user's skill level" is a system for automatically selecting the most suitable English training program based on the evaluated skill level information.

[1961] "Means for obtaining user schedule information" refers to a system that allows the server to obtain schedule information such as the user's Google Calendar.

[1962] The "means for scheduling a training menu during free time in the acquired schedule information" is a system for identifying free time in the acquired schedule and allocating an English training session to that time.

[1963] The "means for transmitting to the user device at the start time of the scheduled training menu" is a system for sending a notification to the user's device, such as a smartphone or tablet, at the set training start time.

[1964] "Means for automatically starting a learning program installed on a user device at a specified time" refers to a system for automatically starting learning software installed on a user's terminal at a specified time.

[1965] The "means for optimizing learning content based on the store environment" is a system that collects environmental data such as noise levels and congestion levels in physical stores and optimizes English training content accordingly.

[1966] The "means for recognizing the user's emotional state and adjusting the learning content" is a system that uses a camera and microphone to analyze the user's facial expressions and tone of voice, and adjusts the English training content according to the recognized emotions.

[1967] The present invention is a system for effectively supporting users' English learning in a brick-and-mortar store, and in particular, a system that can recognize users' emotions and optimize the learning experience. The system includes the following main elements:

[1968] Main functions and processes of the server

[1969] 1. User data reception function

[1970] The server receives data such as TOEIC scores and English skill levels sent by users, providing the initial information for creating a personalized learning plan for each user.

[1971] 2. Skill evaluation function

[1972] The server evaluates the user's skill level based on the received English proficiency data. For example, if the TOEIC score is 650, the user's skill level is evaluated as "intermediate."

[1973] 3. Emotion recognition function

[1974] The server analyzes facial expression and voice data sent from the user's device and uses an emotion recognition engine to determine the user's emotional state. For example, it acquires information such as whether the user is tired or stressed. The hardware used includes a camera and microphone, and the software used includes a facial expression recognition API and a voice recognition API.

[1975] 4. Training menu selection function

[1976] The server selects the most appropriate training menu based on the user's skill level and emotional state. For example, if the user is tired, a menu with a lower level of difficulty can be selected.

[1977] 5. Schedule management function

[1978] The server retrieves the user's calendar information, finds available time slots, and schedules training sessions within those times. The software used includes the Google Calendar API.

[1979] 6. Notification function

[1980] The server sends a notification to the user device at the start time of the scheduled workout, which includes a menu that is fine-tuned according to the user's emotional state.

[1981] Main functions and processing of the user terminal

[1982] 1. In-store detection function

[1983] The user device uses Wi-Fi or Bluetooth beacons to detect when the user is in a physical store.

[1984] 2. Environment recognition function

[1985] The user device recognizes the noise level and occupancy status inside the store and sends the information to the server. The hardware used includes a microphone, and the software used includes a noise-canceling algorithm.

[1986] 3. Notification reception function

[1987] The user terminal receives the notification sent from the server and notifies the user of the start time and training content.

[1988] 4. App auto-start function

[1989] Upon receiving the notification, the learning program installed on the user's terminal will automatically start at the specified time.

[1990] 5. Learning Progression Function

[1991] The user device analyzes the user's facial expressions and voice data in real time and also records emotional changes during learning.

[1992] 6. Progress data transmission function

[1993] When the learning session ends, the user device transmits the training progress and emotional data to the server.

[1994] Specific examples

[1995] For example, suppose a user with a TOEIC score of 650 is registered in the system. In this case, the server evaluates the skill level as "intermediate." Then, it analyzes the user's facial expression and voice data and determines that the user is feeling fatigued. The server selects a relatively easy task from "intermediate reading training" and schedules it for an available time in the user's calendar. It determines that this training session will take place, for example, from 7:30 to 8:00 AM on a weekday. A notification is sent to the user's device, and the learning program automatically starts at the specified time. The user performs the "intermediate reading training," and after it is completed, progress data and emotional data are sent to the server.

[1996] Prompt Sentence Examples

[1997] "I'd like to do some listening practice during the cafe time today. If the cafe is quiet, I'd like more difficult questions, and if it's noisy, I'd like easier questions."

[1998] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1999] Step 1: Receiving user data

[2000] The server receives data such as TOEIC scores and English skill levels sent by users. When the user enters this data in the application and presses the submit button, the server receives the data and stores it in a database.

[2001] Input: TOEIC score, English skill level data

[2002] Output: Save user data to database

[2003] Step 2: Skills Assessment

[2004] The server evaluates the user's skill level based on the received English proficiency data. For example, if the TOEIC score is 650, the server will evaluate it as "intermediate." An evaluation algorithm is applied to this data to determine the user's skill level.

[2005] Input: English proficiency data

[2006] Output: Assessed skill level (e.g., intermediate)

[2007] Step 3: In-store detection

[2008] The user device uses Wi-Fi or Bluetooth beacons to detect when the user is in a physical store, which then transmits specific store environment information to the server.

[2009] Input: Signals from Wi-Fi or Bluetooth beacons

[2010] Output: Confirmation that the user is in the store

[2011] Step 4: Environmental awareness

[2012] The user device recognizes the noise level and congestion status inside the store and sends the information to the server. Specifically, it uses a microphone to collect surrounding sounds and analyzes the noise level.

[2013] Input: Audio data from the microphone

[2014] Output: Noise level, congestion data

[2015] Step 5: Emotion Recognition

[2016] The server analyzes the facial expression and voice data sent from the user's device and uses an emotion recognition engine to determine the user's emotional state, for example, extracting information such as whether the user is tired or stressed.

[2017] Input: facial expression data, voice data

[2018] Output: Emotional state assessment (e.g., tired)

[2019] Step 6: Choose your training menu

[2020] The server selects the most appropriate training menu based on the user's skill level and emotional state: for example, if the user is tired, it selects a reading training with a lower level of difficulty.

[2021] Input: Skill level, emotional state

[2022] Output: Selected training menu

[2023] Step 7: Retrieve schedule information

[2024] The server retrieves the user's calendar information, using the Google Calendar API to read the user's schedule and identify available times.

[2025] Input: User's Google Calendar information

[2026] Output: Free / busy information

[2027] Step 8: Training Schedule

[2028] The server schedules the training menu for the available time slots in the acquired schedule information. For example, the server may set a training session for the available time slots between 7:30 and 8:00 AM on weekdays.

[2029] Input: Free / busy information, training menu

[2030] Output: Scheduled training sessions

[2031] Step 9: Send notification

[2032] The server sends a notification to the user terminal at the start time of the scheduled training menu, which also includes the training content.

[2033] Input: Schedule a training session

[2034] Output: Notification to user terminal

[2035] Step 10: App Auto-Launch

[2036] The user terminal automatically starts the installed learning program at the specified time. Upon receiving the notification, the user terminal starts the program and begins training.

[2037] Input: Notification content

[2038] Output: Automatic launch of learning program

[2039] Step 11: Record your learning progress and emotional changes

[2040] The user device analyzes the user's facial expressions and voice data in real time and records emotional changes during learning. Specifically, it collects and analyzes data using a camera and microphone.

[2041] Input: Real-time facial expression data, voice data

[2042] Output: Recorded data of emotional changes

[2043] Step 12: Send progress data

[2044] Once the training session is over, the user device transmits training progress and emotional data to the server, which stores this data in a database for further analysis.

[2045] Input: Progress data, emotion data

[2046] Output: Send data to the server

[2047] Examples of prompts:

[2048] "I'd like to do some listening practice during the cafe time today. If the cafe is quiet, I'd like more difficult questions, and if it's noisy, I'd like easier questions."

[2049] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[2050] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[2051] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[2052] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[2053] FIG. 9 illustrates an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and behaviors arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[2054] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[2055] The inside of emotion map 400 represents what is going on in the mind...

Claims

1. means for receiving English proficiency data from a user; means for assessing the user's skill level based on the received English proficiency data; A means for automatically selecting a training menu according to the skill level of the user; A means for obtaining schedule information of a user; A means for scheduling a training menu during free time in the acquired schedule information; means for sending a notification to a user terminal at the start time of a scheduled training menu; and means for automatically starting a learning application installed on a user terminal at a specified time.

2. 2. The system according to claim 1, further comprising means for acquiring calendar information as schedule information of the user terminal.

3. The system of claim 1 further comprising means for capturing and transmitting to a server progress data of a user through a learning application.

Images