System

Abstract

A system is provided.SOLUTION: A system comprising means for a user to set a learning plan and a goal, means for explaining the content learned by the user by voice, means for converting the voice into text data, means for inputting the text data to a generative artificial intelligence and performing analysis, means for providing evaluation and feedback of learning content based on the analysis result, means for automatically creating a review test based on the feedback, means for evaluating the test result and recommending the next learning content, and means for updating the learning plan according to the progress of the user.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] ---

[0005] Many people, including working adults in their 20s to 50s, seek efficient learning to improve their skills, but face challenges in consolidating knowledge and continuing their studies. Furthermore, traditional input-focused learning methods make it difficult to accurately assess one's own level of understanding and progress, making effective review and relearning difficult. The purpose of this invention is to promote output-focused learning and support users in efficiently consolidating knowledge and improving their understanding. [Means for solving the problem]

[0006] The present invention provides a means for users to set study plans and goals, explain what they have learned, convert the explanations from voice to text data, use generative AI to analyze the text data and evaluate and provide feedback on the learning content, automatically create review tests based on the feedback, evaluate test results and recommend next study content, and update the study plan according to the user's progress. This allows users to deepen their understanding by outputting their own learning content and further their learning by receiving evaluations and feedback from the generative AI.

[0007] ---

[0008] "User" refers to an individual who uses the system to learn.

[0009] A "study plan" refers to a plan that includes goals set by a user and specific steps to achieve those goals.

[0010] "Goal" refers to a specific learning outcome that a user aims to achieve.

[0011] "Speech" refers to an acoustic signal input by a user.

[0012] "Audio instruction means" refers to a device consisting of hardware and software for receiving audio input.

[0013] "Text data" refers to digital data converted from voice input into characters.

[0014] "Means for converting into text data" refers to speech recognition technology for converting voice signals into text information.

[0015] "Generative artificial intelligence" refers to AI technologies that analyze data, evaluate learning content, and generate feedback.

[0016] "Means of analysis" refers to algorithms and methods for evaluating learning content based on text data.

[0017] "Feedback" refers to information including evaluations, comments, and advice regarding the user's learning content.

[0018] A "review test" refers to a test that allows users to review and solidify what they have learned.

[0019] "Means for automated creation" refers to algorithms or programs for generating the content of review tests.

[0020] "Test results" refers to the evaluation of the answers provided by the user to the review test.

[0021] "Recommendation methods" refers to methods for suggesting what to learn next based on test results and the user's progress.

[0022] "Means for updating the study plan" refers to algorithms or programs for modifying or updating an existing study plan according to the user's learning progress.

[0023] --- [Brief explanation of the drawings]

[0024] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0025] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0026] First, the terms used in the following description will be explained.

[0027] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0028] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0029] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0030] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0031] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0032] [First embodiment]

[0033] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0034] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0035] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0036] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0037] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0038] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0039] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0040] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0041] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0042] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0043] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0044] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0045] ---

[0046] The present invention is a system that allows users to study efficiently and effectively. This system involves a series of processes in which a user sets a study plan and goals, explains what they have learned, and a generative AI analyzes the content, provides feedback, and automatically creates review tests as needed.

[0047] System configuration

[0048] The system includes the following major components:

[0049] 1. User device: A device used by the user as an interface to set up learning plans, input data by voice, take tests, and perform other operations.

[0050] 2. Server: As a central processing unit, it receives data from users and performs analysis, feedback generation, automatic test generation, etc.

[0051] 3. Generative AI: This runs on the server and includes an algorithm that analyzes the learning content sent by the user and generates evaluations and feedback.

[0052] Specific functions of the system

[0053] 1. Study plan and goal setting

[0054] When a user logs in to the system, the terminal displays an interface for inputting their learning plan and goals. Through this interface, the user sets their own learning goals (e.g., obtaining a qualification, achieving a specific score, etc.) and creates a learning plan based on them. The server receives this information and stores it in a database.

[0055] 2. Explaining what has been learned and growing the generative AI

[0056] As the user progresses through their learning, the device prompts them to "explain what they learned." The user then explains what they learned aloud, and the device converts that speech into text data. The converted text data is sent to a server, where the AI ​​analyzes the content and generates an analysis result.

[0057] 3. Evaluation and feedback of what you have learned

[0058] The generation AI evaluates the user's explanation and generates feedback on any errors or insufficient understanding. The server compiles the evaluation results and feedback and sends them to the device. Through this feedback, the user can check their own understanding and make any necessary corrections.

[0059] 4. Automatic creation of review tests

[0060] The AI ​​automatically generates review tests based on the learning content with the highest confidence. The server customizes the tests, setting the difficulty and content according to the user's progress. The tests are sent to the device, and the user can take them to solidify the content they have learned.

[0061] 5. Evaluation of test results and recommendations for next study

[0062] The user takes the test and sends their answers to the server. The server evaluates the test results and stores the scores in a database. Based on the evaluation results, the server generates recommendations for what to study next and what points need to be re-studyed, and sends this to the device.

[0063] Specific examples

[0064] Setting up a study plan

[0065] When User A logs in to the system for the first time, the terminal displays "Please set your study plan and goals." User A enters "I'm aiming for a TOEIC score of 900," and the server receives the information and stores it in the database. The server then automatically generates an appropriate study plan and displays it to the user.

[0066] Explanation of what was learned

[0067] After learning English grammar, User A explains aloud to his / her device, "This week we learned about relative pronouns." The device converts the speech into text and sends it to the server. The generation AI analyzes the text and generates feedback saying, "You need to relearn the difference between the nominative and possessive case." The server sends this feedback to the device, and User A confirms it again.

[0068] Automatic creation of review tests

[0069] The server generates a test on relative pronouns and sends it to the device. User A takes the test and submits their answers, and the server evaluates the results. The evaluation result is displayed on the device along with feedback such as "You need to review the use of possessives a little more."

[0070] ---

[0071] As described above, the present invention is a system that provides a series of functions to efficiently and effectively advance the user's learning process. Analysis and feedback by generative AI, as well as the automatic creation of review tests, can deepen the user's understanding and help them solidify their knowledge.

[0072] The processing flow will be explained below.

[0073] ---

[0074] Step 1:

[0075] A user logs in to the system.

[0076] The terminal displays the login interface.

[0077] The user enters their login information and goes through the authentication process.

[0078] Step 2:

[0079] A learning plan and goal setting screen is displayed to the user whose device has been authenticated.

[0080] The user inputs their study plan and goals (e.g., aiming for a TOEIC score of 900).

[0081] The terminal transmits the input information to the server.

[0082] Step 3:

[0083] The server stores the user's learning plan and goals in a database.

[0084] The server generates an initial learning plan and schedule based on the received information.

[0085] The server sends the generated learning plan to the terminal.

[0086] Step 4:

[0087] The user progresses through the learning and is ready to explain what they learned.

[0088] The device prompts the user to "Describe what you learned."

[0089] Audibly describe what the user learned.

[0090] Step 5:

[0091] The terminal converts the user's voice input into text data.

[0092] The terminal transmits the converted text data to the server.

[0093] Step 6:

[0094] The server inputs the text data into the generation AI and begins analysis.

[0095] The generative AI evaluates the learning content based on text data.

[0096] Step 7:

[0097] The generative AI analyzes the user's explanation and generates evaluation results and feedback.

[0098] The server formats the generated evaluation results and feedback and sends them to the device.

[0099] Step 8:

[0100] The device displays the feedback to the user.

[0101] The user reviews the feedback and makes any necessary corrections.

[0102] Step 9:

[0103] The server instructs the AI ​​to generate review tests based on the feedback.

[0104] Generative AI determines the content of the review test and automatically creates the test.

[0105] Step 10:

[0106] The server customizes the generated tests and configures them as the user progresses.

[0107] The server sends the test to the device.

[0108] Step 11:

[0109] The device displays a test notification to the user, prompting them to take the test.

[0110] The user performs the test on the device.

[0111] The user enters the answers to the test and the terminal sends the answers to the server.

[0112] Step 12:

[0113] The server evaluates the test results and stores the scores in a database.

[0114] The server generates additional feedback based on the test results.

[0115] Step 13:

[0116] The device displays additional feedback to the user, prompting them to relearn.

[0117] The user re-learns and explains the new learning content to the terminal.

[0118] Step 14:

[0119] The server periodically evaluates the user's learning progress.

[0120] The server automatically updates the learning plan based on progress.

[0121] The server sends recommendations to the device on what to learn next and how to learn it.

[0122] Step 15:

[0123] The device displays recommended information to the user, supporting continuous learning.

[0124] ---

[0125] The above is the specific processing flow of the system, which effectively supports the user's learning process and aims to ensure that knowledge is continuously retained through evaluation and feedback by the generative AI.

[0126] Example 1

[0127] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0128] Conventional learning support systems have had difficulty providing feedback and review content tailored to each user's individual learning progress and level of understanding. This has led to problems such as reduced learning efficiency and effectiveness. Furthermore, it has been difficult to evaluate the specific level of understanding of the content a user has learned and to appropriately recommend the next learning content.

[0129] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0130] In this invention, the server includes means for a user to set a study plan and goals, means for audibly explaining what the user has learned, means for converting the speech into text data, means for inputting the text data into a generative artificial intelligence and analyzing it, means for evaluating the study content and providing feedback based on the analysis results, means for automatically creating review tests based on the feedback, means for evaluating the test results and recommending next study content, means for updating the study plan according to the user's progress, means for user login authentication, means for storing the user's study content and results in a database, and means for customizing the content of review tests based on the user's progress. This makes it possible to improve the efficiency and effectiveness of users' learning and realize optimal feedback and recommendations for next study content according to individual learning progress.

[0131] A "user" is a person who uses the system to set up a learning plan and manage the progress of their learning.

[0132] A "server" is a central processing unit that receives data from users, performs analysis, feedback, and automatically generates tests, and provides these results to users.

[0133] A "terminal" is a device that a user uses as an interface, and is used to perform operations such as setting up a study plan, inputting voice data, and taking tests.

[0134] A "study plan" is a specific study schedule and procedure for achieving a goal set by a user.

[0135] A "goal" is a specific learning outcome or grade that a user aims to achieve.

[0136] "Audio" refers to the spoken words that a user makes to explain what they have learned.

[0137] "Text data" is voice data converted into text information, and is the input data for analysis by the generative AI model.

[0138] "Generative AI" includes algorithms and techniques for analyzing user-provided data and generating ratings and feedback.

[0139] "Analysis" is the process of evaluating content based on text data and identifying errors or areas of lack of understanding.

[0140] "Feedback" refers to evaluation information on areas for improvement and learning content provided to the user based on the analysis results.

[0141] A "review test" is a confirmation test created to help users solidify what they have learned.

[0142] "Recommendations" are information that suggests what content the user should study next based on their learning results.

[0143] "Progress" refers to the process or state in which a user advances their studies according to a study plan.

[0144] "Login authentication" is a procedure for verifying a user's identity when accessing a system.

[0145] A "database" is a storage device for saving the user's learning content and results.

[0146] "Customization" refers to the process of adjusting the content and difficulty level according to the user's progress and level of understanding.

[0147] The present invention is a system that allows users to study efficiently and effectively. This system involves a series of processes in which a user sets a study plan and goals, explains what they have learned, and a generative AI analyzes the content, provides feedback, and automatically creates review tests as needed.

[0148] System configuration

[0149] The system includes the following major components:

[0150] 1. User device: A device used by users as an interface to set up learning plans, input voice commands, take tests, etc. User devices include PCs, smartphones, tablets, etc.

[0151] 2. Server: A central processing unit that receives data from users, analyzes it, creates feedback, automatically generates tests, etc. The server is a high-performance computer, and the database can also be stored within the server.

[0152] 3. Generative AI: This includes algorithms that run on a server, analyze the learning content sent by the user, and generate evaluations and feedback. Generative AI is realized using natural language processing and machine learning techniques.

[0153] Study plan and goal setting

[0154] When a user logs in to the system, the terminal displays an interface for inputting their learning plan and goals. Through this interface, the user sets their own learning goals (e.g., obtaining a qualification, achieving a specific score, etc.) and creates a learning plan based on them. The server receives this information and stores it in a database.

[0155] Explanation of what was learned

[0156] As the user progresses through their learning, the device prompts them to "explain what they learned." The user then explains what they learned aloud, and the device converts that speech into text data. The converted text data is sent to a server, where the AI ​​analyzes the content and generates an analysis result.

[0157] Assessment and feedback of what you learned

[0158] The generative AI evaluates the user's explanation and generates feedback on any errors or insufficient understanding. The server compiles the evaluation results and feedback and sends them to the device. Through this feedback, the user can check their own understanding and make any necessary corrections.

[0159] Automatic creation of review tests

[0160] The AI ​​automatically generates review tests based on the learning content with the highest confidence. The server customizes these tests, setting the difficulty and content according to the user's progress. The tests are sent to the device, and the user can take them to solidify what they have learned.

[0161] Evaluation of test results and recommendations for next study content

[0162] The user takes the test and sends their answers to the server. The server evaluates the test results and stores the scores in a database. Based on the evaluation results, the server generates recommendations for what to study next and what points need to be re-studyed, and sends this information to the device.

[0163] Specific examples

[0164] Setting up a study plan

[0165] When User A logs in to the system for the first time, the terminal displays "Please set your study plan and goals." User A enters "I'm aiming for a TOEIC score of 900," and the server receives the information and stores it in the database. The server then automatically generates an appropriate study plan and displays it to the user.

[0166] Explanation of what was learned

[0167] After learning English grammar, User A explains aloud to his / her device, "This week we learned about relative pronouns." The device converts the speech into text and sends it to the server. The generation AI analyzes the text and generates feedback saying, "You need to relearn the difference between the nominative and possessive case." The server sends this feedback to the device, and User A confirms it again.

[0168] Automatic creation of review tests

[0169] The server generates a test on relative pronouns and sends it to the device. User A takes the test and submits their answers, and the server evaluates the results. The evaluation result is displayed on the device along with feedback such as "You need to review the use of possessives a little more."

[0170] Prompt Sentence Examples

[0171] "Please explain what you learned this week."

[0172] Please suggest what we should learn next.

[0173] "Evaluate the results of the test and provide feedback."

[0174] In this way, the system for implementing the invention provides a series of functions to efficiently and effectively advance the user's learning process. Analysis and feedback using a generative AI model, as well as the automatic creation of review tests, can deepen the user's understanding and help them solidify their knowledge.

[0175] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0176] Step 1: User Login

[0177] A user logs in to the system using a terminal. The terminal displays an interface for entering a user ID and password, and the user enters the information and presses the "Login" button. The server authenticates the received login information and retrieves the user's account information from the database. If authentication is successful, the server returns a login success status to the terminal and displays the main menu to the user.

[0178] Input: User ID, Password

[0179] Output: Login successful status, main menu displayed

[0180] Specific behavior:

[0181] 1. The device displays the login page.

[0182] 2. The user enters their ID and password and submits it.

[0183] 3. The server receives the login information and authenticates it with the database.

[0184] 4. When authentication is successful, the main menu is displayed on the device.

[0185] Step 2: Setting a study plan and goals

[0186] The device displays a prompt saying, "Please set your study plan and goals." The user then inputs their study goals and plan into the interface. For example, they can set a goal such as "Aim for a TOEIC score of 900." The server receives this information and stores it in a database. The server then automatically suggests study content related to the user's goals and displays them as a list on the device.

[0187] Input: Learning objectives, learning plan

[0188] Output: Saved learning plans, suggested learning content

[0189] Specific behavior:

[0190] 1. The device displays an interface that prompts the user to enter their learning plan and goals.

[0191] 2. The user enters and submits the learning objectives.

[0192] 3. The server stores this information in a database.

[0193] 4. The server suggests relevant learning content and displays it on the device.

[0194] Step 3: Explain what you learned

[0195] As the user progresses through their learning, the device prompts them to "explain what they learned." The user then explains what they learned by voice. For example, they might say, "This week we learned about relative pronouns." The device converts the voice data into text data and sends it to the server. The generative AI analyzes this text data and understands its content.

[0196] Input: Audio explanation of what you're learning

[0197] Output: Text data, analysis results

[0198] Specific behavior:

[0199] 1. The device prompts the user for voice input.

[0200] 2. Explain in audio what the user learned.

[0201] 3. The device converts the voice into text data.

[0202] 4. The text data is sent to the server and analyzed by the generating AI.

[0203] Step 4: Assessment and feedback on what was learned

[0204] The generation AI analyzes the text data and evaluates the content of the user's explanation. As a result of the analysis, it extracts any errors or parts of insufficient understanding and generates feedback based on this. For example, it may create feedback such as "You need to relearn the difference between the nominative and possessive case." The server compiles this feedback and sends it to the device, where it is displayed to the user.

[0205] Input: Analysis results of text data

[0206] Output: Feedback

[0207] Specific behavior:

[0208] 1. Generative AI analyzes text data.

[0209] 2. Detect errors or areas of lack of understanding.

[0210] 3. Feedback is generated and sent to the device by the server.

[0211] 4. The device displays feedback to the user.

[0212] Step 5: Automatically create review tests

[0213] The AI ​​automatically generates a review test based on the user's level of understanding. The server generates specific questions based on the user's learning content and assessment results. For example, it creates multiple-choice questions and fill-in-the-blank questions about relative pronouns. The test is sent to the device and prepared for the user to take.

[0214] Input: Learning content, evaluation results

[0215] Output: Review test

[0216] Specific behavior:

[0217] 1. Generative AI generates test questions based on the evaluation results.

[0218] 2. Customize the server-generated tests.

[0219] 3. Send the test to the device and display it to the user.

[0220] Step 6: Evaluate test results and recommend next study content

[0221] The user completes the test and sends the answers to the server via their device. The server evaluates the test results and stores the scores in a database. Based on the evaluation results, the generative AI recommends what to study next and what points need to be re-studyed. For example, specific advice such as "You need to review the use of possessives a little more" is displayed on the device.

[0222] Input: Test Answers

[0223] Output: Evaluation results, recommendations for next learning content

[0224] Specific behavior:

[0225] 1. A user takes a test and submits their answers.

[0226] 2. The server evaluates the test results and stores the scores in a database.

[0227] 3. The generative AI recommends the next learning content based on the evaluation results.

[0228] 4. The server sends the recommendation information to the device and displays it to the user.

[0229] (Application example 1)

[0230] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0231] Conventional learning support systems lack the functionality to provide detailed feedback and appropriate review tests to help users progress through their studies efficiently and effectively. They also lack a system that can flexibly update learning plans according to the user's progress, making them unsuitable tools for maximizing the effectiveness of learning. Furthermore, they lack sufficient accuracy and flexibility when it comes to explaining learning content through voice input and providing analysis and feedback using generative AI. These issues need to be resolved.

[0232] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0233] In this invention, the server includes: means for a user to set a study plan and goals; means for audibly explaining what the user has learned; means for converting the speech into text data; means for inputting the text data into a generative artificial intelligence for analysis; means for evaluating the study content and providing feedback based on the analysis results; means for automatically creating review tests based on the feedback; means for evaluating the test results and recommending next study content; means for updating the study plan according to the user's progress; means for generating test questions based on a specified study topic using a smart device; and means for analyzing the content explained by the user audibly when generating feedback to the user using the generative artificial intelligence. This enables the user to study efficiently and effectively and receive appropriate feedback and review tests.

[0234] "Means for setting study plans and goals" is a system that allows users to clarify the goals they want to achieve and set specific study steps and schedules based on those goals.

[0235] The "means for explaining learned content by voice" is a system that allows the user to explain what they have learned by voice input.

[0236] "Means for converting voice into text data" refers to a technique for converting voice information input by a user into text information.

[0237] "Means of inputting text data into a generative artificial intelligence for analysis" refers to the function of inputting converted text information into an AI and analyzing its content.

[0238] The "means for evaluating learning content and providing feedback" is a system that evaluates learning content based on the content analyzed by AI and provides users with feedback on areas for improvement and their level of understanding.

[0239] The "means for automatically generating review tests" is a mechanism for automatically generating test questions for users to review based on the provided feedback.

[0240] "Means for evaluating test results and recommending next study content" is a function that evaluates the results of a test taken by the user and recommends the next study content based on the results.

[0241] The "means for updating the study plan according to the user's progress" is a system that dynamically updates the set study plan according to the user's progress in learning.

[0242] "Means for generating test questions based on a specified learning topic using a smart device" refers to technology that uses a mobile or wearable device to create test questions related to a learning topic.

[0243] "Means for analyzing the content explained by a user through voice when generating feedback to the user using generative artificial intelligence" refers to a system in which AI analyzes the learning content explained through voice and generates feedback based on the results.

[0244] This invention is a system that allows users to study efficiently and effectively. The system provides functions such as setting study plans and goals, explaining study content through voice input, analyzing text and providing feedback using generative AI, automatically creating review tests, and recommending next study content.

[0245] Hardware and software used

[0246] Hardware: Smart devices (smartphones, tablets, smart glasses, etc.), servers.

[0247] Software: Speech recognition libraries (e.g. speech_recognition), generative AI libraries (e.g. Huggingface's transformers library).

[0248] Specific functions of the system

[0249] 1. Study plan and goal setting

[0250] Users use the device to set their study plans and goals by entering their goals and selecting relevant study topics through an on-screen interface.

[0251] Example prompt sentence:

[0252] set_learning_plan(goal="Pass the qualification exam", topics=["Grammar", "Reading"])

[0253] 2. Explanation of learning content by voice input

[0254] As the user progresses through the learning process, they explain what they have learned by voice into the device, which uses speech recognition technology to convert this speech into text data.

[0255] Example prompt sentence:

[0256] This week we learned about relative pronouns

[0257] 3. Analysis and feedback by generative AI

[0258] The converted text data is sent to a server where it is analyzed by a generative AI model, which then generates feedback on the user's level of understanding and key points based on the analysis results.

[0259] Example prompt sentence:

[0260] Analyze the following and provide feedback:This week we learned about relative pronouns.

[0261] 4. Automatic creation of review tests

[0262] Based on the feedback, a review test is generated, which corresponds to the learning topic and can be taken on a smart device.

[0263] Example prompt sentence:

[0264] generate_test(["relative pronoun", "subjunctive mood"])

[0265] 5. Evaluation of test results and recommendations for next study

[0266] The test results taken by the user are evaluated on the server and the next study content is recommended, allowing the user to continuously update their optimal study plan according to their own progress.

[0267] Specific examples

[0268] Setting up a study plan

[0269] When User A logs in to the system for the first time, the terminal displays "Please set your study plan and goals." User A enters "I'm aiming for a TOEIC score of 900," and the server receives the information and stores it in the database. The server then automatically generates an appropriate study plan and displays it to the user.

[0270] Voice input explained

[0271] After learning English grammar, User A explains to his / her device, "This week we learned about relative pronouns." The speech is converted into text and sent to the server. The generation AI analyzes the text and generates feedback saying, "You need to relearn the difference between the nominative and possessive case." The server sends this feedback to the device, and User A confirms it again.

[0272] Automatic creation of review tests

[0273] The server generates a test on relative pronouns and sends it to the device. User A takes the test and submits their answers, and the server evaluates the results. The evaluation result is displayed on the device along with feedback such as "You need to review the use of possessives a little more."

[0274] This system allows users to study efficiently and effectively, and provides appropriate feedback and review tests.

[0275] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0276] Step 1: Setting a study plan and goals

[0277] Input: The user inputs the "goal" and "learning topic" using the terminal.

[0278] Specific operation: The terminal displays a user interface, and the user inputs a goal (e.g., TOEIC score of 900 points) and a learning topic (e.g., listening and reading).

[0279] Output: The server receives these inputs and stores them in a database.

[0280] Data processing: The server organizes the input data and formats it as initial information for automatically generating a learning plan.

[0281] Step 2: Explain the learning content by voice input

[0282] Input: The user speaks what they learned into the device

[0283] Specific operation: When the user speaks a description, the device activates a speech recognition module (e.g., speech_recognition) and converts the speech into text data.

[0284] Output: Text data is sent to the server.

[0285] Data processing: The device converts the voice data into text data and formats it appropriately before sending it to the server.

[0286] Step 3: Analysis and feedback by generative AI

[0287] Input: Text data is sent to the server

[0288] How it works: The server inputs text data into a generative AI model (e.g., Huggingface's transformers), analyzes the content, and generates feedback based on the analysis results.

[0289] Output: The generated feedback is sent to the user terminal.

[0290] Data processing: The server converts the text data into a format suitable for the generative AI model, analyzes the analysis results, and formats them as feedback.

[0291] Step 4: Automatically create review tests

[0292] Input: Information based on generated feedback and learning topics

[0293] Specific operation: The server automatically generates review test questions based on the results of the generative AI model.

[0294] Output: The generated test questions are sent to the user's terminal.

[0295] Data processing: The server combines data to generate test questions based on the learning topic and feedback, and formats it into question format.

[0296] Step 5: Evaluate test results and recommend next study content

[0297] Input: The user takes the review test and enters the answers into the device.

[0298] Specific operation: The device sends the user's test answers to the server, which evaluates the results and recommends the next learning content based on the evaluation results.

[0299] Output: The evaluation results and the next learning content are sent to the user's terminal.

[0300] Data processing: The server analyzes the test answer data, generates evaluation results, and determines the recommended content for the next topic to learn.

[0301] This series of processes allows users to study efficiently and effectively, from setting up a study plan to reviewing the content they have learned, taking review tests, and receiving recommendations for the next study session.

[0302] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0303] ---

[0304] This invention is a system that allows users to study efficiently and effectively, and provides more personalized feedback by combining an emotion engine that recognizes the user's emotions. This system involves a series of processes in which the user sets a study plan and goals, explains what they have learned, and a generative AI analyzes the content and provides feedback, and automatically creates review tests as needed.

[0305] System configuration

[0306] The system includes the following major components:

[0307] 1. User device: A device used by the user as an interface to set up learning plans, input data by voice, take tests, and perform other operations.

[0308] 2. Server: As a central processing unit, it receives data from users and performs analysis, feedback generation, automatic test generation, etc.

[0309] 3. Generative AI: This runs on the server and includes an algorithm that analyzes the learning content sent by the user and generates evaluations and feedback.

[0310] 4. Emotion Engine: A system for recognizing emotions from the user's voice input and adjusting feedback based on the analysis results.

[0311] Specific functions of the system

[0312] 1. Study plan and goal setting

[0313] When a user logs in to the system, the terminal displays an interface for inputting their learning plan and goals. Through this interface, the user sets their own learning goals (e.g., obtaining a qualification, achieving a specific score, etc.) and creates a learning plan based on them. The server receives this information and stores it in a database.

[0314] 2. Explaining what has been learned and growing the generative AI

[0315] As the user progresses through their learning, the device prompts them to "explain what they learned." The user then explains what they learned aloud, and the device converts that speech into text data. The converted text data is sent to a server, where the AI ​​analyzes the content and generates an analysis result.

[0316] 3. Emotion Recognition and Analysis Using an Emotion Engine

[0317] The emotion engine recognizes the user's emotional state from their voice input. The emotion engine's analysis results are provided to the generative AI, which then adjusts the feedback taking into account the user's emotional state. For example, if the user is feeling anxious, the feedback will be provided in a gentler tone.

[0318] 4. Assessment and feedback of what you learned

[0319] The generative AI evaluates the user's explanation and creates feedback on any errors or insufficient understanding. Taking into account the results of the emotion analysis by the emotion engine, the server compiles the evaluation results and feedback and sends them to the device. Through this feedback, the user can check their level of understanding and make any necessary corrections.

[0320] 5. Automatic creation of review tests

[0321] The AI ​​automatically generates review tests based on the learning content with the highest confidence. The server customizes the tests, setting the difficulty and content according to the user's progress. The tests are sent to the device, and the user can take them to solidify the content they have learned.

[0322] 6. Evaluation of test results and recommendations for next study

[0323] The user takes the test and sends their answers to the server. The server evaluates the test results and stores the scores in a database. Based on the evaluation results, the server generates recommendations for what to study next and what points need to be re-studyed, and sends this to the device.

[0324] Specific examples

[0325] Setting up a study plan

[0326] When User B logs in to the system for the first time, the terminal displays "Please set your study plan and goals." User B enters "My goal is a TOEIC score of 900," and the server receives the information and stores it in the database. The server then automatically generates an appropriate study plan and displays it to the user.

[0327] Explanation of what was learned

[0328] After learning English grammar, User B explains to his device, "This week we learned about relative pronouns." The device converts the speech into text and sends it to the server. The generation AI analyzes the text and generates feedback saying, "You need to relearn the difference between the nominative and possessive." The emotion engine recognizes from User B's voice that "you seem impatient," and provides the feedback in a softer tone.

[0329] Automatic creation of review tests

[0330] The server generates a test on relative pronouns and sends it to the device. User B takes the test and submits their answers, and the server evaluates the results. The evaluation result is displayed on the device along with feedback such as "You need to review the use of possessives a little more."

[0331] ---

[0332] As described above, the present invention is a system that provides a series of functions to efficiently and effectively advance the user's learning process. Through analysis and feedback by generative AI, emotion recognition and adjustment by an emotion engine, and automatic creation of review tests, the system can deepen the user's understanding and solidify their knowledge.

[0333] The processing flow will be explained below.

[0334] ---

[0335] Step 1:

[0336] A user logs in to the system.

[0337] The terminal displays the login interface.

[0338] The user enters their login information and goes through the authentication process.

[0339] Step 2:

[0340] A learning plan and goal setting screen is displayed to the user whose device has been authenticated.

[0341] The user inputs their study plan and goals (e.g., aiming for a TOEIC score of 900).

[0342] The terminal transmits the input information to the server.

[0343] Step 3:

[0344] The server stores the user's learning plan and goals in a database.

[0345] The server generates an initial learning plan and schedule based on the received information.

[0346] The server sends the generated learning plan to the terminal.

[0347] Step 4:

[0348] The user progresses through the learning and is ready to explain what they learned.

[0349] The device prompts the user to "Describe what you learned."

[0350] Audibly describe what the user learned.

[0351] Step 5:

[0352] The terminal converts the user's voice input into text data.

[0353] The terminal transmits the converted text data to the server.

[0354] Step 6:

[0355] The server sends the text data to the emotion engine.

[0356] The emotion engine recognizes the user's emotional state from their voice.

[0357] The emotion engine sends the emotion analysis results to the server.

[0358] Step 7:

[0359] The server inputs the text data into the generation AI and begins analysis.

[0360] The generative AI evaluates the learning content based on text data and sentiment analysis results.

[0361] Step 8:

[0362] The generative AI analyzes the user's explanation and generates evaluation results and feedback.

[0363] The generative AI takes into account the user's emotional state and adjusts the tone and content of the feedback.

[0364] The server formats the generated evaluation results and feedback and sends them to the device.

[0365] Step 9:

[0366] The device displays the feedback to the user.

[0367] The user reviews the feedback and makes any necessary corrections.

[0368] Step 10:

[0369] The server instructs the AI ​​to generate review tests based on the feedback.

[0370] Generative AI determines the content of the review test and automatically creates the test.

[0371] Step 11:

[0372] The server customizes the generated tests and configures them as the user progresses.

[0373] The server sends the test to the device.

[0374] Step 12:

[0375] The device displays a test notification to the user, prompting them to take the test.

[0376] The user performs the test on the device.

[0377] The user enters the answers to the test and the terminal sends the answers to the server.

[0378] Step 13:

[0379] The server evaluates the test results and stores the scores in a database.

[0380] The server generates additional feedback based on the test results.

[0381] Step 14:

[0382] The device displays additional feedback to the user, prompting them to relearn.

[0383] The user re-learns and explains the new learning content to the terminal.

[0384] Step 15:

[0385] The server periodically evaluates the user's learning progress.

[0386] The server automatically updates the learning plan based on progress.

[0387] The server sends recommendations to the device on what to learn next and how to learn it.

[0388] Step 16:

[0389] The device displays recommended information to the user, supporting continuous learning.

[0390] ---

[0391] The above is the specific processing flow of the system, which effectively supports the user's learning process and aims to solidify knowledge through evaluation and feedback by the generative AI, as well as emotion recognition and adjustment by the emotion engine.

[0392] Example 2

[0393] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0394] Conventional learning support systems provide uniform feedback without considering the user's emotional state, making it difficult to provide appropriate instruction tailored to individual needs. Furthermore, evaluation of the user's learning content and recommendations for the next lesson are often done manually, resulting in reduced learning efficiency. Furthermore, in many cases, only standard test questions are provided, resulting in a lack of customized tests tailored to the user's level of understanding. A system that can solve these issues and provide a more personalized learning experience is needed.

[0395] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0396] In this invention, the server includes means for a user to set a study plan and goals, means for audibly explaining what the user has learned, means for converting the speech into text data, means for inputting the text data into a generative artificial intelligence model and analyzing it, means for evaluating the study content and providing feedback based on the analysis results, means for recognizing the user's emotional state, means for adjusting the feedback taking the user's emotional state into account, means for automatically creating a review test based on the feedback, means for evaluating the test results and recommending the next study content, and means for updating the study plan according to the user's progress. This makes it possible to provide feedback that takes the user's emotional state into account and tests that are customized according to the user's level of understanding.

[0397] A "user" is an individual or group who uses the system to set a learning plan and goals and progress through the learning content.

[0398] "Study planning" is the process of setting specific study content and schedules to be achieved within a specific period based on the study goals that the user wants to achieve.

[0399] A "goal" is a specific outcome or performance that a user wants to achieve through the system.

[0400] "Audio explanation means" is a method by which a user verbally conveys what they have learned to the system.

[0401] "Means for converting into text data" refers to a technology for converting a user's voice input into text format data.

[0402] A "generative artificial intelligence model" is an artificial intelligence algorithm that analyzes data entered by the user and evaluates learning content and generates feedback.

[0403] "Means of analysis" refers to the process of using a generative artificial intelligence model to understand and evaluate the learning content explained by the user.

[0404] "Means for providing feedback" refers to a method for informing users of their learning progress and areas for improvement based on the analysis results.

[0405] "Means for recognizing emotional states" refers to technology for identifying emotions from a user's voice or text data and detecting changes in the user's emotions.

[0406] "Feedback tailoring" is the process of optimizing the content and tone of the feedback provided, taking into account the user's emotional state.

[0407] The "means for automatically creating review tests" is a technology that automatically generates review test questions based on the user's learning content and level of understanding.

[0408] "Means for evaluating test results" refers to the process of analyzing the results of the test taken by the user and evaluating their performance.

[0409] "Means for recommending next learning content" is a process that recommends the next learning content or points that need to be re-learned based on test results and the user's progress.

[0410] The "means for updating the study plan" is a method for appropriately modifying an existing study plan in accordance with the user's progress and updating it to match the latest study goals.

[0411] The present invention provides a learning support system that helps users study efficiently and effectively, and in particular has a feedback function that takes into account the user's emotional state. This system is composed of the following main hardware and software components:

[0412] Hardware and software used

[0413] 1. User Device

[0414] A device that a user uses as an interface to perform operations such as setting up a study plan, inputting voice data, taking tests, and receiving feedback. Specific examples include personal computers, smartphones, and tablets. These devices use voice recognition software (e.g., Google® Cloud Speech-to-Text) to convert the user's speech into text.

[0415] 2. Server

[0416] As a central processing unit, it receives data from users and performs analysis, creates feedback, automatically generates tests, etc. On the server, a generative AI model (e.g., GPT-4 (registered trademark)) and an emotion engine (e.g., Affectiva) run to analyze the user's learning content and emotional state.

[0417] 3. Generative AI Models

[0418] The generative AI model, which runs on a server and includes algorithms that analyze the text data submitted by users, evaluate their learning, generate feedback, and create review tests, is known as a large-scale language model.

[0419] 4. Emotion Engine

[0420] The emotion engine is a system for recognizing the emotional state of a user from their voice input and providing the analysis results to a generative AI model, which then adjusts the feedback according to the user's emotions.

[0421] Specific processing flow and functions

[0422] Study plan and goal setting

[0423] When a user logs in to the system, the terminal displays the message "Please set your study plan and goals." The user enters their study goal (e.g., get a high score on a test) and the terminal sends this information to the server, which receives it and stores it in a database.

[0424] Specific behavior:

[0425] The user inputs "I want to make a study plan with the goal of passing a qualification exam" into the terminal, and the server receives the information and displays a confirmation message to the user.

[0426] Explanation of what was learned

[0427] After the user has progressed through the learning process, the device will prompt them to "explain what you learned." The user will explain aloud, and the device will use speech recognition software to convert the speech into text.

[0428] Specific behavior:

[0429] When a user speaks to the device, "I learned about differential and integral calculus this weekend," the device uses voice recognition software to convert this into text data, "I learned about differential and integral calculus this weekend," and sends it to the server.

[0430] Analysis of learning content and generation of feedback

[0431] The generative AI model on the server analyzes the text data sent by the user and evaluates the learning content. The emotion engine recognizes emotions from the user's voice and provides the results to the generative AI model. The generative AI model adjusts the feedback taking the emotional state into account.

[0432] Specific behavior:

[0433] If the server assesses that the learning content is fully understood and the emotion engine recognizes that the user is showing signs of impatience during the explanation, the generative AI model will provide gentle feedback such as, "You have a solid understanding of the basics of calculus. Let's try some applied problems next."

[0434] Automatic creation of review tests

[0435] Based on the analysis results, the generative AI model automatically generates review tests to improve the user's understanding. The server selects appropriate test questions and sends them to the device.

[0436] Specific behavior:

[0437] The server generates five questions on the basic concepts of calculus, which the terminal displays as a test to the user. The user takes the test and sends the answers to the server.

[0438] Test result evaluation and study recommendations

[0439] The server evaluates the test results and recommends the next study content to the user based on the results, allowing the user to develop an efficient study plan that is in line with their own learning progress.

[0440] Specific behavior:

[0441] After the user completes the test and the server evaluates it, a recommendation is displayed on the device saying, "We recommend that you review the basic concepts again before moving on to the application questions."

[0442] Prompt Sentence Examples

[0443] "Set a study plan and goals."

[0444] "Please explain what you learned."

[0445] "Please explain what we learned this week in audio."

[0446] "Please begin the review test."

[0447] The above is a specific embodiment for carrying out the invention. By using this system, users can receive personalized feedback according to their learning progress and promote effective learning.

[0448] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0449] Step 1:

[0450] A user logs into the system

[0451] Input: Username and Password

[0452] Data processing: The device sends the entered information to the server. The server performs user authentication and, if successful, displays the home screen.

[0453] Output: The user can access the home screen.

[0454] Specific behavior:

[0455] The user enters their username and password into the device and presses the send button. The device sends this information to the server, which then authenticates them and displays the home screen on the device.

[0456] Step 2:

[0457] Your device will display the study plan and goal setting screen.

[0458] Input: Learning objectives and plan content

[0459] Data processing: The terminal receives the user's input and sends it to the server, which stores the information in a database.

[0460] Output: The configured learning plan is saved with a confirmation message.

[0461] Specific behavior:

[0462] The user enters "I'm aiming for a TOEIC score of 900" and presses "Confirm." The device sends this information to the server, which saves it in the database. A message indicating that saving has been completed is displayed on the device.

[0463] Step 3:

[0464] The user explains the learning content aloud

[0465] Input: User voice input

[0466] Data processing: Use speech recognition software (e.g., Google Cloud Speech-to-Text) to convert voice input into text data. The text is sent to the server.

[0467] Output: The learning content in text format is saved on the server.

[0468] Specific behavior:

[0469] The user speaks to the device, saying, "This weekend I learned about calculus." The device converts the speech to text using Google Cloud Speech-to-Text and sends it to the server. The text data is stored on the server.

[0470] Step 4:

[0471] The server analyzes the learning content

[0472] Input: Text data (user learning content)

[0473] Data processing: A generative AI model (e.g., GPT-4) analyzes the text data and evaluates what it has learned.

[0474] Output: Analysis results (evaluation details)

[0475] Specific behavior:

[0476] The server generates an analysis result that says, "The basics of differential and integral calculus are understood, but a review is needed before moving on to applied problems."

[0477] Step 5:

[0478] Emotion engine recognizes emotional states from speech

[0479] Input: Audio data

[0480] Data processing: An emotion engine (e.g., Affectiva) analyzes the audio data and recognizes the emotional state.

[0481] Output: Emotional state data

[0482] Specific behavior:

[0483] The user sends a voice that indicates impatience while giving an explanation to the server, and the emotion engine recognizes the "impatience."

[0484] Step 6:

[0485] Server provides tailored feedback

[0486] Input: Analysis results and emotional state

[0487] Data processing: Generative AI takes into account your emotional state and adjusts the content and tone of your feedback.

[0488] Output: Regulated Feedback

[0489] Specific behavior:

[0490] The server generates feedback in a gentle tone, saying, "You have a solid understanding of the basics of differential and integral calculus. Now let's try some applied problems." The feedback is displayed on the terminal.

[0491] Step 7:

[0492] Automatically create review quizzes

[0493] Input: Analysis results

[0494] Data processing: Generative AI automatically generates review tests based on the analysis results.

[0495] Output: A set of review test questions

[0496] Specific behavior:

[0497] The server generates "five problems on the basic concepts of differential and integral calculus" and sends them to the terminal as a test.

[0498] Step 8:

[0499] User takes the test and submits the results

[0500] Input: User's test answers

[0501] Data processing: The terminal sends the user's answers to the server, which evaluates the answers.

[0502] Output: Evaluation results

[0503] Specific behavior:

[0504] The user answers the questions and submits the results. The server evaluates the test results and generates feedback such as "Area of ​​improvement: You need to use possessives and relative pronouns appropriately."

[0505] Step 9:

[0506] Recommend what to study next

[0507] Input: Evaluation result

[0508] Data processing: The server generates the next learning content based on the evaluation results and makes recommendations.

[0509] Output: Recommended learning content

[0510] Specific behavior:

[0511] The server generates a message saying, "Next, we recommend that you learn about the use of possessives and relative pronouns," and displays it on the terminal.

[0512] These are the main processing steps of the system, which allows users to receive personalized feedback based on their learning progress and effectively advance their learning.

[0513] (Application example 2)

[0514] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0515] Previous learning support systems relied on standard methods to evaluate and provide feedback on users' learning content, but were unable to consider the emotional state of each individual user. As a result, learning effectiveness could decline if users became stressed or lost motivation. Furthermore, the customization of review tests and study plans did not reflect the user's psychological state, resulting in a lack of personalized learning support.

[0516] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0517] In this invention, the server includes: means for a user to set a study plan and goals; means for audibly explaining what the user has learned; means for converting the voice into text data; means for inputting the text data into a generative artificial intelligence and analyzing it; means including an emotion engine for recognizing the emotional state of the user from the voice input; means for evaluating the study content and providing feedback based on the analysis results and the emotional state; means for automatically creating review tests based on the feedback; means for evaluating the test results and recommending next study content; and means for updating the study plan according to the user's progress. This makes it possible to create personalized feedback and tests that take the user's emotional state into consideration, which is expected to improve learning effectiveness.

[0518] "Users"

[0519] refers to an individual who uses a learning system to carry out learning activities.

[0520] "Study Plan"

[0521] A learning plan is a plan that describes specific learning content and schedules to achieve the goals set by the user.

[0522] "Voice input"

[0523] is a means for users to provide information to the system by dictating what they have learned.

[0524] "Text data"

[0525] It is data that converts voice input into text format.

[0526] "Generative Artificial Intelligence"

[0527] is a system that includes algorithms and programs for analyzing text data converted from voice input.

[0528] "Emotion Engine"

[0529] is a system for recognizing and analyzing a user's emotional state from their voice input.

[0530] "feedback"

[0531] This is information that includes evaluations and advice on what the user has learned.

[0532] "Review test"

[0533] These are tests that the system automatically creates to help users confirm and solidify what they have learned.

[0534] "Next lesson content"

[0535] This refers to the content or topic that the user should study next.

[0536] "Recommendation"

[0537] This refers to the learning content and methods that the system suggests based on the user's learning progress and test results.

[0538] "progress"

[0539] This refers to the user's level of achievement and progress as they progress through their studies.

[0540] "update"

[0541] This refers to modifying and updating a user's study plan based on the latest information and progress.

[0542] This invention provides a system for enabling users to study efficiently and effectively. The system includes means for setting a user's study plan and goals and for explaining the user's learned content via voice. A server converts the user's voice into text data, which is then input into a generative AI model for analysis. The system also uses an emotion engine to recognize the user's emotional state from the user's voice input and adjusts feedback based on this. The system automatically creates review tests based on the feedback, evaluates the test results, and recommends next study content. The system also includes means for updating the study plan according to the user's progress.

[0543] System configuration

[0544] The system includes the following major components:

[0545] 1. User device: A device used by the user as an interface to set up learning plans, input voice, take tests, etc. User devices can be VR headsets, smartphones, PCs, etc.

[0546] 2. Server: As a central processing unit, it receives data from users and performs analysis, feedback generation, automatic test generation, etc.

[0547] 3. Generative AI model: Runs on the server and includes an algorithm that analyzes the learning content sent by the user and generates evaluation and feedback.

[0548] 4. Emotion Engine: A system for recognizing emotions from the user's voice input and adjusting feedback based on the analysis results.

[0549] Specific functions of the system

[0550] 1. Study plan and goal setting

[0551] When a user logs in to the system, the terminal displays an interface for inputting their learning plan and goals. Through this interface, the user sets their own learning goals (e.g., obtaining a qualification, achieving a specific score, etc.) and creates a learning plan based on them. The server receives this information and stores it in a database.

[0552] 2. Explaining what has been learned and analyzing the generative AI model

[0553] After the user has progressed through their learning, the device prompts them to "explain what they have learned." The user then explains what they have learned by voice, and the device converts that speech into text data. The converted text data is sent to a server, where the generative AI model analyzes the content and generates analysis results.

[0554] 3. Emotion Recognition and Analysis Using an Emotion Engine

[0555] The emotion engine recognizes the user's emotional state from their voice input. The emotion engine's analysis results are fed into a generative AI model, which then adjusts the feedback based on the user's emotional state. For example, if the user is feeling anxious, the feedback will be delivered in a gentler tone.

[0556] 4. Assessment and feedback of what you learned

[0557] The generative AI model evaluates the user's explanation and generates feedback on any errors or insufficient understanding. Taking into account the emotion analysis results of the emotion engine, the server compiles the evaluation results and feedback and sends them to the device. Through this feedback, the user can check their own level of understanding and make any necessary corrections.

[0558] 5. Automatic creation of review tests

[0559] The generative AI model automatically generates review tests based on the learning content with the highest confidence. The server customizes these tests, setting the difficulty and content according to the user's progress. The tests are sent to the device, and the user can take them to solidify what they have learned.

[0560] 6. Evaluation of test results and recommendations for next study

[0561] The user takes the test and sends their answers to the server. The server evaluates the test results and stores the scores in a database. Based on the evaluation results, the server generates recommendations for what to study next and what points need to be re-studyed, and sends this to the device.

[0562] Specific examples

[0563] Setting up a study plan

[0564] When a user logs in to the system for the first time, the terminal displays, "Please set your study plan and goals." The user enters, "My goal is a TOEIC score of 900," and the server receives the information and stores it in the database. The server then automatically generates an appropriate study plan and displays it to the user.

[0565] Explanation of what was learned

[0566] After learning English grammar, a user speaks to their device, saying, "This week we learned about relative pronouns." The device converts the speech into text and sends it to the server. The generative AI model analyzes the text and generates feedback saying, "You need to relearn the difference between nominative and possessive." The emotion engine recognizes from the user's voice that "you seem impatient" and provides the feedback in a softer tone. An example of feedback would be, "Let's review the difference between nominative and possessive again."

[0567] Automatic creation of review tests

[0568] The server generates a test on relative pronouns and sends it to the device. The user takes the test, submits their answers, and the server evaluates the results. The evaluation result is displayed on the device along with feedback such as "You need to review the use of possessives a little more."

[0569] Prompt Sentence Examples

[0570] "To deepen your understanding of neural networks, let's next tackle some practical problems."

[0571] As described above, this invention is a system that utilizes an emotion engine and a generative AI model to support the user's learning process and provide a more effective learning experience.

[0572] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0573] Step 1:

[0574] The device displays an interface for the user to set up a learning plan and goals. The user enters their learning goals (e.g., obtaining a qualification, achieving a specific score, etc.) on this screen, and the device sends the information to the server. The server stores the received information in a database.

[0575] Input: User-entered learning plans and goals

[0576] Output: Learning plans and goals received and stored by the server

[0577] Specific operation: When a user enters "My goal is a TOEIC score of 900" and clicks the data submission button, the data is sent from the device to the server, which then writes and saves this data in the database.

[0578] Step 2:

[0579] After the user has progressed through their studies, the device prompts them to "explain what they have learned." The user explains what they have learned by voice, and the device converts the voice into text data.

[0580] Input: User voice input

[0581] Output: Learning content converted into text data

[0582] What it does: A user says, "This week we learned about relative pronouns," and the system recognizes the speech and converts it into text.

[0583] Step 3:

[0584] The converted text data is sent from the device to a server and input into a generative AI model, which analyzes the content and generates an evaluation and feedback.

[0585] Input: Learning content converted into text data

[0586] Output: Analysis results, evaluation, and feedback from the generative AI model

[0587] Specific operation: The text data "I learned about relative pronouns" is sent to the generative AI model for analysis. The generative AI model generates feedback saying "You need to re-learn the difference between nominative and possessive cases."

[0588] Step 4:

[0589] The emotion engine recognizes the user's emotional state from their voice input, and the analysis results of the emotion engine are provided to the generative AI model, which then adjusts the feedback taking into account the user's emotional state.

[0590] Input: User's voice data

[0591] Output: User's emotional state (e.g., impatience)

[0592] Specific operation: The emotion engine recognizes "impatience" from the user's voice and provides the analysis results to the generative AI model. The generative AI model then adjusts the voice to "provide feedback in a gentler tone because the user is impatient."

[0593] Step 5:

[0594] The server sends the adjusted feedback to the device, which then displays the feedback to the user, who then checks the feedback and determines their level of understanding.

[0595] Input: Calibrated feedback

[0596] Output: Feedback that is displayed to the user

[0597] What happens: The server sends the adjusted feedback "Let's review the difference between nominative and possessive again" to the device, which then displays it to the user.

[0598] Step 6:

[0599] The generative AI model automatically generates review tests based on the user's learning, and the server customizes the tests. The tests are then sent to the user's device, where they are then taken.

[0600] Input: tailored feedback, what the user has learned

[0601] Output: Auto-generated review test

[0602] Specific operation: A test on relative pronouns is generated, adjusted to an appropriate level of difficulty by the server, and sent to the device. The user then takes the test.

[0603] Step 7:

[0604] The user takes the test and sends the answers from the device to the server. The server evaluates the results and stores them in a database. It then generates recommendations for what to study next and what points need to be re-studyed, and sends them to the device.

[0605] Input: Test answer data

[0606] Output: Evaluation results and next training recommendation

[0607] What it does: The user takes the test and submits their answers, and the server evaluates the results. The evaluation result is displayed on the device along with feedback such as "You need to review the use of possessives a little more."

[0608] The above processing steps enable the system to create personalized feedback and tests that take into account the user's emotional state, which is expected to improve learning effectiveness.

[0609] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0610] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0611] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0612] [Second embodiment]

[0613] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0614] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0615] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0616] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0617] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0618] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0619] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0620] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0621] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0622] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0623] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0624] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0625] ---

[0626] The present invention is a system that allows users to study efficiently and effectively. This system involves a series of processes in which a user sets a study plan and goals, explains what they have learned, and a generative AI analyzes the content, provides feedback, and automatically creates review tests as needed.

[0627] System configuration

[0628] The system includes the following major components:

[0629] 1. User device: A device used by the user as an interface to set up learning plans, input data by voice, take tests, and perform other operations.

[0630] 2. Server: As a central processing unit, it receives data from users and performs analysis, feedback generation, automatic test generation, etc.

[0631] 3. Generative AI: This runs on the server and includes an algorithm that analyzes the learning content sent by the user and generates evaluations and feedback.

[0632] Specific functions of the system

[0633] 1. Study plan and goal setting

[0634] When a user logs in to the system, the terminal displays an interface for inputting their learning plan and goals. Through this interface, the user sets their own learning goals (e.g., obtaining a qualification, achieving a specific score, etc.) and creates a learning plan based on them. The server receives this information and stores it in a database.

[0635] 2. Explaining what has been learned and growing the generative AI

[0636] As the user progresses through their learning, the device prompts them to "explain what they learned." The user then explains what they learned aloud, and the device converts that speech into text data. The converted text data is sent to a server, where the AI ​​analyzes the content and generates an analysis result.

[0637] 3. Evaluation and feedback of what you have learned

[0638] The generation AI evaluates the user's explanation and generates feedback on any errors or insufficient understanding. The server compiles the evaluation results and feedback and sends them to the device. Through this feedback, the user can check their own understanding and make any necessary corrections.

[0639] 4. Automatic creation of review tests

[0640] The AI ​​automatically generates review tests based on the learning content with the highest confidence. The server customizes the tests, setting the difficulty and content according to the user's progress. The tests are sent to the device, and the user can take them to solidify the content they have learned.

[0641] 5. Evaluation of test results and recommendations for next study

[0642] The user takes the test and sends their answers to the server. The server evaluates the test results and stores the scores in a database. Based on the evaluation results, the server generates recommendations for what to study next and what points need to be re-studyed, and sends this to the device.

[0643] Specific examples

[0644] Setting up a study plan

[0645] When User A logs in to the system for the first time, the terminal displays "Please set your study plan and goals." User A enters "I'm aiming for a TOEIC score of 900," and the server receives the information and stores it in the database. The server then automatically generates an appropriate study plan and displays it to the user.

[0646] Explanation of what was learned

[0647] After learning English grammar, User A explains aloud to his / her device, "This week we learned about relative pronouns." The device converts the speech into text and sends it to the server. The generation AI analyzes the text and generates feedback saying, "You need to relearn the difference between the nominative and possessive case." The server sends this feedback to the device, and User A confirms it again.

[0648] Automatic creation of review tests

[0649] The server generates a test on relative pronouns and sends it to the device. User A takes the test and submits their answers, and the server evaluates the results. The evaluation result is displayed on the device along with feedback such as "You need to review the use of possessives a little more."

[0650] ---

[0651] As described above, the present invention is a system that provides a series of functions to efficiently and effectively advance the user's learning process. Analysis and feedback by generative AI, as well as the automatic creation of review tests, can deepen the user's understanding and help them solidify their knowledge.

[0652] The processing flow will be explained below.

[0653] ---

[0654] Step 1:

[0655] A user logs in to the system.

[0656] The terminal displays the login interface.

[0657] The user enters their login information and goes through the authentication process.

[0658] Step 2:

[0659] A learning plan and goal setting screen is displayed to the user whose device has been authenticated.

[0660] The user inputs their study plan and goals (e.g., aiming for a TOEIC score of 900).

[0661] The terminal transmits the input information to the server.

[0662] Step 3:

[0663] The server stores the user's learning plan and goals in a database.

[0664] The server generates an initial learning plan and schedule based on the received information.

[0665] The server sends the generated learning plan to the terminal.

[0666] Step 4:

[0667] The user progresses through the learning and is ready to explain what they learned.

[0668] The device prompts the user to "Describe what you learned."

[0669] Audibly describe what the user learned.

[0670] Step 5:

[0671] The terminal converts the user's voice input into text data.

[0672] The terminal transmits the converted text data to the server.

[0673] Step 6:

[0674] The server inputs the text data into the generation AI and begins analysis.

[0675] The generative AI evaluates the learning content based on text data.

[0676] Step 7:

[0677] The generative AI analyzes the user's explanation and generates evaluation results and feedback.

[0678] The server formats the generated evaluation results and feedback and sends them to the device.

[0679] Step 8:

[0680] The device displays the feedback to the user.

[0681] The user reviews the feedback and makes any necessary corrections.

[0682] Step 9:

[0683] The server instructs the AI ​​to generate review tests based on the feedback.

[0684] Generative AI determines the content of the review test and automatically creates the test.

[0685] Step 10:

[0686] The server customizes the generated tests and configures them as the user progresses.

[0687] The server sends the test to the device.

[0688] Step 11:

[0689] The device displays a test notification to the user, prompting them to take the test.

[0690] The user performs the test on the device.

[0691] The user enters the answers to the test and the terminal sends the answers to the server.

[0692] Step 12:

[0693] The server evaluates the test results and stores the scores in a database.

[0694] The server generates additional feedback based on the test results.

[0695] Step 13:

[0696] The device displays additional feedback to the user, prompting them to relearn.

[0697] The user re-learns and explains the new learning content to the terminal.

[0698] Step 14:

[0699] The server periodically evaluates the user's learning progress.

[0700] The server automatically updates the learning plan based on progress.

[0701] The server sends recommendations to the device on what to learn next and how to learn it.

[0702] Step 15:

[0703] The device displays recommended information to the user, supporting continuous learning.

[0704] ---

[0705] The above is the specific processing flow of the system, which effectively supports the user's learning process and aims to ensure that knowledge is continuously retained through evaluation and feedback by the generative AI.

[0706] Example 1

[0707] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0708] Conventional learning support systems have had difficulty providing feedback and review content tailored to each user's individual learning progress and level of understanding. This has led to problems such as reduced learning efficiency and effectiveness. Furthermore, it has been difficult to evaluate the specific level of understanding of the content a user has learned and to appropriately recommend the next learning content.

[0709] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0710] In this invention, the server includes means for a user to set a study plan and goals, means for audibly explaining what the user has learned, means for converting the speech into text data, means for inputting the text data into a generative artificial intelligence and analyzing it, means for evaluating the study content and providing feedback based on the analysis results, means for automatically creating review tests based on the feedback, means for evaluating the test results and recommending next study content, means for updating the study plan according to the user's progress, means for user login authentication, means for storing the user's study content and results in a database, and means for customizing the content of review tests based on the user's progress. This makes it possible to improve the efficiency and effectiveness of users' learning and realize optimal feedback and recommendations for next study content according to individual learning progress.

[0711] A "user" is a person who uses the system to set up a learning plan and manage the progress of their learning.

[0712] A "server" is a central processing unit that receives data from users, performs analysis, feedback, and automatically generates tests, and provides these results to users.

[0713] A "terminal" is a device that a user uses as an interface, and is used to perform operations such as setting up a study plan, inputting voice data, and taking tests.

[0714] A "study plan" is a specific study schedule and procedure for achieving a goal set by a user.

[0715] A "goal" is a specific learning outcome or grade that a user aims to achieve.

[0716] "Audio" refers to the spoken words that a user makes to explain what they have learned.

[0717] "Text data" is voice data converted into text information, and is the input data for analysis by the generative AI model.

[0718] "Generative AI" includes algorithms and techniques for analyzing user-provided data and generating ratings and feedback.

[0719] "Analysis" is the process of evaluating content based on text data and identifying errors or areas of lack of understanding.

[0720] "Feedback" refers to evaluation information on areas for improvement and learning content provided to the user based on the analysis results.

[0721] A "review test" is a confirmation test created to help users solidify what they have learned.

[0722] "Recommendations" are information that suggests what content the user should study next based on their learning results.

[0723] "Progress" refers to the process or state in which a user advances their studies according to a study plan.

[0724] "Login authentication" is a procedure for verifying a user's identity when accessing a system.

[0725] A "database" is a storage device for saving the user's learning content and results.

[0726] "Customization" refers to the process of adjusting the content and difficulty level according to the user's progress and level of understanding.

[0727] The present invention is a system that allows users to study efficiently and effectively. This system involves a series of processes in which a user sets a study plan and goals, explains what they have learned, and a generative AI analyzes the content, provides feedback, and automatically creates review tests as needed.

[0728] System configuration

[0729] The system includes the following major components:

[0730] 1. User device: A device used by users as an interface to set up learning plans, input voice commands, take tests, etc. User devices include PCs, smartphones, tablets, etc.

[0731] 2. Server: A central processing unit that receives data from users, analyzes it, creates feedback, automatically generates tests, etc. The server is a high-performance computer, and the database can also be stored within the server.

[0732] 3. Generative AI: This includes algorithms that run on a server, analyze the learning content sent by the user, and generate evaluations and feedback. Generative AI is realized using natural language processing and machine learning techniques.

[0733] Study plan and goal setting

[0734] When a user logs in to the system, the terminal displays an interface for inputting their learning plan and goals. Through this interface, the user sets their own learning goals (e.g., obtaining a qualification, achieving a specific score, etc.) and creates a learning plan based on them. The server receives this information and stores it in a database.

[0735] Explanation of what was learned

[0736] As the user progresses through their learning, the device prompts them to "explain what they learned." The user then explains what they learned aloud, and the device converts that speech into text data. The converted text data is sent to a server, where the AI ​​analyzes the content and generates an analysis result.

[0737] Assessment and feedback of what you learned

[0738] The generative AI evaluates the user's explanation and generates feedback on any errors or insufficient understanding. The server compiles the evaluation results and feedback and sends them to the device. Through this feedback, the user can check their own understanding and make any necessary corrections.

[0739] Automatic creation of review tests

[0740] The AI ​​automatically generates review tests based on the learning content with the highest confidence. The server customizes these tests, setting the difficulty and content according to the user's progress. The tests are sent to the device, and the user can take them to solidify what they have learned.

[0741] Evaluation of test results and recommendations for next study content

[0742] The user takes the test and sends their answers to the server. The server evaluates the test results and stores the scores in a database. Based on the evaluation results, the server generates recommendations for what to study next and what points need to be re-studyed, and sends this information to the device.

[0743] Specific examples

[0744] Setting up a study plan

[0745] When User A logs in to the system for the first time, the terminal displays "Please set your study plan and goals." User A enters "I'm aiming for a TOEIC score of 900," and the server receives the information and stores it in the database. The server then automatically generates an appropriate study plan and displays it to the user.

[0746] Explanation of what was learned

[0747] After learning English grammar, User A explains aloud to his / her device, "This week we learned about relative pronouns." The device converts the speech into text and sends it to the server. The generation AI analyzes the text and generates feedback saying, "You need to relearn the difference between the nominative and possessive case." The server sends this feedback to the device, and User A confirms it again.

[0748] Automatic creation of review tests

[0749] The server generates a test on relative pronouns and sends it to the device. User A takes the test and submits their answers, and the server evaluates the results. The evaluation result is displayed on the device along with feedback such as "You need to review the use of possessives a little more."

[0750] Prompt Sentence Examples

[0751] "Please explain what you learned this week."

[0752] Please suggest what we should learn next.

[0753] "Evaluate the results of the test and provide feedback."

[0754] In this way, the system for implementing the invention provides a series of functions to efficiently and effectively advance the user's learning process. Analysis and feedback using a generative AI model, as well as the automatic creation of review tests, can deepen the user's understanding and help them solidify their knowledge.

[0755] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0756] Step 1: User Login

[0757] A user logs in to the system using a terminal. The terminal displays an interface for entering a user ID and password, and the user enters the information and presses the "Login" button. The server authenticates the received login information and retrieves the user's account information from the database. If authentication is successful, the server returns a login success status to the terminal and displays the main menu to the user.

[0758] Input: User ID, Password

[0759] Output: Login successful status, main menu displayed

[0760] Specific behavior:

[0761] 1. The device displays the login page.

[0762] 2. The user enters their ID and password and submits it.

[0763] 3. The server receives the login information and authenticates it with the database.

[0764] 4. When authentication is successful, the main menu is displayed on the device.

[0765] Step 2: Setting a study plan and goals

[0766] The device displays a prompt saying, "Please set your study plan and goals." The user then inputs their study goals and plan into the interface. For example, they can set a goal such as "Aim for a TOEIC score of 900." The server receives this information and stores it in a database. The server then automatically suggests study content related to the user's goals and displays them as a list on the device.

[0767] Input: Learning objectives, learning plan

[0768] Output: Saved learning plans, suggested learning content

[0769] Specific behavior:

[0770] 1. The device displays an interface that prompts the user to enter their learning plan and goals.

[0771] 2. The user enters and submits the learning objectives.

[0772] 3. The server stores this information in a database.

[0773] 4. The server suggests relevant learning content and displays it on the device.

[0774] Step 3: Explain what you learned

[0775] As the user progresses through their learning, the device prompts them to "explain what they learned." The user then explains what they learned by voice. For example, they might say, "This week we learned about relative pronouns." The device converts the voice data into text data and sends it to the server. The generative AI analyzes this text data and understands its content.

[0776] Input: Audio explanation of what you're learning

[0777] Output: Text data, analysis results

[0778] Specific behavior:

[0779] 1. The device prompts the user for voice input.

[0780] 2. Explain in audio what the user learned.

[0781] 3. The device converts the voice into text data.

[0782] 4. The text data is sent to the server and analyzed by the generating AI.

[0783] Step 4: Assessment and feedback on what was learned

[0784] The generation AI analyzes the text data and evaluates the content of the user's explanation. As a result of the analysis, it extracts any errors or parts of insufficient understanding and generates feedback based on this. For example, it may create feedback such as "You need to relearn the difference between the nominative and possessive case." The server compiles this feedback and sends it to the device, where it is displayed to the user.

[0785] Input: Analysis results of text data

[0786] Output: Feedback

[0787] Specific behavior:

[0788] 1. Generative AI analyzes text data.

[0789] 2. Detect errors or areas of lack of understanding.

[0790] 3. Feedback is generated and sent to the device by the server.

[0791] 4. The device displays feedback to the user.

[0792] Step 5: Automatically create review tests

[0793] The AI ​​automatically generates a review test based on the user's level of understanding. The server generates specific questions based on the user's learning content and assessment results. For example, it creates multiple-choice questions and fill-in-the-blank questions about relative pronouns. The test is sent to the device and prepared for the user to take.

[0794] Input: Learning content, evaluation results

[0795] Output: Review test

[0796] Specific behavior:

[0797] 1. Generative AI generates test questions based on the evaluation results.

[0798] 2. Customize the server-generated tests.

[0799] 3. Send the test to the device and display it to the user.

[0800] Step 6: Evaluate test results and recommend next study content

[0801] The user completes the test and sends the answers to the server via their device. The server evaluates the test results and stores the scores in a database. Based on the evaluation results, the generative AI recommends what to study next and what points need to be re-studyed. For example, specific advice such as "You need to review the use of possessives a little more" is displayed on the device.

[0802] Input: Test Answers

[0803] Output: Evaluation results, recommendations for next learning content

[0804] Specific behavior:

[0805] 1. A user takes a test and submits their answers.

[0806] 2. The server evaluates the test results and stores the scores in a database.

[0807] 3. The generative AI recommends the next learning content based on the evaluation results.

[0808] 4. The server sends the recommendation information to the device and displays it to the user.

[0809] (Application example 1)

[0810] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0811] Conventional learning support systems lack the functionality to provide detailed feedback and appropriate review tests to help users progress through their studies efficiently and effectively. They also lack a system that can flexibly update learning plans according to the user's progress, making them unsuitable tools for maximizing the effectiveness of learning. Furthermore, they lack sufficient accuracy and flexibility when it comes to explaining learning content through voice input and providing analysis and feedback using generative AI. These issues need to be resolved.

[0812] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0813] In this invention, the server includes: means for a user to set a study plan and goals; means for audibly explaining what the user has learned; means for converting the speech into text data; means for inputting the text data into a generative artificial intelligence for analysis; means for evaluating the study content and providing feedback based on the analysis results; means for automatically creating review tests based on the feedback; means for evaluating the test results and recommending next study content; means for updating the study plan according to the user's progress; means for generating test questions based on a specified study topic using a smart device; and means for analyzing the content explained by the user audibly when generating feedback to the user using the generative artificial intelligence. This enables the user to study efficiently and effectively and receive appropriate feedback and review tests.

[0814] "Means for setting study plans and goals" is a system that allows users to clarify the goals they want to achieve and set specific study steps and schedules based on those goals.

[0815] The "means for explaining learned content by voice" is a system that allows the user to explain what they have learned by voice input.

[0816] "Means for converting voice into text data" refers to a technique for converting voice information input by a user into text information.

[0817] "Means of inputting text data into a generative artificial intelligence for analysis" refers to the function of inputting converted text information into an AI and analyzing its content.

[0818] The "means for evaluating learning content and providing feedback" is a system that evaluates learning content based on the content analyzed by AI and provides users with feedback on areas for improvement and their level of understanding.

[0819] The "means for automatically generating review tests" is a mechanism for automatically generating test questions for users to review based on the provided feedback.

[0820] "Means for evaluating test results and recommending next study content" is a function that evaluates the results of a test taken by the user and recommends the next study content based on the results.

[0821] The "means for updating the study plan according to the user's progress" is a system that dynamically updates the set study plan according to the user's progress in learning.

[0822] "Means for generating test questions based on a specified learning topic using a smart device" refers to technology that uses a mobile or wearable device to create test questions related to a learning topic.

[0823] "Means for analyzing the content explained by a user through voice when generating feedback to the user using generative artificial intelligence" refers to a system in which AI analyzes the learning content explained through voice and generates feedback based on the results.

[0824] This invention is a system that allows users to study efficiently and effectively. The system provides functions such as setting study plans and goals, explaining study content through voice input, analyzing text and providing feedback using generative AI, automatically creating review tests, and recommending next study content.

[0825] Hardware and software used

[0826] Hardware: Smart devices (smartphones, tablets, smart glasses, etc.), servers.

[0827] Software: Speech recognition libraries (e.g. speech_recognition), generative AI libraries (e.g. Huggingface's transformers library).

[0828] Specific functions of the system

[0829] 1. Study plan and goal setting

[0830] Users use the device to set their study plans and goals by entering their goals and selecting relevant study topics through an on-screen interface.

[0831] Example prompt sentence:

[0832] set_learning_plan(goal="Pass the qualification exam", topics=["Grammar", "Reading"])

[0833] 2. Explanation of learning content by voice input

[0834] As the user progresses through the learning process, they explain what they have learned by voice into the device, which uses speech recognition technology to convert this speech into text data.

[0835] Example prompt sentence:

[0836] This week we learned about relative pronouns

[0837] 3. Analysis and feedback by generative AI

[0838] The converted text data is sent to a server where it is analyzed by a generative AI model, which then generates feedback on the user's level of understanding and key points based on the analysis results.

[0839] Example prompt sentence:

[0840] Analyze the following and provide feedback:This week we learned about relative pronouns.

[0841] 4. Automatic creation of review tests

[0842] Based on the feedback, a review test is generated, which corresponds to the learning topic and can be taken on a smart device.

[0843] Example prompt sentence:

[0844] generate_test(["relative pronoun", "subjunctive mood"])

[0845] 5. Evaluation of test results and recommendations for next study

[0846] The test results taken by the user are evaluated on the server and the next study content is recommended, allowing the user to continuously update their optimal study plan according to their own progress.

[0847] Specific examples

[0848] Setting up a study plan

[0849] When User A logs in to the system for the first time, the terminal displays "Please set your study plan and goals." User A enters "I'm aiming for a TOEIC score of 900," and the server receives the information and stores it in the database. The server then automatically generates an appropriate study plan and displays it to the user.

[0850] Voice input explained

[0851] After learning English grammar, User A explains to his / her device, "This week we learned about relative pronouns." The speech is converted into text and sent to the server. The generation AI analyzes the text and generates feedback saying, "You need to relearn the difference between the nominative and possessive case." The server sends this feedback to the device, and User A confirms it again.

[0852] Automatic creation of review tests

[0853] The server generates a test on relative pronouns and sends it to the device. User A takes the test and submits their answers, and the server evaluates the results. The evaluation result is displayed on the device along with feedback such as "You need to review the use of possessives a little more."

[0854] This system allows users to study efficiently and effectively, and provides appropriate feedback and review tests.

[0855] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0856] Step 1: Setting a study plan and goals

[0857] Input: The user inputs the "goal" and "learning topic" using the terminal.

[0858] Specific operation: The terminal displays a user interface, and the user inputs a goal (e.g., TOEIC score of 900 points) and a learning topic (e.g., listening and reading).

[0859] Output: The server receives these inputs and stores them in a database.

[0860] Data processing: The server organizes the input data and formats it as initial information for automatically generating a learning plan.

[0861] Step 2: Explain the learning content by voice input

[0862] Input: The user speaks what they learned into the device

[0863] Specific operation: When the user speaks a description, the device activates a speech recognition module (e.g., speech_recognition) and converts the speech into text data.

[0864] Output: Text data is sent to the server.

[0865] Data processing: The device converts the voice data into text data and formats it appropriately before sending it to the server.

[0866] Step 3: Analysis and feedback by generative AI

[0867] Input: Text data is sent to the server

[0868] How it works: The server inputs text data into a generative AI model (e.g., Huggingface's transformers), analyzes the content, and generates feedback based on the analysis results.

[0869] Output: The generated feedback is sent to the user terminal.

[0870] Data processing: The server converts the text data into a format suitable for the generative AI model, analyzes the analysis results, and formats them as feedback.

[0871] Step 4: Automatically create review tests

[0872] Input: Information based on generated feedback and learning topics

[0873] Specific operation: The server automatically generates review test questions based on the results of the generative AI model.

[0874] Output: The generated test questions are sent to the user's terminal.

[0875] Data processing: The server combines data to generate test questions based on the learning topic and feedback, and formats it into question format.

[0876] Step 5: Evaluate test results and recommend next study content

[0877] Input: The user takes the review test and enters the answers into the device.

[0878] Specific operation: The device sends the user's test answers to the server, which evaluates the results and recommends the next learning content based on the evaluation results.

[0879] Output: The evaluation results and the next learning content are sent to the user's terminal.

[0880] Data processing: The server analyzes the test answer data, generates evaluation results, and determines the recommended content for the next topic to learn.

[0881] This series of processes allows users to study efficiently and effectively, from setting up a study plan to reviewing the content they have learned, taking review tests, and receiving recommendations for the next study session.

[0882] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0883] ---

[0884] This invention is a system that allows users to study efficiently and effectively, and provides more personalized feedback by combining an emotion engine that recognizes the user's emotions. This system involves a series of processes in which the user sets a study plan and goals, explains what they have learned, and a generative AI analyzes the content and provides feedback, and automatically creates review tests as needed.

[0885] System configuration

[0886] The system includes the following major components:

[0887] 1. User device: A device used by the user as an interface to set up learning plans, input data by voice, take tests, and perform other operations.

[0888] 2. Server: As a central processing unit, it receives data from users and performs analysis, feedback generation, automatic test generation, etc.

[0889] 3. Generative AI: This runs on the server and includes an algorithm that analyzes the learning content sent by the user and generates evaluations and feedback.

[0890] 4. Emotion Engine: A system for recognizing emotions from the user's voice input and adjusting feedback based on the analysis results.

[0891] Specific functions of the system

[0892] 1. Study plan and goal setting

[0893] When a user logs in to the system, the terminal displays an interface for inputting their learning plan and goals. Through this interface, the user sets their own learning goals (e.g., obtaining a qualification, achieving a specific score, etc.) and creates a learning plan based on them. The server receives this information and stores it in a database.

[0894] 2. Explaining what has been learned and growing the generative AI

[0895] As the user progresses through their learning, the device prompts them to "explain what they learned." The user then explains what they learned aloud, and the device converts that speech into text data. The converted text data is sent to a server, where the AI ​​analyzes the content and generates an analysis result.

[0896] 3. Emotion Recognition and Analysis Using an Emotion Engine

[0897] The emotion engine recognizes the user's emotional state from their voice input. The emotion engine's analysis results are provided to the generative AI, which then adjusts the feedback taking into account the user's emotional state. For example, if the user is feeling anxious, the feedback will be provided in a gentler tone.

[0898] 4. Assessment and feedback of what you learned

[0899] The generative AI evaluates the user's explanation and creates feedback on any errors or insufficient understanding. Taking into account the results of the emotion analysis by the emotion engine, the server compiles the evaluation results and feedback and sends them to the device. Through this feedback, the user can check their level of understanding and make any necessary corrections.

[0900] 5. Automatic creation of review tests

[0901] The AI ​​automatically generates review tests based on the learning content with the highest confidence. The server customizes the tests, setting the difficulty and content according to the user's progress. The tests are sent to the device, and the user can take them to solidify the content they have learned.

[0902] 6. Evaluation of test results and recommendations for next study

[0903] The user takes the test and sends their answers to the server. The server evaluates the test results and stores the scores in a database. Based on the evaluation results, the server generates recommendations for what to study next and what points need to be re-studyed, and sends this to the device.

[0904] Specific examples

[0905] Setting up a study plan

[0906] When User B logs in to the system for the first time, the terminal displays "Please set your study plan and goals." User B enters "My goal is a TOEIC score of 900," and the server receives the information and stores it in the database. The server then automatically generates an appropriate study plan and displays it to the user.

[0907] Explanation of what was learned

[0908] After learning English grammar, User B explains to his device, "This week we learned about relative pronouns." The device converts the speech into text and sends it to the server. The generation AI analyzes the text and generates feedback saying, "You need to relearn the difference between the nominative and possessive." The emotion engine recognizes from User B's voice that "you seem impatient," and provides the feedback in a softer tone.

[0909] Automatic creation of review tests

[0910] The server generates a test on relative pronouns and sends it to the device. User B takes the test and submits their answers, and the server evaluates the results. The evaluation result is displayed on the device along with feedback such as "You need to review the use of possessives a little more."

[0911] ---

[0912] As described above, the present invention is a system that provides a series of functions to efficiently and effectively advance the user's learning process. Through analysis and feedback by generative AI, emotion recognition and adjustment by an emotion engine, and automatic creation of review tests, the system can deepen the user's understanding and solidify their knowledge.

[0913] The processing flow will be explained below.

[0914] ---

[0915] Step 1:

[0916] A user logs in to the system.

[0917] The terminal displays the login interface.

[0918] The user enters their login information and goes through the authentication process.

[0919] Step 2:

[0920] A learning plan and goal setting screen is displayed to the user whose device has been authenticated.

[0921] The user inputs their study plan and goals (e.g., aiming for a TOEIC score of 900).

[0922] The terminal transmits the input information to the server.

[0923] Step 3:

[0924] The server stores the user's learning plan and goals in a database.

[0925] The server generates an initial learning plan and schedule based on the received information.

[0926] The server sends the generated learning plan to the terminal.

[0927] Step 4:

[0928] The user progresses through the learning and is ready to explain what they learned.

[0929] The device prompts the user to "Describe what you learned."

[0930] Audibly describe what the user learned.

[0931] Step 5:

[0932] The terminal converts the user's voice input into text data.

[0933] The terminal transmits the converted text data to the server.

[0934] Step 6:

[0935] The server sends the text data to the emotion engine.

[0936] The emotion engine recognizes the user's emotional state from their voice.

[0937] The emotion engine sends the emotion analysis results to the server.

[0938] Step 7:

[0939] The server inputs the text data into the generation AI and begins analysis.

[0940] The generative AI evaluates the learning content based on text data and sentiment analysis results.

[0941] Step 8:

[0942] The generative AI analyzes the user's explanation and generates evaluation results and feedback.

[0943] The generative AI takes into account the user's emotional state and adjusts the tone and content of the feedback.

[0944] The server formats the generated evaluation results and feedback and sends them to the device.

[0945] Step 9:

[0946] The device displays the feedback to the user.

[0947] The user reviews the feedback and makes any necessary corrections.

[0948] Step 10:

[0949] The server instructs the AI ​​to generate review tests based on the feedback.

[0950] Generative AI determines the content of the review test and automatically creates the test.

[0951] Step 11:

[0952] The server customizes the generated tests and configures them as the user progresses.

[0953] The server sends the test to the device.

[0954] Step 12:

[0955] The device displays a test notification to the user, prompting them to take the test.

[0956] The user performs the test on the device.

[0957] The user enters the answers to the test and the terminal sends the answers to the server.

[0958] Step 13:

[0959] The server evaluates the test results and stores the scores in a database.

[0960] The server generates additional feedback based on the test results.

[0961] Step 14:

[0962] The device displays additional feedback to the user, prompting them to relearn.

[0963] The user re-learns and explains the new learning content to the terminal.

[0964] Step 15:

[0965] The server periodically evaluates the user's learning progress.

[0966] The server automatically updates the learning plan based on progress.

[0967] The server sends recommendations to the device on what to learn next and how to learn it.

[0968] Step 16:

[0969] The device displays recommended information to the user, supporting continuous learning.

[0970] ---

[0971] The above is the specific processing flow of the system, which effectively supports the user's learning process and aims to solidify knowledge through evaluation and feedback by the generative AI, as well as emotion recognition and adjustment by the emotion engine.

[0972] Example 2

[0973] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0974] Conventional learning support systems provide uniform feedback without considering the user's emotional state, making it difficult to provide appropriate instruction tailored to individual needs. Furthermore, evaluation of the user's learning content and recommendations for the next lesson are often done manually, resulting in reduced learning efficiency. Furthermore, in many cases, only standard test questions are provided, resulting in a lack of customized tests tailored to the user's level of understanding. A system that can solve these issues and provide a more personalized learning experience is needed.

[0975] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0976] In this invention, the server includes means for a user to set a study plan and goals, means for audibly explaining what the user has learned, means for converting the speech into text data, means for inputting the text data into a generative artificial intelligence model and analyzing it, means for evaluating the study content and providing feedback based on the analysis results, means for recognizing the user's emotional state, means for adjusting the feedback taking the user's emotional state into account, means for automatically creating a review test based on the feedback, means for evaluating the test results and recommending the next study content, and means for updating the study plan according to the user's progress. This makes it possible to provide feedback that takes the user's emotional state into account and tests that are customized according to the user's level of understanding.

[0977] A "user" is an individual or group who uses the system to set a learning plan and goals and progress through the learning content.

[0978] "Study planning" is the process of setting specific study content and schedules to be achieved within a specific period based on the study goals that the user wants to achieve.

[0979] A "goal" is a specific outcome or performance that a user wants to achieve through the system.

[0980] "Audio explanation means" is a method by which a user verbally conveys what they have learned to the system.

[0981] "Means for converting into text data" refers to a technology for converting a user's voice input into text format data.

[0982] A "generative artificial intelligence model" is an artificial intelligence algorithm that analyzes data entered by the user and evaluates learning content and generates feedback.

[0983] "Means of analysis" refers to the process of using a generative artificial intelligence model to understand and evaluate the learning content explained by the user.

[0984] "Means for providing feedback" refers to a method for informing users of their learning progress and areas for improvement based on the analysis results.

[0985] "Means for recognizing emotional states" refers to technology for identifying emotions from a user's voice or text data and detecting changes in the user's emotions.

[0986] "Feedback tailoring" is the process of optimizing the content and tone of the feedback provided, taking into account the user's emotional state.

[0987] The "means for automatically creating review tests" is a technology that automatically generates review test questions based on the user's learning content and level of understanding.

[0988] "Means for evaluating test results" refers to the process of analyzing the results of the test taken by the user and evaluating their performance.

[0989] "Means for recommending next learning content" is a process that recommends the next learning content or points that need to be re-learned based on test results and the user's progress.

[0990] The "means for updating the study plan" is a method for appropriately modifying an existing study plan in accordance with the user's progress and updating it to match the latest study goals.

[0991] The present invention provides a learning support system that helps users study efficiently and effectively, and in particular has a feedback function that takes into account the user's emotional state. This system is composed of the following main hardware and software components:

[0992] Hardware and software used

[0993] 1. User Device

[0994] A device that a user uses as an interface to set up a study plan, input data by voice, take tests, receive feedback, etc. Examples include personal computers, smartphones, and tablets. These devices use voice recognition software (e.g., Google Cloud Speech-to-Text) to convert the user's speech into text.

[0995] 2. Server

[0996] As a central processing unit, it receives data from users and performs analysis, creates feedback, automatically generates tests, etc. On the server, generative AI models (e.g., GPT-4) and emotion engines (e.g., Affectiva) run to analyze the user's learning content and emotional state.

[0997] 3. Generative AI Models

[0998] The generative AI model, which runs on a server and includes algorithms that analyze the text data submitted by users, evaluate their learning, generate feedback, and create review tests, is known as a large-scale language model.

[0999] 4. Emotion Engine

[1000] The emotion engine is a system for recognizing the emotional state of a user from their voice input and providing the analysis results to a generative AI model, which then adjusts the feedback according to the user's emotions.

[1001] Specific processing flow and functions

[1002] Study plan and goal setting

[1003] When a user logs in to the system, the terminal displays the message "Please set your study plan and goals." The user enters their study goal (e.g., get a high score on a test) and the terminal sends this information to the server, which receives it and stores it in a database.

[1004] Specific behavior:

[1005] The user inputs "I want to make a study plan with the goal of passing a qualification exam" into the terminal, and the server receives the information and displays a confirmation message to the user.

[1006] Explanation of what was learned

[1007] After the user has progressed through the learning process, the device will prompt them to "explain what you learned." The user will explain aloud, and the device will use speech recognition software to convert the speech into text.

[1008] Specific behavior:

[1009] When a user speaks to the device, "I learned about differential and integral calculus this weekend," the device uses voice recognition software to convert this into text data, "I learned about differential and integral calculus this weekend," and sends it to the server.

[1010] Analysis of learning content and generation of feedback

[1011] The generative AI model on the server analyzes the text data sent by the user and evaluates the learning content. The emotion engine recognizes emotions from the user's voice and provides the results to the generative AI model. The generative AI model adjusts the feedback taking the emotional state into account.

[1012] Specific behavior:

[1013] If the server assesses that the learning content is fully understood and the emotion engine recognizes that the user is showing signs of impatience during the explanation, the generative AI model will provide gentle feedback such as, "You have a solid understanding of the basics of calculus. Let's try some applied problems next."

[1014] Automatic creation of review tests

[1015] Based on the analysis results, the generative AI model automatically generates review tests to improve the user's understanding. The server selects appropriate test questions and sends them to the device.

[1016] Specific behavior:

[1017] The server generates five questions on the basic concepts of calculus, which the terminal displays as a test to the user. The user takes the test and sends the answers to the server.

[1018] Test result evaluation and study recommendations

[1019] The server evaluates the test results and recommends the next study content to the user based on the results, allowing the user to develop an efficient study plan that is in line with their own learning progress.

[1020] Specific behavior:

[1021] After the user completes the test and the server evaluates it, a recommendation is displayed on the device saying, "We recommend that you review the basic concepts again before moving on to the application questions."

[1022] Prompt Sentence Examples

[1023] "Set a study plan and goals."

[1024] "Please explain what you learned."

[1025] "Please explain what we learned this week in audio."

[1026] "Please begin the review test."

[1027] The above is a specific embodiment for carrying out the invention. By using this system, users can receive personalized feedback according to their learning progress and promote effective learning.

[1028] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1029] Step 1:

[1030] A user logs into the system

[1031] Input: Username and Password

[1032] Data processing: The device sends the entered information to the server. The server performs user authentication and, if successful, displays the home screen.

[1033] Output: The user can access the home screen.

[1034] Specific behavior:

[1035] The user enters their username and password into the device and presses the send button. The device sends this information to the server, which then authenticates them and displays the home screen on the device.

[1036] Step 2:

[1037] Your device will display the study plan and goal setting screen.

[1038] Input: Learning objectives and plan content

[1039] Data processing: The terminal receives the user's input and sends it to the server, which stores the information in a database.

[1040] Output: The configured learning plan is saved with a confirmation message.

[1041] Specific behavior:

[1042] The user enters "I'm aiming for a TOEIC score of 900" and presses "Confirm." The device sends this information to the server, which saves it in the database. A message indicating that saving has been completed is displayed on the device.

[1043] Step 3:

[1044] The user explains the learning content aloud

[1045] Input: User voice input

[1046] Data processing: Use speech recognition software (e.g., Google Cloud Speech-to-Text) to convert voice input into text data. The text is sent to the server.

[1047] Output: The learning content in text format is saved on the server.

[1048] Specific behavior:

[1049] The user speaks to the device, saying, "This weekend I learned about calculus." The device converts the speech to text using Google Cloud Speech-to-Text and sends it to the server. The text data is stored on the server.

[1050] Step 4:

[1051] The server analyzes the learning content

[1052] Input: Text data (user learning content)

[1053] Data processing: A generative AI model (e.g., GPT-4) analyzes the text data and evaluates what it has learned.

[1054] Output: Analysis results (evaluation details)

[1055] Specific behavior:

[1056] The server generates an analysis result that says, "The basics of differential and integral calculus are understood, but a review is needed before moving on to applied problems."

[1057] Step 5:

[1058] Emotion engine recognizes emotional states from speech

[1059] Input: Audio data

[1060] Data processing: An emotion engine (e.g., Affectiva) analyzes the audio data and recognizes the emotional state.

[1061] Output: Emotional state data

[1062] Specific behavior:

[1063] The user sends a voice that indicates impatience while giving an explanation to the server, and the emotion engine recognizes the "impatience."

[1064] Step 6:

[1065] Server provides tailored feedback

[1066] Input: Analysis results and emotional state

[1067] Data processing: Generative AI takes into account your emotional state and adjusts the content and tone of your feedback.

[1068] Output: Regulated Feedback

[1069] Specific behavior:

[1070] The server generates feedback in a gentle tone, saying, "You have a solid understanding of the basics of differential and integral calculus. Now let's try some applied problems." The feedback is displayed on the terminal.

[1071] Step 7:

[1072] Automatically create review quizzes

[1073] Input: Analysis results

[1074] Data processing: Generative AI automatically generates review tests based on the analysis results.

[1075] Output: A set of review test questions

[1076] Specific behavior:

[1077] The server generates "five problems on the basic concepts of differential and integral calculus" and sends them to the terminal as a test.

[1078] Step 8:

[1079] User takes the test and submits the results

[1080] Input: User's test answers

[1081] Data processing: The terminal sends the user's answers to the server, which evaluates the answers.

[1082] Output: Evaluation results

[1083] Specific behavior:

[1084] The user answers the questions and submits the results. The server evaluates the test results and generates feedback such as "Area of ​​improvement: You need to use possessives and relative pronouns appropriately."

[1085] Step 9:

[1086] Recommend what to study next

[1087] Input: Evaluation result

[1088] Data processing: The server generates the next learning content based on the evaluation results and makes recommendations.

[1089] Output: Recommended learning content

[1090] Specific behavior:

[1091] The server generates a message saying, "Next, we recommend that you learn about the use of possessives and relative pronouns," and displays it on the terminal.

[1092] These are the main processing steps of the system, which allows users to receive personalized feedback based on their learning progress and effectively advance their learning.

[1093] (Application example 2)

[1094] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1095] Previous learning support systems relied on standard methods to evaluate and provide feedback on users' learning content, but were unable to consider the emotional state of each individual user. As a result, learning effectiveness could decline if users became stressed or lost motivation. Furthermore, the customization of review tests and study plans did not reflect the user's psychological state, resulting in a lack of personalized learning support.

[1096] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1097] In this invention, the server includes: means for a user to set a study plan and goals; means for audibly explaining what the user has learned; means for converting the voice into text data; means for inputting the text data into a generative artificial intelligence and analyzing it; means including an emotion engine for recognizing the emotional state of the user from the voice input; means for evaluating the study content and providing feedback based on the analysis results and the emotional state; means for automatically creating review tests based on the feedback; means for evaluating the test results and recommending next study content; and means for updating the study plan according to the user's progress. This makes it possible to create personalized feedback and tests that take the user's emotional state into consideration, which is expected to improve learning effectiveness.

[1098] "Users"

[1099] refers to an individual who uses a learning system to carry out learning activities.

[1100] "Study Plan"

[1101] A learning plan is a plan that describes specific learning content and schedules to achieve the goals set by the user.

[1102] "Voice input"

[1103] is a means for users to provide information to the system by dictating what they have learned.

[1104] "Text data"

[1105] It is data that converts voice input into text format.

[1106] "Generative Artificial Intelligence"

[1107] is a system that includes algorithms and programs for analyzing text data converted from voice input.

[1108] "Emotion Engine"

[1109] is a system for recognizing and analyzing a user's emotional state from their voice input.

[1110] "feedback"

[1111] This is information that includes evaluations and advice on what the user has learned.

[1112] "Review test"

[1113] These are tests that the system automatically creates to help users confirm and solidify what they have learned.

[1114] "Next lesson content"

[1115] This refers to the content or topic that the user should study next.

[1116] "Recommendation"

[1117] This refers to the learning content and methods that the system suggests based on the user's learning progress and test results.

[1118] "progress"

[1119] This refers to the user's level of achievement and progress as they progress through their studies.

[1120] "update"

[1121] This refers to modifying and updating a user's study plan based on the latest information and progress.

[1122] This invention provides a system for enabling users to study efficiently and effectively. The system includes means for setting a user's study plan and goals and for explaining the user's learned content via voice. A server converts the user's voice into text data, which is then input into a generative AI model for analysis. The system also uses an emotion engine to recognize the user's emotional state from the user's voice input and adjusts feedback based on this. The system automatically creates review tests based on the feedback, evaluates the test results, and recommends next study content. The system also includes means for updating the study plan according to the user's progress.

[1123] System configuration

[1124] The system includes the following major components:

[1125] 1. User device: A device used by the user as an interface to set up learning plans, input voice, take tests, etc. User devices can be VR headsets, smartphones, PCs, etc.

[1126] 2. Server: As a central processing unit, it receives data from users and performs analysis, feedback generation, automatic test generation, etc.

[1127] 3. Generative AI model: Runs on the server and includes an algorithm that analyzes the learning content sent by the user and generates evaluation and feedback.

[1128] 4. Emotion Engine: A system for recognizing emotions from the user's voice input and adjusting feedback based on the analysis results.

[1129] Specific functions of the system

[1130] 1. Study plan and goal setting

[1131] When a user logs in to the system, the terminal displays an interface for inputting their learning plan and goals. Through this interface, the user sets their own learning goals (e.g., obtaining a qualification, achieving a specific score, etc.) and creates a learning plan based on them. The server receives this information and stores it in a database.

[1132] 2. Explaining what has been learned and analyzing the generative AI model

[1133] After the user has progressed through their learning, the device prompts them to "explain what they have learned." The user then explains what they have learned by voice, and the device converts that speech into text data. The converted text data is sent to a server, where the generative AI model analyzes the content and generates analysis results.

[1134] 3. Emotion Recognition and Analysis Using an Emotion Engine

[1135] The emotion engine recognizes the user's emotional state from their voice input. The emotion engine's analysis results are fed into a generative AI model, which then adjusts the feedback based on the user's emotional state. For example, if the user is feeling anxious, the feedback will be delivered in a gentler tone.

[1136] 4. Assessment and feedback of what you learned

[1137] The generative AI model evaluates the user's explanation and generates feedback on any errors or insufficient understanding. Taking into account the emotion analysis results of the emotion engine, the server compiles the evaluation results and feedback and sends them to the device. Through this feedback, the user can check their own level of understanding and make any necessary corrections.

[1138] 5. Automatic creation of review tests

[1139] The generative AI model automatically generates review tests based on the learning content with the highest confidence. The server customizes these tests, setting the difficulty and content according to the user's progress. The tests are sent to the device, and the user can take them to solidify what they have learned.

[1140] 6. Evaluation of test results and recommendations for next study

[1141] The user takes the test and sends their answers to the server. The server evaluates the test results and stores the scores in a database. Based on the evaluation results, the server generates recommendations for what to study next and what points need to be re-studyed, and sends this to the device.

[1142] Specific examples

[1143] Setting up a study plan

[1144] When a user logs in to the system for the first time, the terminal displays, "Please set your study plan and goals." The user enters, "My goal is a TOEIC score of 900," and the server receives the information and stores it in the database. The server then automatically generates an appropriate study plan and displays it to the user.

[1145] Explanation of what was learned

[1146] After learning English grammar, a user speaks to their device, saying, "This week we learned about relative pronouns." The device converts the speech into text and sends it to the server. The generative AI model analyzes the text and generates feedback saying, "You need to relearn the difference between nominative and possessive." The emotion engine recognizes from the user's voice that "you seem impatient" and provides the feedback in a softer tone. An example of feedback would be, "Let's review the difference between nominative and possessive again."

[1147] Automatic creation of review tests

[1148] The server generates a test on relative pronouns and sends it to the device. The user takes the test, submits their answers, and the server evaluates the results. The evaluation result is displayed on the device along with feedback such as "You need to review the use of possessives a little more."

[1149] Prompt Sentence Examples

[1150] "To deepen your understanding of neural networks, let's next tackle some practical problems."

[1151] As described above, this invention is a system that utilizes an emotion engine and a generative AI model to support the user's learning process and provide a more effective learning experience.

[1152] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1153] Step 1:

[1154] The device displays an interface for the user to set up a learning plan and goals. The user enters their learning goals (e.g., obtaining a qualification, achieving a specific score, etc.) on this screen, and the device sends the information to the server. The server stores the received information in a database.

[1155] Input: User-entered learning plans and goals

[1156] Output: Learning plans and goals received and stored by the server

[1157] Specific operation: When a user enters "My goal is a TOEIC score of 900" and clicks the data submission button, the data is sent from the device to the server, which then writes and saves this data in the database.

[1158] Step 2:

[1159] After the user has progressed through their studies, the device prompts them to "explain what they have learned." The user explains what they have learned by voice, and the device converts the voice into text data.

[1160] Input: User voice input

[1161] Output: Learning content converted into text data

[1162] What it does: A user says, "This week we learned about relative pronouns," and the system recognizes the speech and converts it into text.

[1163] Step 3:

[1164] The converted text data is sent from the device to a server and input into a generative AI model, which analyzes the content and generates an evaluation and feedback.

[1165] Input: Learning content converted into text data

[1166] Output: Analysis results, evaluation, and feedback from the generative AI model

[1167] Specific operation: The text data "I learned about relative pronouns" is sent to the generative AI model for analysis. The generative AI model generates feedback saying "You need to re-learn the difference between nominative and possessive cases."

[1168] Step 4:

[1169] The emotion engine recognizes the user's emotional state from their voice input, and the analysis results of the emotion engine are provided to the generative AI model, which then adjusts the feedback taking into account the user's emotional state.

[1170] Input: User's voice data

[1171] Output: User's emotional state (e.g., impatience)

[1172] Specific operation: The emotion engine recognizes "impatience" from the user's voice and provides the analysis results to the generative AI model. The generative AI model then adjusts the voice to "provide feedback in a gentler tone because the user is impatient."

[1173] Step 5:

[1174] The server sends the adjusted feedback to the device, which then displays the feedback to the user, who then checks the feedback and determines their level of understanding.

[1175] Input: Calibrated feedback

[1176] Output: Feedback that is displayed to the user

[1177] What happens: The server sends the adjusted feedback "Let's review the difference between nominative and possessive again" to the device, which then displays it to the user.

[1178] Step 6:

[1179] The generative AI model automatically generates review tests based on the user's learning, and the server customizes the tests. The tests are then sent to the user's device, where they are then taken.

[1180] Input: tailored feedback, what the user has learned

[1181] Output: Auto-generated review test

[1182] Specific operation: A test on relative pronouns is generated, adjusted to an appropriate level of difficulty by the server, and sent to the device. The user then takes the test.

[1183] Step 7:

[1184] The user takes the test and sends the answers from the device to the server. The server evaluates the results and stores them in a database. It then generates recommendations for what to study next and what points need to be re-studyed, and sends them to the device.

[1185] Input: Test answer data

[1186] Output: Evaluation results and next training recommendation

[1187] What it does: The user takes the test and submits their answers, and the server evaluates the results. The evaluation result is displayed on the device along with feedback such as "You need to review the use of possessives a little more."

[1188] The above processing steps enable the system to create personalized feedback and tests that take into account the user's emotional state, which is expected to improve learning effectiveness.

[1189] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1190] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1191] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1192] [Third embodiment]

[1193] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1194] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1195] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1196] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1197] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1198] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1199] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1200] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1201] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1202] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1203] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1204] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1205] ---

[1206] The present invention is a system that allows users to study efficiently and effectively. This system involves a series of processes in which a user sets a study plan and goals, explains what they have learned, and a generative AI analyzes the content, provides feedback, and automatically creates review tests as needed.

[1207] System configuration

[1208] The system includes the following major components:

[1209] 1. User device: A device used by the user as an interface to set up learning plans, input data by voice, take tests, and perform other operations.

[1210] 2. Server: As a central processing unit, it receives data from users and performs analysis, feedback generation, automatic test generation, etc.

[1211] 3. Generative AI: This runs on the server and includes an algorithm that analyzes the learning content sent by the user and generates evaluations and feedback.

[1212] Specific functions of the system

[1213] 1. Study plan and goal setting

[1214] When a user logs in to the system, the terminal displays an interface for inputting their learning plan and goals. Through this interface, the user sets their own learning goals (e.g., obtaining a qualification, achieving a specific score, etc.) and creates a learning plan based on them. The server receives this information and stores it in a database.

[1215] 2. Explaining what has been learned and growing the generative AI

[1216] As the user progresses through their learning, the device prompts them to "explain what they learned." The user then explains what they learned aloud, and the device converts that speech into text data. The converted text data is sent to a server, where the AI ​​analyzes the content and generates an analysis result.

[1217] 3. Evaluation and feedback of what you have learned

[1218] The generation AI evaluates the user's explanation and generates feedback on any errors or insufficient understanding. The server compiles the evaluation results and feedback and sends them to the device. Through this feedback, the user can check their own understanding and make any necessary corrections.

[1219] 4. Automatic creation of review tests

[1220] The AI ​​automatically generates review tests based on the learning content with the highest confidence. The server customizes the tests, setting the difficulty and content according to the user's progress. The tests are sent to the device, and the user can take them to solidify the content they have learned.

[1221] 5. Evaluation of test results and recommendations for next study

[1222] The user takes the test and sends their answers to the server. The server evaluates the test results and stores the scores in a database. Based on the evaluation results, the server generates recommendations for what to study next and what points need to be re-studyed, and sends this to the device.

[1223] Specific examples

[1224] Setting up a study plan

[1225] When User A logs in to the system for the first time, the terminal displays "Please set your study plan and goals." User A enters "I'm aiming for a TOEIC score of 900," and the server receives the information and stores it in the database. The server then automatically generates an appropriate study plan and displays it to the user.

[1226] Explanation of what was learned

[1227] After learning English grammar, User A explains aloud to his / her device, "This week we learned about relative pronouns." The device converts the speech into text and sends it to the server. The generation AI analyzes the text and generates feedback saying, "You need to relearn the difference between the nominative and possessive case." The server sends this feedback to the device, and User A confirms it again.

[1228] Automatic creation of review tests

[1229] The server generates a test on relative pronouns and sends it to the device. User A takes the test and submits their answers, and the server evaluates the results. The evaluation result is displayed on the device along with feedback such as "You need to review the use of possessives a little more."

[1230] ---

[1231] As described above, the present invention is a system that provides a series of functions to efficiently and effectively advance the user's learning process. Analysis and feedback by generative AI, as well as the automatic creation of review tests, can deepen the user's understanding and help them solidify their knowledge.

[1232] The processing flow will be explained below.

[1233] ---

[1234] Step 1:

[1235] A user logs in to the system.

[1236] The terminal displays the login interface.

[1237] The user enters their login information and goes through the authentication process.

[1238] Step 2:

[1239] A learning plan and goal setting screen is displayed to the user whose device has been authenticated.

[1240] The user inputs their study plan and goals (e.g., aiming for a TOEIC score of 900).

[1241] The terminal transmits the input information to the server.

[1242] Step 3:

[1243] The server stores the user's learning plan and goals in a database.

[1244] The server generates an initial learning plan and schedule based on the received information.

[1245] The server sends the generated learning plan to the terminal.

[1246] Step 4:

[1247] The user progresses through the learning and is ready to explain what they learned.

[1248] The device prompts the user to "Describe what you learned."

[1249] Audibly describe what the user learned.

[1250] Step 5:

[1251] The terminal converts the user's voice input into text data.

[1252] The terminal transmits the converted text data to the server.

[1253] Step 6:

[1254] The server inputs the text data into the generation AI and begins analysis.

[1255] The generative AI evaluates the learning content based on text data.

[1256] Step 7:

[1257] The generative AI analyzes the user's explanation and generates evaluation results and feedback.

[1258] The server formats the generated evaluation results and feedback and sends them to the device.

[1259] Step 8:

[1260] The device displays the feedback to the user.

[1261] The user reviews the feedback and makes any necessary corrections.

[1262] Step 9:

[1263] The server instructs the AI ​​to generate review tests based on the feedback.

[1264] Generative AI determines the content of the review test and automatically creates the test.

[1265] Step 10:

[1266] The server customizes the generated tests and configures them as the user progresses.

[1267] The server sends the test to the device.

[1268] Step 11:

[1269] The device displays a test notification to the user, prompting them to take the test.

[1270] The user performs the test on the device.

[1271] The user enters the answers to the test and the terminal sends the answers to the server.

[1272] Step 12:

[1273] The server evaluates the test results and stores the scores in a database.

[1274] The server generates additional feedback based on the test results.

[1275] Step 13:

[1276] The device displays additional feedback to the user, prompting them to relearn.

[1277] The user re-learns and explains the new learning content to the terminal.

[1278] Step 14:

[1279] The server periodically evaluates the user's learning progress.

[1280] The server automatically updates the learning plan based on progress.

[1281] The server sends recommendations to the device on what to learn next and how to learn it.

[1282] Step 15:

[1283] The device displays recommended information to the user, supporting continuous learning.

[1284] ---

[1285] The above is the specific processing flow of the system, which effectively supports the user's learning process and aims to ensure that knowledge is continuously retained through evaluation and feedback by the generative AI.

[1286] Example 1

[1287] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1288] Conventional learning support systems have had difficulty providing feedback and review content tailored to each user's individual learning progress and level of understanding. This has led to problems such as reduced learning efficiency and effectiveness. Furthermore, it has been difficult to evaluate the specific level of understanding of the content a user has learned and to appropriately recommend the next learning content.

[1289] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1290] In this invention, the server includes means for a user to set a study plan and goals, means for audibly explaining what the user has learned, means for converting the speech into text data, means for inputting the text data into a generative artificial intelligence and analyzing it, means for evaluating the study content and providing feedback based on the analysis results, means for automatically creating review tests based on the feedback, means for evaluating the test results and recommending next study content, means for updating the study plan according to the user's progress, means for user login authentication, means for storing the user's study content and results in a database, and means for customizing the content of review tests based on the user's progress. This makes it possible to improve the efficiency and effectiveness of users' learning and realize optimal feedback and recommendations for next study content according to individual learning progress.

[1291] A "user" is a person who uses the system to set up a learning plan and manage the progress of their learning.

[1292] A "server" is a central processing unit that receives data from users, performs analysis, feedback, and automatically generates tests, and provides these results to users.

[1293] A "terminal" is a device that a user uses as an interface, and is used to perform operations such as setting up a study plan, inputting voice data, and taking tests.

[1294] A "study plan" is a specific study schedule and procedure for achieving a goal set by a user.

[1295] A "goal" is a specific learning outcome or grade that a user aims to achieve.

[1296] "Audio" refers to the spoken words that a user makes to explain what they have learned.

[1297] "Text data" is voice data converted into text information, and is the input data for analysis by the generative AI model.

[1298] "Generative AI" includes algorithms and techniques for analyzing user-provided data and generating ratings and feedback.

[1299] "Analysis" is the process of evaluating content based on text data and identifying errors or areas of lack of understanding.

[1300] "Feedback" refers to evaluation information on areas for improvement and learning content provided to the user based on the analysis results.

[1301] A "review test" is a confirmation test created to help users solidify what they have learned.

[1302] "Recommendations" are information that suggests what content the user should study next based on their learning results.

[1303] "Progress" refers to the process or state in which a user advances their studies according to a study plan.

[1304] "Login authentication" is a procedure for verifying a user's identity when accessing a system.

[1305] A "database" is a storage device for saving the user's learning content and results.

[1306] "Customization" refers to the process of adjusting the content and difficulty level according to the user's progress and level of understanding.

[1307] The present invention is a system that allows users to study efficiently and effectively. This system involves a series of processes in which a user sets a study plan and goals, explains what they have learned, and a generative AI analyzes the content, provides feedback, and automatically creates review tests as needed.

[1308] System configuration

[1309] The system includes the following major components:

[1310] 1. User device: A device used by users as an interface to set up learning plans, input voice commands, take tests, etc. User devices include PCs, smartphones, tablets, etc.

[1311] 2. Server: A central processing unit that receives data from users, analyzes it, creates feedback, automatically generates tests, etc. The server is a high-performance computer, and the database can also be stored within the server.

[1312] 3. Generative AI: This includes algorithms that run on a server, analyze the learning content sent by the user, and generate evaluations and feedback. Generative AI is realized using natural language processing and machine learning techniques.

[1313] Study plan and goal setting

[1314] When a user logs in to the system, the terminal displays an interface for inputting their learning plan and goals. Through this interface, the user sets their own learning goals (e.g., obtaining a qualification, achieving a specific score, etc.) and creates a learning plan based on them. The server receives this information and stores it in a database.

[1315] Explanation of what was learned

[1316] As the user progresses through their learning, the device prompts them to "explain what they learned." The user then explains what they learned aloud, and the device converts that speech into text data. The converted text data is sent to a server, where the AI ​​analyzes the content and generates an analysis result.

[1317] Assessment and feedback of what you learned

[1318] The generative AI evaluates the user's explanation and generates feedback on any errors or insufficient understanding. The server compiles the evaluation results and feedback and sends them to the device. Through this feedback, the user can check their own understanding and make any necessary corrections.

[1319] Automatic creation of review tests

[1320] The AI ​​automatically generates review tests based on the learning content with the highest confidence. The server customizes these tests, setting the difficulty and content according to the user's progress. The tests are sent to the device, and the user can take them to solidify what they have learned.

[1321] Evaluation of test results and recommendations for next study content

[1322] The user takes the test and sends their answers to the server. The server evaluates the test results and stores the scores in a database. Based on the evaluation results, the server generates recommendations for what to study next and what points need to be re-studyed, and sends this information to the device.

[1323] Specific examples

[1324] Setting up a study plan

[1325] When User A logs in to the system for the first time, the terminal displays "Please set your study plan and goals." User A enters "I'm aiming for a TOEIC score of 900," and the server receives the information and stores it in the database. The server then automatically generates an appropriate study plan and displays it to the user.

[1326] Explanation of what was learned

[1327] After learning English grammar, User A explains aloud to his / her device, "This week we learned about relative pronouns." The device converts the speech into text and sends it to the server. The generation AI analyzes the text and generates feedback saying, "You need to relearn the difference between the nominative and possessive case." The server sends this feedback to the device, and User A confirms it again.

[1328] Automatic creation of review tests

[1329] The server generates a test on relative pronouns and sends it to the device. User A takes the test and submits their answers, and the server evaluates the results. The evaluation result is displayed on the device along with feedback such as "You need to review the use of possessives a little more."

[1330] Prompt Sentence Examples

[1331] "Please explain what you learned this week."

[1332] Please suggest what we should learn next.

[1333] "Evaluate the results of the test and provide feedback."

[1334] In this way, the system for implementing the invention provides a series of functions to efficiently and effectively advance the user's learning process. Analysis and feedback using a generative AI model, as well as the automatic creation of review tests, can deepen the user's understanding and help them solidify their knowledge.

[1335] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1336] Step 1: User Login

[1337] A user logs in to the system using a terminal. The terminal displays an interface for entering a user ID and password, and the user enters the information and presses the "Login" button. The server authenticates the received login information and retrieves the user's account information from the database. If authentication is successful, the server returns a login success status to the terminal and displays the main menu to the user.

[1338] Input: User ID, Password

[1339] Output: Login successful status, main menu displayed

[1340] Specific behavior:

[1341] 1. The device displays the login page.

[1342] 2. The user enters their ID and password and submits it.

[1343] 3. The server receives the login information and authenticates it with the database.

[1344] 4. When authentication is successful, the main menu is displayed on the device.

[1345] Step 2: Setting a study plan and goals

[1346] The device displays a prompt saying, "Please set your study plan and goals." The user then inputs their study goals and plan into the interface. For example, they can set a goal such as "Aim for a TOEIC score of 900." The server receives this information and stores it in a database. The server then automatically suggests study content related to the user's goals and displays them as a list on the device.

[1347] Input: Learning objectives, learning plan

[1348] Output: Saved learning plans, suggested learning content

[1349] Specific behavior:

[1350] 1. The device displays an interface that prompts the user to enter their learning plan and goals.

[1351] 2. The user enters and submits the learning objectives.

[1352] 3. The server stores this information in a database.

[1353] 4. The server suggests relevant learning content and displays it on the device.

[1354] Step 3: Explain what you learned

[1355] As the user progresses through their learning, the device prompts them to "explain what they learned." The user then explains what they learned by voice. For example, they might say, "This week we learned about relative pronouns." The device converts the voice data into text data and sends it to the server. The generative AI analyzes this text data and understands its content.

[1356] Input: Audio explanation of what you're learning

[1357] Output: Text data, analysis results

[1358] Specific behavior:

[1359] 1. The device prompts the user for voice input.

[1360] 2. Explain in audio what the user learned.

[1361] 3. The device converts the voice into text data.

[1362] 4. The text data is sent to the server and analyzed by the generating AI.

[1363] Step 4: Assessment and feedback on what was learned

[1364] The generation AI analyzes the text data and evaluates the content of the user's explanation. As a result of the analysis, it extracts any errors or parts of insufficient understanding and generates feedback based on this. For example, it may create feedback such as "You need to relearn the difference between the nominative and possessive case." The server compiles this feedback and sends it to the device, where it is displayed to the user.

[1365] Input: Analysis results of text data

[1366] Output: Feedback

[1367] Specific behavior:

[1368] 1. Generative AI analyzes text data.

[1369] 2. Detect errors or areas of lack of understanding.

[1370] 3. Feedback is generated and sent to the device by the server.

[1371] 4. The device displays feedback to the user.

[1372] Step 5: Automatically create review tests

[1373] The AI ​​automatically generates a review test based on the user's level of understanding. The server generates specific questions based on the user's learning content and assessment results. For example, it creates multiple-choice questions and fill-in-the-blank questions about relative pronouns. The test is sent to the device and prepared for the user to take.

[1374] Input: Learning content, evaluation results

[1375] Output: Review test

[1376] Specific behavior:

[1377] 1. Generative AI generates test questions based on the evaluation results.

[1378] 2. Customize the server-generated tests.

[1379] 3. Send the test to the device and display it to the user.

[1380] Step 6: Evaluate test results and recommend next study content

[1381] The user completes the test and sends the answers to the server via their device. The server evaluates the test results and stores the scores in a database. Based on the evaluation results, the generative AI recommends what to study next and what points need to be re-studyed. For example, specific advice such as "You need to review the use of possessives a little more" is displayed on the device.

[1382] Input: Test Answers

[1383] Output: Evaluation results, recommendations for next learning content

[1384] Specific behavior:

[1385] 1. A user takes a test and submits their answers.

[1386] 2. The server evaluates the test results and stores the scores in a database.

[1387] 3. The generative AI recommends the next learning content based on the evaluation results.

[1388] 4. The server sends the recommendation information to the device and displays it to the user.

[1389] (Application example 1)

[1390] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1391] Conventional learning support systems lack the functionality to provide detailed feedback and appropriate review tests to help users progress through their studies efficiently and effectively. They also lack a system that can flexibly update learning plans according to the user's progress, making them unsuitable tools for maximizing the effectiveness of learning. Furthermore, they lack sufficient accuracy and flexibility when it comes to explaining learning content through voice input and providing analysis and feedback using generative AI. These issues need to be resolved.

[1392] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1393] In this invention, the server includes: means for a user to set a study plan and goals; means for audibly explaining what the user has learned; means for converting the speech into text data; means for inputting the text data into a generative artificial intelligence for analysis; means for evaluating the study content and providing feedback based on the analysis results; means for automatically creating review tests based on the feedback; means for evaluating the test results and recommending next study content; means for updating the study plan according to the user's progress; means for generating test questions based on a specified study topic using a smart device; and means for analyzing the content explained by the user audibly when generating feedback to the user using the generative artificial intelligence. This enables the user to study efficiently and effectively and receive appropriate feedback and review tests.

[1394] "Means for setting study plans and goals" is a system that allows users to clarify the goals they want to achieve and set specific study steps and schedules based on those goals.

[1395] The "means for explaining learned content by voice" is a system that allows the user to explain what they have learned by voice input.

[1396] "Means for converting voice into text data" refers to a technique for converting voice information input by a user into text information.

[1397] "Means of inputting text data into a generative artificial intelligence for analysis" refers to the function of inputting converted text information into an AI and analyzing its content.

[1398] The "means for evaluating learning content and providing feedback" is a system that evaluates learning content based on the content analyzed by AI and provides users with feedback on areas for improvement and their level of understanding.

[1399] The "means for automatically generating review tests" is a mechanism for automatically generating test questions for users to review based on the provided feedback.

[1400] "Means for evaluating test results and recommending next study content" is a function that evaluates the results of a test taken by the user and recommends the next study content based on the results.

[1401] The "means for updating the study plan according to the user's progress" is a system that dynamically updates the set study plan according to the user's progress in learning.

[1402] "Means for generating test questions based on a specified learning topic using a smart device" refers to technology that uses a mobile or wearable device to create test questions related to a learning topic.

[1403] "Means for analyzing the content explained by a user through voice when generating feedback to the user using generative artificial intelligence" refers to a system in which AI analyzes the learning content explained through voice and generates feedback based on the results.

[1404] This invention is a system that allows users to study efficiently and effectively. The system provides functions such as setting study plans and goals, explaining study content through voice input, analyzing text and providing feedback using generative AI, automatically creating review tests, and recommending next study content.

[1405] Hardware and software used

[1406] Hardware: Smart devices (smartphones, tablets, smart glasses, etc.), servers.

[1407] Software: Speech recognition libraries (e.g. speech_recognition), generative AI libraries (e.g. Huggingface's transformers library).

[1408] Specific functions of the system

[1409] 1. Study plan and goal setting

[1410] Users use the device to set their study plans and goals by entering their goals and selecting relevant study topics through an on-screen interface.

[1411] Example prompt sentence:

[1412] set_learning_plan(goal="Pass the qualification exam", topics=["Grammar", "Reading"])

[1413] 2. Explanation of learning content by voice input

[1414] As the user progresses through the learning process, they explain what they have learned by voice into the device, which uses speech recognition technology to convert this speech into text data.

[1415] Example prompt sentence:

[1416] This week we learned about relative pronouns

[1417] 3. Analysis and feedback by generative AI

[1418] The converted text data is sent to a server where it is analyzed by a generative AI model, which then generates feedback on the user's level of understanding and key points based on the analysis results.

[1419] Example prompt sentence:

[1420] Analyze the following and provide feedback:This week we learned about relative pronouns.

[1421] 4. Automatic creation of review tests

[1422] Based on the feedback, a review test is generated, which corresponds to the learning topic and can be taken on a smart device.

[1423] Example prompt sentence:

[1424] generate_test(["relative pronoun", "subjunctive mood"])

[1425] 5. Evaluation of test results and recommendations for next study

[1426] The test results taken by the user are evaluated on the server and the next study content is recommended, allowing the user to continuously update their optimal study plan according to their own progress.

[1427] Specific examples

[1428] Setting up a study plan

[1429] When User A logs in to the system for the first time, the terminal displays "Please set your study plan and goals." User A enters "I'm aiming for a TOEIC score of 900," and the server receives the information and stores it in the database. The server then automatically generates an appropriate study plan and displays it to the user.

[1430] Voice input explained

[1431] After learning English grammar, User A explains to his / her device, "This week we learned about relative pronouns." The speech is converted into text and sent to the server. The generation AI analyzes the text and generates feedback saying, "You need to relearn the difference between the nominative and possessive case." The server sends this feedback to the device, and User A confirms it again.

[1432] Automatic creation of review tests

[1433] The server generates a test on relative pronouns and sends it to the device. User A takes the test and submits their answers, and the server evaluates the results. The evaluation result is displayed on the device along with feedback such as "You need to review the use of possessives a little more."

[1434] This system allows users to study efficiently and effectively, and provides appropriate feedback and review tests.

[1435] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1436] Step 1: Setting a study plan and goals

[1437] Input: The user inputs the "goal" and "learning topic" using the terminal.

[1438] Specific operation: The terminal displays a user interface, and the user inputs a goal (e.g., TOEIC score of 900 points) and a learning topic (e.g., listening and reading).

[1439] Output: The server receives these inputs and stores them in a database.

[1440] Data processing: The server organizes the input data and formats it as initial information for automatically generating a learning plan.

[1441] Step 2: Explain the learning content by voice input

[1442] Input: The user speaks what they learned into the device

[1443] Specific operation: When the user speaks a description, the device activates a speech recognition module (e.g., speech_recognition) and converts the speech into text data.

[1444] Output: Text data is sent to the server.

[1445] Data processing: The device converts the voice data into text data and formats it appropriately before sending it to the server.

[1446] Step 3: Analysis and feedback by generative AI

[1447] Input: Text data is sent to the server

[1448] How it works: The server inputs text data into a generative AI model (e.g., Huggingface's transformers), analyzes the content, and generates feedback based on the analysis results.

[1449] Output: The generated feedback is sent to the user terminal.

[1450] Data processing: The server converts the text data into a format suitable for the generative AI model, analyzes the analysis results, and formats them as feedback.

[1451] Step 4: Automatically create review tests

[1452] Input: Information based on generated feedback and learning topics

[1453] Specific operation: The server automatically generates review test questions based on the results of the generative AI model.

[1454] Output: The generated test questions are sent to the user's terminal.

[1455] Data processing: The server combines data to generate test questions based on the learning topic and feedback, and formats it into question format.

[1456] Step 5: Evaluate test results and recommend next study content

[1457] Input: The user takes the review test and enters the answers into the device.

[1458] Specific operation: The device sends the user's test answers to the server, which evaluates the results and recommends the next learning content based on the evaluation results.

[1459] Output: The evaluation results and the next learning content are sent to the user's terminal.

[1460] Data processing: The server analyzes the test answer data, generates evaluation results, and determines the recommended content for the next topic to learn.

[1461] This series of processes allows users to study efficiently and effectively, from setting up a study plan to reviewing the content they have learned, taking review tests, and receiving recommendations for the next study session.

[1462] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1463] ---

[1464] This invention is a system that allows users to study efficiently and effectively, and provides more personalized feedback by combining an emotion engine that recognizes the user's emotions. This system involves a series of processes in which the user sets a study plan and goals, explains what they have learned, and a generative AI analyzes the content and provides feedback, and automatically creates review tests as needed.

[1465] System configuration

[1466] The system includes the following major components:

[1467] 1. User device: A device used by the user as an interface to set up learning plans, input data by voice, take tests, and perform other operations.

[1468] 2. Server: As a central processing unit, it receives data from users and performs analysis, feedback generation, automatic test generation, etc.

[1469] 3. Generative AI: This runs on the server and includes an algorithm that analyzes the learning content sent by the user and generates evaluations and feedback.

[1470] 4. Emotion Engine: A system for recognizing emotions from the user's voice input and adjusting feedback based on the analysis results.

[1471] Specific functions of the system

[1472] 1. Study plan and goal setting

[1473] When a user logs in to the system, the terminal displays an interface for inputting their learning plan and goals. Through this interface, the user sets their own learning goals (e.g., obtaining a qualification, achieving a specific score, etc.) and creates a learning plan based on them. The server receives this information and stores it in a database.

[1474] 2. Explaining what has been learned and growing the generative AI

[1475] As the user progresses through their learning, the device prompts them to "explain what they learned." The user then explains what they learned aloud, and the device converts that speech into text data. The converted text data is sent to a server, where the AI ​​analyzes the content and generates an analysis result.

[1476] 3. Emotion Recognition and Analysis Using an Emotion Engine

[1477] The emotion engine recognizes the user's emotional state from their voice input. The emotion engine's analysis results are provided to the generative AI, which then adjusts the feedback taking into account the user's emotional state. For example, if the user is feeling anxious, the feedback will be provided in a gentler tone.

[1478] 4. Assessment and feedback of what you learned

[1479] The generative AI evaluates the user's explanation and creates feedback on any errors or insufficient understanding. Taking into account the results of the emotion analysis by the emotion engine, the server compiles the evaluation results and feedback and sends them to the device. Through this feedback, the user can check their level of understanding and make any necessary corrections.

[1480] 5. Automatic creation of review tests

[1481] The AI ​​automatically generates review tests based on the learning content with the highest confidence. The server customizes the tests, setting the difficulty and content according to the user's progress. The tests are sent to the device, and the user can take them to solidify the content they have learned.

[1482] 6. Evaluation of test results and recommendations for next study

[1483] The user takes the test and sends their answers to the server. The server evaluates the test results and stores the scores in a database. Based on the evaluation results, the server generates recommendations for what to study next and what points need to be re-studyed, and sends this to the device.

[1484] Specific examples

[1485] Setting up a study plan

[1486] When User B logs in to the system for the first time, the terminal displays "Please set your study plan and goals." User B enters "My goal is a TOEIC score of 900," and the server receives the information and stores it in the database. The server then automatically generates an appropriate study plan and displays it to the user.

[1487] Explanation of what was learned

[1488] After learning English grammar, User B explains to his device, "This week we learned about relative pronouns." The device converts the speech into text and sends it to the server. The generation AI analyzes the text and generates feedback saying, "You need to relearn the difference between the nominative and possessive." The emotion engine recognizes from User B's voice that "you seem impatient," and provides the feedback in a softer tone.

[1489] Automatic creation of review tests

[1490] The server generates a test on relative pronouns and sends it to the device. User B takes the test and submits their answers, and the server evaluates the results. The evaluation result is displayed on the device along with feedback such as "You need to review the use of possessives a little more."

[1491] ---

[1492] As described above, the present invention is a system that provides a series of functions to efficiently and effectively advance the user's learning process. Through analysis and feedback by generative AI, emotion recognition and adjustment by an emotion engine, and automatic creation of review tests, the system can deepen the user's understanding and solidify their knowledge.

[1493] The processing flow will be explained below.

[1494] ---

[1495] Step 1:

[1496] A user logs in to the system.

[1497] The terminal displays the login interface.

[1498] The user enters their login information and goes through the authentication process.

[1499] Step 2:

[1500] A learning plan and goal setting screen is displayed to the user whose device has been authenticated.

[1501] The user inputs their study plan and goals (e.g., aiming for a TOEIC score of 900).

[1502] The terminal transmits the input information to the server.

[1503] Step 3:

[1504] The server stores the user's learning plan and goals in a database.

[1505] The server generates an initial learning plan and schedule based on the received information.

[1506] The server sends the generated learning plan to the terminal.

[1507] Step 4:

[1508] The user progresses through the learning and is ready to explain what they learned.

[1509] The device prompts the user to "Describe what you learned."

[1510] Audibly describe what the user learned.

[1511] Step 5:

[1512] The terminal converts the user's voice input into text data.

[1513] The terminal transmits the converted text data to the server.

[1514] Step 6:

[1515] The server sends the text data to the emotion engine.

[1516] The emotion engine recognizes the user's emotional state from their voice.

[1517] The emotion engine sends the emotion analysis results to the server.

[1518] Step 7:

[1519] The server inputs the text data into the generation AI and begins analysis.

[1520] The generative AI evaluates the learning content based on text data and sentiment analysis results.

[1521] Step 8:

[1522] The generative AI analyzes the user's explanation and generates evaluation results and feedback.

[1523] The generative AI takes into account the user's emotional state and adjusts the tone and content of the feedback.

[1524] The server formats the generated evaluation results and feedback and sends them to the device.

[1525] Step 9:

[1526] The device displays the feedback to the user.

[1527] The user reviews the feedback and makes any necessary corrections.

[1528] Step 10:

[1529] The server instructs the AI ​​to generate review tests based on the feedback.

[1530] Generative AI determines the content of the review test and automatically creates the test.

[1531] Step 11:

[1532] The server customizes the generated tests and configures them as the user progresses.

[1533] The server sends the test to the device.

[1534] Step 12:

[1535] The device displays a test notification to the user, prompting them to take the test.

[1536] The user performs the test on the device.

[1537] The user enters the answers to the test and the terminal sends the answers to the server.

[1538] Step 13:

[1539] The server evaluates the test results and stores the scores in a database.

[1540] The server generates additional feedback based on the test results.

[1541] Step 14:

[1542] The device displays additional feedback to the user, prompting them to relearn.

[1543] The user re-learns and explains the new learning content to the terminal.

[1544] Step 15:

[1545] The server periodically evaluates the user's learning progress.

[1546] The server automatically updates the learning plan based on progress.

[1547] The server sends recommendations to the device on what to learn next and how to learn it.

[1548] Step 16:

[1549] The device displays recommended information to the user, supporting continuous learning.

[1550] ---

[1551] The above is the specific processing flow of the system, which effectively supports the user's learning process and aims to solidify knowledge through evaluation and feedback by the generative AI, as well as emotion recognition and adjustment by the emotion engine.

[1552] Example 2

[1553] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1554] Conventional learning support systems provide uniform feedback without considering the user's emotional state, making it difficult to provide appropriate instruction tailored to individual needs. Furthermore, evaluation of the user's learning content and recommendations for the next lesson are often done manually, resulting in reduced learning efficiency. Furthermore, in many cases, only standard test questions are provided, resulting in a lack of customized tests tailored to the user's level of understanding. A system that can solve these issues and provide a more personalized learning experience is needed.

[1555] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1556] In this invention, the server includes means for a user to set a study plan and goals, means for audibly explaining what the user has learned, means for converting the speech into text data, means for inputting the text data into a generative artificial intelligence model and analyzing it, means for evaluating the study content and providing feedback based on the analysis results, means for recognizing the user's emotional state, means for adjusting the feedback taking the user's emotional state into account, means for automatically creating a review test based on the feedback, means for evaluating the test results and recommending the next study content, and means for updating the study plan according to the user's progress. This makes it possible to provide feedback that takes the user's emotional state into account and tests that are customized according to the user's level of understanding.

[1557] A "user" is an individual or group who uses the system to set a learning plan and goals and progress through the learning content.

[1558] "Study planning" is the process of setting specific study content and schedules to be achieved within a specific period based on the study goals that the user wants to achieve.

[1559] A "goal" is a specific outcome or performance that a user wants to achieve through the system.

[1560] "Audio explanation means" is a method by which a user verbally conveys what they have learned to the system.

[1561] "Means for converting into text data" refers to a technology for converting a user's voice input into text format data.

[1562] A "generative artificial intelligence model" is an artificial intelligence algorithm that analyzes data entered by the user and evaluates learning content and generates feedback.

[1563] "Means of analysis" refers to the process of using a generative artificial intelligence model to understand and evaluate the learning content explained by the user.

[1564] "Means for providing feedback" refers to a method for informing users of their learning progress and areas for improvement based on the analysis results.

[1565] "Means for recognizing emotional states" refers to technology for identifying emotions from a user's voice or text data and detecting changes in the user's emotions.

[1566] "Feedback tailoring" is the process of optimizing the content and tone of the feedback provided, taking into account the user's emotional state.

[1567] The "means for automatically creating review tests" is a technology that automatically generates review test questions based on the user's learning content and level of understanding.

[1568] "Means for evaluating test results" refers to the process of analyzing the results of the test taken by the user and evaluating their performance.

[1569] "Means for recommending next learning content" is a process that recommends the next learning content or points that need to be re-learned based on test results and the user's progress.

[1570] The "means for updating the study plan" is a method for appropriately modifying an existing study plan in accordance with the user's progress and updating it to match the latest study goals.

[1571] The present invention provides a learning support system that helps users study efficiently and effectively, and in particular has a feedback function that takes into account the user's emotional state. This system is composed of the following main hardware and software components:

[1572] Hardware and software used

[1573] 1. User Device

[1574] A device that a user uses as an interface to set up a study plan, input data by voice, take tests, receive feedback, etc. Examples include personal computers, smartphones, and tablets. These devices use voice recognition software (e.g., Google Cloud Speech-to-Text) to convert the user's speech into text.

[1575] 2. Server

[1576] As a central processing unit, it receives data from users and performs analysis, creates feedback, automatically generates tests, etc. On the server, generative AI models (e.g., GPT-4) and emotion engines (e.g., Affectiva) run to analyze the user's learning content and emotional state.

[1577] 3. Generative AI Models

[1578] The generative AI model, which runs on a server and includes algorithms that analyze the text data submitted by users, evaluate their learning, generate feedback, and create review tests, is known as a large-scale language model.

[1579] 4. Emotion Engine

[1580] The emotion engine is a system for recognizing the emotional state of a user from their voice input and providing the analysis results to a generative AI model, which then adjusts the feedback according to the user's emotions.

[1581] Specific processing flow and functions

[1582] Study plan and goal setting

[1583] When a user logs in to the system, the terminal displays the message "Please set your study plan and goals." The user enters their study goal (e.g., get a high score on a test) and the terminal sends this information to the server, which receives it and stores it in a database.

[1584] Specific behavior:

[1585] The user inputs "I want to make a study plan with the goal of passing a qualification exam" into the terminal, and the server receives the information and displays a confirmation message to the user.

[1586] Explanation of what was learned

[1587] After the user has progressed through the learning process, the device will prompt them to "explain what you learned." The user will explain aloud, and the device will use speech recognition software to convert the speech into text.

[1588] Specific behavior:

[1589] When a user speaks to the device, "I learned about differential and integral calculus this weekend," the device uses voice recognition software to convert this into text data, "I learned about differential and integral calculus this weekend," and sends it to the server.

[1590] Analysis of learning content and generation of feedback

[1591] The generative AI model on the server analyzes the text data sent by the user and evaluates the learning content. The emotion engine recognizes emotions from the user's voice and provides the results to the generative AI model. The generative AI model adjusts the feedback taking the emotional state into account.

[1592] Specific behavior:

[1593] If the server assesses that the learning content is fully understood and the emotion engine recognizes that the user is showing signs of impatience during the explanation, the generative AI model will provide gentle feedback such as, "You have a solid understanding of the basics of calculus. Let's try some applied problems next."

[1594] Automatic creation of review tests

[1595] Based on the analysis results, the generative AI model automatically generates review tests to improve the user's understanding. The server selects appropriate test questions and sends them to the device.

[1596] Specific behavior:

[1597] The server generates five questions on the basic concepts of calculus, which the terminal displays as a test to the user. The user takes the test and sends the answers to the server.

[1598] Test result evaluation and study recommendations

[1599] The server evaluates the test results and recommends the next study content to the user based on the results, allowing the user to develop an efficient study plan that is in line with their own learning progress.

[1600] Specific behavior:

[1601] After the user completes the test and the server evaluates it, a recommendation is displayed on the device saying, "We recommend that you review the basic concepts again before moving on to the application questions."

[1602] Prompt Sentence Examples

[1603] "Set a study plan and goals."

[1604] "Please explain what you learned."

[1605] "Please explain what we learned this week in audio."

[1606] "Please begin the review test."

[1607] The above is a specific embodiment for carrying out the invention. By using this system, users can receive personalized feedback according to their learning progress and promote effective learning.

[1608] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1609] Step 1:

[1610] A user logs into the system

[1611] Input: Username and Password

[1612] Data processing: The device sends the entered information to the server. The server performs user authentication and, if successful, displays the home screen.

[1613] Output: The user can access the home screen.

[1614] Specific behavior:

[1615] The user enters their username and password into the device and presses the send button. The device sends this information to the server, which then authenticates them and displays the home screen on the device.

[1616] Step 2:

[1617] Your device will display the study plan and goal setting screen.

[1618] Input: Learning objectives and plan content

[1619] Data processing: The terminal receives the user's input and sends it to the server, which stores the information in a database.

[1620] Output: The configured learning plan is saved with a confirmation message.

[1621] Specific behavior:

[1622] The user enters "I'm aiming for a TOEIC score of 900" and presses "Confirm." The device sends this information to the server, which saves it in the database. A message indicating that saving has been completed is displayed on the device.

[1623] Step 3:

[1624] The user explains the learning content aloud

[1625] Input: User voice input

[1626] Data processing: Use speech recognition software (e.g., Google Cloud Speech-to-Text) to convert voice input into text data. The text is sent to the server.

[1627] Output: The learning content in text format is saved on the server.

[1628] Specific behavior:

[1629] The user speaks to the device, saying, "This weekend I learned about calculus." The device converts the speech to text using Google Cloud Speech-to-Text and sends it to the server. The text data is stored on the server.

[1630] Step 4:

[1631] The server analyzes the learning content

[1632] Input: Text data (user learning content)

[1633] Data processing: A generative AI model (e.g., GPT-4) analyzes the text data and evaluates what it has learned.

[1634] Output: Analysis results (evaluation details)

[1635] Specific behavior:

[1636] The server generates an analysis result that says, "The basics of differential and integral calculus are understood, but a review is needed before moving on to applied problems."

[1637] Step 5:

[1638] Emotion engine recognizes emotional states from speech

[1639] Input: Audio data

[1640] Data processing: An emotion engine (e.g., Affectiva) analyzes the audio data and recognizes the emotional state.

[1641] Output: Emotional state data

[1642] Specific behavior:

[1643] The user sends a voice that indicates impatience while giving an explanation to the server, and the emotion engine recognizes the "impatience."

[1644] Step 6:

[1645] Server provides tailored feedback

[1646] Input: Analysis results and emotional state

[1647] Data processing: Generative AI takes into account your emotional state and adjusts the content and tone of your feedback.

[1648] Output: Regulated Feedback

[1649] Specific behavior:

[1650] The server generates feedback in a gentle tone, saying, "You have a solid understanding of the basics of differential and integral calculus. Now let's try some applied problems." The feedback is displayed on the terminal.

[1651] Step 7:

[1652] Automatically create review quizzes

[1653] Input: Analysis results

[1654] Data processing: Generative AI automatically generates review tests based on the analysis results.

[1655] Output: A set of review test questions

[1656] Specific behavior:

[1657] The server generates "five problems on the basic concepts of differential and integral calculus" and sends them to the terminal as a test.

[1658] Step 8:

[1659] User takes the test and submits the results

[1660] Input: User's test answers

[1661] Data processing: The terminal sends the user's answers to the server, which evaluates the answers.

[1662] Output: Evaluation results

[1663] Specific behavior:

[1664] The user answers the questions and submits the results. The server evaluates the test results and generates feedback such as "Area of ​​improvement: You need to use possessives and relative pronouns appropriately."

[1665] Step 9:

[1666] Recommend what to study next

[1667] Input: Evaluation result

[1668] Data processing: The server generates the next learning content based on the evaluation results and makes recommendations.

[1669] Output: Recommended learning content

[1670] Specific behavior:

[1671] The server generates a message saying, "Next, we recommend that you learn about the use of possessives and relative pronouns," and displays it on the terminal.

[1672] These are the main processing steps of the system, which allows users to receive personalized feedback based on their learning progress and effectively advance their learning.

[1673] (Application example 2)

[1674] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1675] Previous learning support systems relied on standard methods to evaluate and provide feedback on users' learning content, but were unable to consider the emotional state of each individual user. As a result, learning effectiveness could decline if users became stressed or lost motivation. Furthermore, the customization of review tests and study plans did not reflect the user's psychological state, resulting in a lack of personalized learning support.

[1676] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1677] In this invention, the server includes: means for a user to set a study plan and goals; means for audibly explaining what the user has learned; means for converting the voice into text data; means for inputting the text data into a generative artificial intelligence and analyzing it; means including an emotion engine for recognizing the emotional state of the user from the voice input; means for evaluating the study content and providing feedback based on the analysis results and the emotional state; means for automatically creating review tests based on the feedback; means for evaluating the test results and recommending next study content; and means for updating the study plan according to the user's progress. This makes it possible to create personalized feedback and tests that take the user's emotional state into consideration, which is expected to improve learning effectiveness.

[1678] "Users"

[1679] refers to an individual who uses a learning system to carry out learning activities.

[1680] "Study Plan"

[1681] A learning plan is a plan that describes specific learning content and schedules to achieve the goals set by the user.

[1682] "Voice input"

[1683] is a means for users to provide information to the system by dictating what they have learned.

[1684] "Text data"

[1685] It is data that converts voice input into text format.

[1686] "Generative Artificial Intelligence"

[1687] is a system that includes algorithms and programs for analyzing text data converted from voice input.

[1688] "Emotion Engine"

[1689] is a system for recognizing and analyzing a user's emotional state from their voice input.

[1690] "feedback"

[1691] This is information that includes evaluations and advice on what the user has learned.

[1692] "Review test"

[1693] These are tests that the system automatically creates to help users confirm and solidify what they have learned.

[1694] "Next lesson content"

[1695] This refers to the content or topic that the user should study next.

[1696] "Recommendation"

[1697] This refers to the learning content and methods that the system suggests based on the user's learning progress and test results.

[1698] "progress"

[1699] This refers to the user's level of achievement and progress as they progress through their studies.

[1700] "update"

[1701] This refers to modifying and updating a user's study plan based on the latest information and progress.

[1702] This invention provides a system for enabling users to study efficiently and effectively. The system includes means for setting a user's study plan and goals and for explaining the user's learned content via voice. A server converts the user's voice into text data, which is then input into a generative AI model for analysis. The system also uses an emotion engine to recognize the user's emotional state from the user's voice input and adjusts feedback based on this. The system automatically creates review tests based on the feedback, evaluates the test results, and recommends next study content. The system also includes means for updating the study plan according to the user's progress.

[1703] System configuration

[1704] The system includes the following major components:

[1705] 1. User device: A device used by the user as an interface to set up learning plans, input voice, take tests, etc. User devices can be VR headsets, smartphones, PCs, etc.

[1706] 2. Server: As a central processing unit, it receives data from users and performs analysis, feedback generation, automatic test generation, etc.

[1707] 3. Generative AI model: Runs on the server and includes an algorithm that analyzes the learning content sent by the user and generates evaluation and feedback.

[1708] 4. Emotion Engine: A system for recognizing emotions from the user's voice input and adjusting feedback based on the analysis results.

[1709] Specific functions of the system

[1710] 1. Study plan and goal setting

[1711] When a user logs in to the system, the terminal displays an interface for inputting their learning plan and goals. Through this interface, the user sets their own learning goals (e.g., obtaining a qualification, achieving a specific score, etc.) and creates a learning plan based on them. The server receives this information and stores it in a database.

[1712] 2. Explaining what has been learned and analyzing the generative AI model

[1713] After the user has progressed through their learning, the device prompts them to "explain what they have learned." The user then explains what they have learned by voice, and the device converts that speech into text data. The converted text data is sent to a server, where the generative AI model analyzes the content and generates analysis results.

[1714] 3. Emotion Recognition and Analysis Using an Emotion Engine

[1715] The emotion engine recognizes the user's emotional state from their voice input. The emotion engine's analysis results are fed into a generative AI model, which then adjusts the feedback based on the user's emotional state. For example, if the user is feeling anxious, the feedback will be delivered in a gentler tone.

[1716] 4. Assessment and feedback of what you learned

[1717] The generative AI model evaluates the user's explanation and generates feedback on any errors or insufficient understanding. Taking into account the emotion analysis results of the emotion engine, the server compiles the evaluation results and feedback and sends them to the device. Through this feedback, the user can check their own level of understanding and make any necessary corrections.

[1718] 5. Automatic creation of review tests

[1719] The generative AI model automatically generates review tests based on the learning content with the highest confidence. The server customizes these tests, setting the difficulty and content according to the user's progress. The tests are sent to the device, and the user can take them to solidify what they have learned.

[1720] 6. Evaluation of test results and recommendations for next study

[1721] The user takes the test and sends their answers to the server. The server evaluates the test results and stores the scores in a database. Based on the evaluation results, the server generates recommendations for what to study next and what points need to be re-studyed, and sends this to the device.

[1722] Specific examples

[1723] Setting up a study plan

[1724] When a user logs in to the system for the first time, the terminal displays, "Please set your study plan and goals." The user enters, "My goal is a TOEIC score of 900," and the server receives the information and stores it in the database. The server then automatically generates an appropriate study plan and displays it to the user.

[1725] Explanation of what was learned

[1726] After learning English grammar, a user speaks to their device, saying, "This week we learned about relative pronouns." The device converts the speech into text and sends it to the server. The generative AI model analyzes the text and generates feedback saying, "You need to relearn the difference between nominative and possessive." The emotion engine recognizes from the user's voice that "you seem impatient" and provides the feedback in a softer tone. An example of feedback would be, "Let's review the difference between nominative and possessive again."

[1727] Automatic creation of review tests

[1728] The server generates a test on relative pronouns and sends it to the device. The user takes the test, submits their answers, and the server evaluates the results. The evaluation result is displayed on the device along with feedback such as "You need to review the use of possessives a little more."

[1729] Prompt Sentence Examples

[1730] "To deepen your understanding of neural networks, let's next tackle some practical problems."

[1731] As described above, this invention is a system that utilizes an emotion engine and a generative AI model to support the user's learning process and provide a more effective learning experience.

[1732] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1733] Step 1:

[1734] The device displays an interface for the user to set up a learning plan and goals. The user enters their learning goals (e.g., obtaining a qualification, achieving a specific score, etc.) on this screen, and the device sends the information to the server. The server stores the received information in a database.

[1735] Input: User-entered learning plans and goals

[1736] Output: Learning plans and goals received and stored by the server

[1737] Specific operation: When a user enters "My goal is a TOEIC score of 900" and clicks the data submission button, the data is sent from the device to the server, which then writes and saves this data in the database.

[1738] Step 2:

[1739] After the user has progressed through their studies, the device prompts them to "explain what they have learned." The user explains what they have learned by voice, and the device converts the voice into text data.

[1740] Input: User voice input

[1741] Output: Learning content converted into text data

[1742] What it does: A user says, "This week we learned about relative pronouns," and the system recognizes the speech and converts it into text.

[1743] Step 3:

[1744] The converted text data is sent from the device to a server and input into a generative AI model, which analyzes the content and generates an evaluation and feedback.

[1745] Input: Learning content converted into text data

[1746] Output: Analysis results, evaluation, and feedback from the generative AI model

[1747] Specific operation: The text data "I learned about relative pronouns" is sent to the generative AI model for analysis. The generative AI model generates feedback saying "You need to re-learn the difference between nominative and possessive cases."

[1748] Step 4:

[1749] The emotion engine recognizes the user's emotional state from their voice input, and the analysis results of the emotion engine are provided to the generative AI model, which then adjusts the feedback taking into account the user's emotional state.

[1750] Input: User's voice data

[1751] Output: User's emotional state (e.g., impatience)

[1752] Specific operation: The emotion engine recognizes "impatience" from the user's voice and provides the analysis results to the generative AI model. The generative AI model then adjusts the voice to "provide feedback in a gentler tone because the user is impatient."

[1753] Step 5:

[1754] The server sends the adjusted feedback to the device, which then displays the feedback to the user, who then checks the feedback and determines their level of understanding.

[1755] Input: Calibrated feedback

[1756] Output: Feedback that is displayed to the user

[1757] What happens: The server sends the adjusted feedback "Let's review the difference between nominative and possessive again" to the device, which then displays it to the user.

[1758] Step 6:

[1759] The generative AI model automatically generates review tests based on the user's learning, and the server customizes the tests. The tests are then sent to the user's device, where they are then taken.

[1760] Input: tailored feedback, what the user has learned

[1761] Output: Auto-generated review test

[1762] Specific operation: A test on relative pronouns is generated, adjusted to an appropriate level of difficulty by the server, and sent to the device. The user then takes the test.

[1763] Step 7:

[1764] The user takes the test and sends the answers from the device to the server. The server evaluates the results and stores them in a database. It then generates recommendations for what to study next and what points need to be re-studyed, and sends them to the device.

[1765] Input: Test answer data

[1766] Output: Evaluation results and next training recommendation

[1767] What it does: The user takes the test and submits their answers, and the server evaluates the results. The evaluation result is displayed on the device along with feedback such as "You need to review the use of possessives a little more."

[1768] The above processing steps enable the system to create personalized feedback and tests that take into account the user's emotional state, which is expected to improve learning effectiveness.

[1769] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1770] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1771] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1772] [Fourth embodiment]

[1773] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1774] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1775] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1776] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1777] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1778] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1779] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1780] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1781] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1782] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1783] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1784] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1785] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1786] ---

[1787] The present invention is a system that allows users to study efficiently and effectively. This system involves a series of processes in which a user sets a study plan and goals, explains what they have learned, and a generative AI analyzes the content, provides feedback, and automatically creates review tests as needed.

[1788] System configuration

[1789] The system includes the following major components:

[1790] 1. User device: A device used by the user as an interface to set up learning plans, input data by voice, take tests, and perform other operations.

[1791] 2. Server: As a central processing unit, it receives data from users and performs analysis, feedback generation, automatic test generation, etc.

[1792] 3. Generative AI: This runs on the server and includes an algorithm that analyzes the learning content sent by the user and generates evaluations and feedback.

[1793] Specific functions of the system

[1794] 1. Study plan and goal setting

[1795] When a user logs in to the system, the terminal displays an interface for inputting their learning plan and goals. Through this interface, the user sets their own learning goals (e.g., obtaining a qualification, achieving a specific score, etc.) and creates a learning plan based on them. The server receives this information and stores it in a database.

[1796] 2. Explaining what has been learned and growing the generative AI

[1797] As the user progresses through their learning, the device prompts them to "explain what they learned." The user then explains what they learned aloud, and the device converts that speech into text data. The converted text data is sent to a server, where the AI ​​analyzes the content and generates an analysis result.

[1798] 3. Evaluation and feedback of what you have learned

[1799] The generation AI evaluates the user's explanation and generates feedback on any errors or insufficient understanding. The server compiles the evaluation results and feedback and sends them to the device. Through this feedback, the user can check their own understanding and make any necessary corrections.

[1800] 4. Automatic creation of review tests

[1801] The AI ​​automatically generates review tests based on the learning content with the highest confidence. The server customizes the tests, setting the difficulty and content according to the user's progress. The tests are sent to the device, and the user can take them to solidify the content they have learned.

[1802] 5. Evaluation of test results and recommendations for next study

[1803] The user takes the test and sends their answers to the server. The server evaluates the test results and stores the scores in a database. Based on the evaluation results, the server generates recommendations for what to study next and what points need to be re-studyed, and sends this to the device.

[1804] Specific examples

[1805] Setting up a study plan

[1806] When User A logs in to the system for the first time, the terminal displays "Please set your study plan and goals." User A enters "I'm aiming for a TOEIC score of 900," and the server receives the information and stores it in the database. The server then automatically generates an appropriate study plan and displays it to the user.

[1807] Explanation of what was learned

[1808] After learning English grammar, User A explains aloud to his / her device, "This week we learned about relative pronouns." The device converts the speech into text and sends it to the server. The generation AI analyzes the text and generates feedback saying, "You need to relearn the difference between the nominative and possessive case." The server sends this feedback to the device, and User A confirms it again.

[1809] Automatic creation of review tests

[1810] The server generates a test on relative pronouns and sends it to the device. User A takes the test and submits their answers, and the server evaluates the results. The evaluation result is displayed on the device along with feedback such as "You need to review the use of possessives a little more."

[1811] ---

[1812] As described above, the present invention is a system that provides a series of functions to efficiently and effectively advance the user's learning process. Analysis and feedback by generative AI, as well as the automatic creation of review tests, can deepen the user's understanding and help them solidify their knowledge.

[1813] The processing flow will be explained below.

[1814] ---

[1815] Step 1:

[1816] A user logs in to the system.

[1817] The terminal displays the login interface.

[1818] The user enters their login information and goes through the authentication process.

[1819] Step 2:

[1820] A learning plan and goal setting screen is displayed to the user whose device has been authenticated.

[1821] The user inputs their study plan and goals (e.g., aiming for a TOEIC score of 900).

[1822] The terminal transmits the input information to the server.

[1823] Step 3:

[1824] The server stores the user's learning plan and goals in a database.

[1825] The server generates an initial learning plan and schedule based on the received information.

[1826] The server sends the generated learning plan to the terminal.

[1827] Step 4:

[1828] The user progresses through the learning and is ready to explain what they learned.

[1829] The device prompts the user to "Describe what you learned."

[1830] Audibly describe what the user learned.

[1831] Step 5:

[1832] The terminal converts the user's voice input into text data.

[1833] The terminal transmits the converted text data to the server.

[1834] Step 6:

[1835] The server inputs the text data into the generation AI and begins analysis.

[1836] The generative AI evaluates the learning content based on text data.

[1837] Step 7:

[1838] The generative AI analyzes the user's explanation and generates evaluation results and feedback.

[1839] The server formats the generated evaluation results and feedback and sends them to the device.

[1840] Step 8:

[1841] The device displays the feedback to the user.

[1842] The user reviews the feedback and makes any necessary corrections.

[1843] Step 9:

[1844] The server instructs the AI ​​to generate review tests based on the feedback.

[1845] Generative AI determines the content of the review test and automatically creates the test.

[1846] Step 10:

[1847] The server customizes the generated tests and configures them as the user progresses.

[1848] The server sends the test to the device.

[1849] Step 11:

[1850] The device displays a test notification to the user, prompting them to take the test.

[1851] The user performs the test on the device.

[1852] The user enters the answers to the test and the terminal sends the answers to the server.

[1853] Step 12:

[1854] The server evaluates the test results and stores the scores in a database.

[1855] The server generates additional feedback based on the test results.

[1856] Step 13:

[1857] The device displays additional feedback to the user, prompting them to relearn.

[1858] The user re-learns and explains the new learning content to the terminal.

[1859] Step 14:

[1860] The server periodically evaluates the user's learning progress.

[1861] The server automatically updates the learning plan based on progress.

[1862] The server sends recommendations to the device on what to learn next and how to learn it.

[1863] Step 15:

[1864] The device displays recommended information to the user, supporting continuous learning.

[1865] ---

[1866] The above is the specific processing flow of the system, which effectively supports the user's learning process and aims to ensure that knowledge is continuously retained through evaluation and feedback by the generative AI.

[1867] Example 1

[1868] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1869] Conventional learning support systems have had difficulty providing feedback and review content tailored to each user's individual learning progress and level of understanding. This has led to problems such as reduced learning efficiency and effectiveness. Furthermore, it has been difficult to evaluate the specific level of understanding of the content a user has learned and to appropriately recommend the next learning content.

[1870] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1871] In this invention, the server includes means for a user to set a study plan and goals, means for audibly explaining what the user has learned, means for converting the speech into text data, means for inputting the text data into a generative artificial intelligence and analyzing it, means for evaluating the study content and providing feedback based on the analysis results, means for automatically creating review tests based on the feedback, means for evaluating the test results and recommending next study content, means for updating the study plan according to the user's progress, means for user login authentication, means for storing the user's study content and results in a database, and means for customizing the content of review tests based on the user's progress. This makes it possible to improve the efficiency and effectiveness of users' learning and realize optimal feedback and recommendations for next study content according to individual learning progress.

[1872] A "user" is a person who uses the system to set up a learning plan and manage the progress of their learning.

[1873] A "server" is a central processing unit that receives data from users, performs analysis, feedback, and automatically generates tests, and provides these results to users.

[1874] A "terminal" is a device that a user uses as an interface, and is used to perform operations such as setting up a study plan, inputting voice data, and taking tests.

[1875] A "study plan" is a specific study schedule and procedure for achieving a goal set by a user.

[1876] A "goal" is a specific learning outcome or grade that a user aims to achieve.

[1877] "Audio" refers to the spoken words that a user makes to explain what they have learned.

[1878] "Text data" is voice data converted into text information, and is the input data for analysis by the generative AI model.

[1879] "Generative AI" includes algorithms and techniques for analyzing user-provided data and generating ratings and feedback.

[1880] "Analysis" is the process of evaluating content based on text data and identifying errors or areas of lack of understanding.

[1881] "Feedback" refers to evaluation information on areas for improvement and learning content provided to the user based on the analysis results.

[1882] A "review test" is a confirmation test created to help users solidify what they have learned.

[1883] "Recommendations" are information that suggests what content the user should study next based on their learning results.

[1884] "Progress" refers to the process or state in which a user advances their studies according to a study plan.

[1885] "Login authentication" is a procedure for verifying a user's identity when accessing a system.

[1886] A "database" is a storage device for saving the user's learning content and results.

[1887] "Customization" refers to the process of adjusting the content and difficulty level according to the user's progress and level of understanding.

[1888] The present invention is a system that allows users to study efficiently and effectively. This system involves a series of processes in which a user sets a study plan and goals, explains what they have learned, and a generative AI analyzes the content, provides feedback, and automatically creates review tests as needed.

[1889] System configuration

[1890] The system includes the following major components:

[1891] 1. User device: A device used by users as an interface to set up learning plans, input voice commands, take tests, etc. User devices include PCs, smartphones, tablets, etc.

[1892] 2. Server: A central processing unit that receives data from users, analyzes it, creates feedback, automatically generates tests, etc. The server is a high-performance computer, and the database can also be stored within the server.

[1893] 3. Generative AI: This includes algorithms that run on a server, analyze the learning content sent by the user, and generate evaluations and feedback. Generative AI is realized using natural language processing and machine learning techniques.

[1894] Study plan and goal setting

[1895] When a user logs in to the system, the terminal displays an interface for inputting their learning plan and goals. Through this interface, the user sets their own learning goals (e.g., obtaining a qualification, achieving a specific score, etc.) and creates a learning plan based on them. The server receives this information and stores it in a database.

[1896] Explanation of what was learned

[1897] As the user progresses through their learning, the device prompts them to "explain what they learned." The user then explains what they learned aloud, and the device converts that speech into text data. The converted text data is sent to a server, where the AI ​​analyzes the content and generates an analysis result.

[1898] Assessment and feedback of what you learned

[1899] The generative AI evaluates the user's explanation and generates feedback on any errors or insufficient understanding. The server compiles the evaluation results and feedback and sends them to the device. Through this feedback, the user can check their own understanding and make any necessary corrections.

[1900] Automatic creation of review tests

[1901] The AI ​​automatically generates review tests based on the learning content with the highest confidence. The server customizes these tests, setting the difficulty and content according to the user's progress. The tests are sent to the device, and the user can take them to solidify what they have learned.

[1902] Evaluation of test results and recommendations for next study content

[1903] The user takes the test and sends their answers to the server. The server evaluates the test results and stores the scores in a database. Based on the evaluation results, the server generates recommendations for what to study next and what points need to be re-studyed, and sends this information to the device.

[1904] Specific examples

[1905] Setting up a study plan

[1906] When User A logs in to the system for the first time, the terminal displays "Please set your study plan and goals." User A enters "I'm aiming for a TOEIC score of 900," and the server receives the information and stores it in the database. The server then automatically generates an appropriate study plan and displays it to the user.

[1907] Explanation of what was learned

[1908] After learning English grammar, User A explains aloud to his / her device, "This week we learned about relative pronouns." The device converts the speech into text and sends it to the server. The generation AI analyzes the text and generates feedback saying, "You need to relearn the difference between the nominative and possessive case." The server sends this feedback to the device, and User A confirms it again.

[1909] Automatic creation of review tests

[1910] The server generates a test on relative pronouns and sends it to the device. User A takes the test and submits their answers, and the server evaluates the results. The evaluation result is displayed on the device along with feedback such as "You need to review the use of possessives a little more."

[1911] Prompt Sentence Examples

[1912] "Please explain what you learned this week."

[1913] Please suggest what we should learn next.

[1914] "Evaluate the results of the test and provide feedback."

[1915] In this way, the system for implementing the invention provides a series of functions to efficiently and effectively advance the user's learning process. Analysis and feedback using a generative AI model, as well as the automatic creation of review tests, can deepen the user's understanding and help them solidify their knowledge.

[1916] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1917] Step 1: User Login

[1918] A user logs in to the system using a terminal. The terminal displays an interface for entering a user ID and password, and the user enters the information and presses the "Login" button. The server authenticates the received login information and retrieves the user's account information from the database. If authentication is successful, the server returns a login success status to the terminal and displays the main menu to the user.

[1919] Input: User ID, Password

[1920] Output: Login successful status, main menu displayed

[1921] Specific behavior:

[1922] 1. The device displays the login page.

[1923] 2. The user enters their ID and password and submits it.

[1924] 3. The server receives the login information and authenticates it with the database.

[1925] 4. When authentication is successful, the main menu is displayed on the device.

[1926] Step 2: Setting a study plan and goals

[1927] The device displays a prompt saying, "Please set your study plan and goals." The user then inputs their study goals and plan into the interface. For example, they can set a goal such as "Aim for a TOEIC score of 900." The server receives this information and stores it in a database. The server then automatically suggests study content related to the user's goals and displays them as a list on the device.

[1928] Input: Learning objectives, learning plan

[1929] Output: Saved learning plans, suggested learning content

[1930] Specific behavior:

[1931] 1. The device displays an interface that prompts the user to enter their learning plan and goals.

[1932] 2. The user enters and submits the learning objectives.

[1933] 3. The server stores this information in a database.

[1934] 4. The server suggests relevant learning content and displays it on the device.

[1935] Step 3: Explain what you learned

[1936] As the user progresses through their learning, the device prompts them to "explain what they learned." The user then explains what they learned by voice. For example, they might say, "This week we learned about relative pronouns." The device converts the voice data into text data and sends it to the server. The generative AI analyzes this text data and understands its content.

[1937] Input: Audio explanation of what you're learning

[1938] Output: Text data, analysis results

[1939] Specific behavior:

[1940] 1. The device prompts the user for voice input.

[1941] 2. Explain in audio what the user learned.

[1942] 3. The device converts the voice into text data.

[1943] 4. The text data is sent to the server and analyzed by the generating AI.

[1944] Step 4: Assessment and feedback on what was learned

[1945] The generation AI analyzes the text data and evaluates the content of the user's explanation. As a result of the analysis, it extracts any errors or parts of insufficient understanding and generates feedback based on this. For example, it may create feedback such as "You need to relearn the difference between the nominative and possessive case." The server compiles this feedback and sends it to the device, where it is displayed to the user.

[1946] Input: Analysis results of text data

[1947] Output: Feedback

[1948] Specific behavior:

[1949] 1. Generative AI analyzes text data.

[1950] 2. Detect errors or areas of lack of understanding.

[1951] 3. Feedback is generated and sent to the device by the server.

[1952] 4. The device displays feedback to the user.

[1953] Step 5: Automatically create review tests

[1954] The AI ​​automatically generates a review test based on the user's level of understanding. The server generates specific questions based on the user's learning content and assessment results. For example, it creates multiple-choice questions and fill-in-the-blank questions about relative pronouns. The test is sent to the device and prepared for the user to take.

[1955] Input: Learning content, evaluation results

[1956] Output: Review test

[1957] Specific behavior:

[1958] 1. Generative AI generates test questions based on the evaluation results.

[1959] 2. Customize the server-generated tests.

[1960] 3. Send the test to the device and display it to the user.

[1961] Step 6: Evaluate test results and recommend next study content

[1962] The user completes the test and sends the answers to the server via their device. The server evaluates the test results and stores the scores in a database. Based on the evaluation results, the generative AI recommends what to study next and what points need to be re-studyed. For example, specific advice such as "You need to review the use of possessives a little more" is displayed on the device.

[1963] Input: Test Answers

[1964] Output: Evaluation results, recommendations for next learning content

[1965] Specific behavior:

[1966] 1. A user takes a test and submits their answers.

[1967] 2. The server evaluates the test results and stores the scores in a database.

[1968] 3. The generative AI recommends the next learning content based on the evaluation results.

[1969] 4. The server sends the recommendation information to the device and displays it to the user.

[1970] (Application example 1)

[1971] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1972] Conventional learning support systems lack the functionality to provide detailed feedback and appropriate review tests to help users progress through their studies efficiently and effectively. They also lack a system that can flexibly update learning plans according to the user's progress, making them unsuitable tools for maximizing the effectiveness of learning. Furthermore, they lack sufficient accuracy and flexibility when it comes to explaining learning content through voice input and providing analysis and feedback using generative AI. These issues need to be resolved.

[1973] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1974] In this invention, the server includes: means for a user to set a study plan and goals; means for audibly explaining what the user has learned; means for converting the speech into text data; means for inputting the text data into a generative artificial intelligence for analysis; means for evaluating the study content and providing feedback based on the analysis results; means for automatically creating review tests based on the feedback; means for evaluating the test results and recommending next study content; means for updating the study plan according to the user's progress; means for generating test questions based on a specified study topic using a smart device; and means for analyzing the content explained by the user audibly when generating feedback to the user using the generative artificial intelligence. This enables the user to study efficiently and effectively and receive appropriate feedback and review tests.

[1975] "Means for setting study plans and goals" is a system that allows users to clarify the goals they want to achieve and set specific study steps and schedules based on those goals.

[1976] The "means for explaining learned content by voice" is a system that allows the user to explain what they have learned by voice input.

[1977] "Means for converting voice into text data" refers to a technique for converting voice information input by a user into text information.

[1978] "Means of inputting text data into a generative artificial intelligence for analysis" refers to the function of inputting converted text information into an AI and analyzing its content.

[1979] The "means for evaluating learning content and providing feedback" is a system that evaluates learning content based on the content analyzed by AI and provides users with feedback on areas for improvement and their level of understanding.

[1980] The "means for automatically generating review tests" is a mechanism for automatically generating test questions for users to review based on the provided feedback.

[1981] "Means for evaluating test results and recommending next study content" is a function that evaluates the results of a test taken by the user and recommends the next study content based on the results.

[1982] The "means for updating the study plan according to the user's progress" is a system that dynamically updates the set study plan according to the user's progress in learning.

[1983] "Means for generating test questions based on a specified learning topic using a smart device" refers to technology that uses a mobile or wearable device to create test questions related to a learning topic.

[1984] "Means for analyzing the content explained by a user through voice when generating feedback to the user using generative artificial intelligence" refers to a system in which AI analyzes the learning content explained through voice and generates feedback based on the results.

[1985] This invention is a system that allows users to study efficiently and effectively. The system provides functions such as setting study plans and goals, explaining study content through voice input, analyzing text and providing feedback using generative AI, automatically creating review tests, and recommending next study content.

[1986] Hardware and software used

[1987] Hardware: Smart devices (smartphones, tablets, smart glasses, etc.), servers.

[1988] Software: Speech recognition libraries (e.g. speech_recognition), generative AI libraries (e.g. Huggingface's transformers library).

[1989] Specific functions of the system

[1990] 1. Study plan and goal setting

[1991] Users use the device to set their study plans and goals by entering their goals and selecting relevant study topics through an on-screen interface.

[1992] Example prompt sentence:

[1993] set_learning_plan(goal="Pass the qualification exam", topics=["Grammar", "Reading"])

[1994] 2. Explanation of learning content by voice input

[1995] As the user progresses through the learning process, they explain what they have learned by voice into the device, which uses speech recognition technology to convert this speech into text data.

[1996] Example prompt sentence:

[1997] This week we learned about relative pronouns

[1998] 3. Analysis and feedback by generative AI

[1999] The converted text data is sent to a server where it is analyzed by a generative AI model, which then generates feedback on the user's level of understanding and key points based on the analysis results.

[2000] Example prompt sentence:

[2001] Analyze the following and provide feedback:This week we learned about relative pronouns.

[2002] 4. Automatic creation of review tests

[2003] Based on the feedback, a review test is generated, which corresponds to the learning topic and can be taken on a smart device.

[2004] Example prompt sentence:

[2005] generate_test(["relative pronoun", "subjunctive mood"])

[2006] 5. Evaluation of test results and recommendations for next study

[2007] The test results taken by the user are evaluated on the server and the next study content is recommended, allowing the user to continuously update their optimal study plan according to their own progress.

[2008] Specific examples

[2009] Setting up a study plan

[2010] When User A logs in to the system for the first time, the terminal displays "Please set your study plan and goals." User A enters "I'm aiming for a TOEIC score of 900," and the server receives the information and stores it in the database. The server then automatically generates an appropriate study plan and displays it to the user.

[2011] Voice input explained

[2012] After learning English grammar, User A explains to his / her device, "This week we learned about relative pronouns." The speech is converted into text and sent to the server. The generation AI analyzes the text and generates feedback saying, "You need to relearn the difference between the nominative and possessive case." The server sends this feedback to the device, and User A confirms it again.

[2013] Automatic creation of review tests

[2014] The server generates a test on relative pronouns and sends it to the device. User A takes the test and submits their answers, and the server evaluates the results. The evaluation result is displayed on the device along with feedback such as "You need to review the use of possessives a little more."

[2015] This system allows users to study efficiently and effectively, and provides appropriate feedback and review tests.

[2016] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[2017] Step 1: Setting a study plan and goals

[2018] Input: The user inputs the "goal" and "learning topic" using the terminal.

[2019] Specific operation: The terminal displays a user interface, and the user inputs a goal (e.g., TOEIC score of 900 points) and a learning topic (e.g., listening and reading).

[2020] Output: The server receives these inputs and stores them in a database.

[2021] Data processing: The server organizes the input data and formats it as initial information for automatically generating a learning plan.

[2022] Step 2: Explain the learning content by voice input

[2023] Input: The user speaks what they learned into the device

[2024] Specific operation: When the user speaks a description, the device activates a speech recognition module (e.g., speech_recognition) and converts the speech into text data.

[2025] Output: Text data is sent to the server.

[2026] Data processing: The device converts the voice data into text data and formats it appropriately before sending it to the server.

[2027] Step 3: Analysis and feedback by generative AI

[2028] Input: Text data is sent to the server

[2029] How it works: The server inputs text data into a generative AI model (e.g., Huggingface's transformers), analyzes the content, and generates feedback based on the analysis results.

[2030] Output: The generated feedback is sent to the user terminal.

[2031] Data processing: The server converts the text data into a format suitable for the generative AI model, analyzes the analysis results, and formats them as feedback.

[2032] Step 4: Automatically create review tests

[2033] Input: Information based on generated feedback and learning topics

[2034] Specific operation: The server automatically generates review test questions based on the results of the generative AI model.

[2035] Output: The generated test questions are sent to the user's terminal.

[2036] Data processing: The server combines data to generate test questions based on the learning topic and feedback, and formats it into question format.

[2037] Step 5: Evaluate test results and recommend next study content

[2038] Input: The user takes the review test and enters the answers into the device.

[2039] Specific operation: The device sends the user's test answers to the server, which evaluates the results and recommends the next learning content based on the evaluation results.

[2040] Output: The evaluation results and the next learning content are sent to the user's terminal.

[2041] Data processing: The server analyzes the test answer data, generates evaluation results, and determines the recommended content for the next topic to learn.

[2042] This series of processes allows users to study efficiently and effectively, from setting up a study plan to reviewing the content they have learned, taking review tests, and receiving recommendations for the next study session.

[2043] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[2044] ---

[2045] This invention is a system that allows users to study efficiently and effectively, and provides more personalized feedback by combining an emotion engine that recognizes the user's emotions. This system involves a series of processes in which the user sets a study plan and goals, explains what they have learned, and a generative AI analyzes the content and provides feedback, and automatically creates review tests as needed.

[2046] System configuration

[2047] The system includes the following major components:

[2048] 1. User device: A device used by the user as an interface to set up learning plans, input data by voice, take tests, and perform other operations.

[2049] 2. Server: As a central processing unit, it receives data from users and performs analysis, feedback generation, automatic test generation, etc.

[2050] 3. Generative AI: This runs on the server and includes an algorithm that analyzes the learning content sent by the user and generates evaluations and feedback.

[2051] 4. Emotion Engine: A system for recognizing emotions from the user's voice input and adjusting feedback based on the analysis results.

[2052] Specific functions of the system

[2053] 1. Study plan and goal setting

[2054] When a user logs in to the system, the terminal displays an interface for inputting their learning plan and goals. Through this interface, the user sets their own learning goals (e.g., obtaining a qualification, achieving a specific score, etc.) and creates a learning plan based on them. The server receives this information and stores it in a database.

[2055] 2. Explaining what has been learned and growing the generative AI

[2056] As the user progresses through their learning, the device prompts them to "explain what they learned." The user then explains what they learned aloud, and the device converts that speech into text data. The converted text data is sent to a server, where the AI ​​analyzes the content and generates an analysis result.

[2057] 3. Emotion Recognition and Analysis Using an Emotion Engine

[2058] The emotion engine recognizes the user's emotional state from their voice input. The emotion engine's analysis results are provided to the generative AI, which then adjusts the feedback taking into account the user's emotional state. For example, if the user is feeling anxious, the feedback will be provided in a gentler tone.

[2059] 4. Assessment and feedback of what you learned

[2060] The generative AI evaluates the user's explanation and creates feedback on any errors or insufficient understanding. Taking into account the results of the emotion analysis by the emotion engine, the server compiles the evaluation results and feedback and sends them to the device. Through this feedback, the user can check their level of understanding and make any necessary corrections.

[2061] 5. Automatic creation of review tests

[2062] The AI ​​automatically generates review tests based on the learning content with the highest confidence. The server customizes the tests, setting the difficulty and content according to the user's progress. The tests are sent to the device, and the user can take them to solidify the content they have learned.

[2063] 6. Evaluation of test results and recommendations for next study

[2064] The user takes the test and sends their answers to the server. The server evaluates the test results and stores the scores in a database. Based on the evaluation results, the server generates recommendations for what to study next and what points need to be re-studyed, and sends this to the device.

[2065] Specific examples

[2066] Setting up a study plan

[2067] When User B logs in to the system for the first time, the terminal displays "Please set your study plan and goals." User B enters "My goal is a TOEIC score of 900," and the server receives the information and stores it in the database. The server then automatically generates an appropriate study plan and displays it to the user.

[2068] Explanation of what was learned

[2069] After learning English grammar, User B explains to his device, "This week we learned about relative pronouns." The device converts the speech into text and sends it to the server. The generation AI analyzes the text and generates feedback saying, "You need to relearn the difference between the nominative and possessive." The emotion engine recognizes from User B's voice that "you seem impatient," and provides the feedback in a softer tone.

[2070] Automatic creation of review tests

[2071] The server generates a test on relative pronouns and sends it to the device. User B takes the test and submits their answers, and the server evaluates the results. The evaluation result is displayed on the device along with feedback such as "You need to review the use of possessives a little more."

[2072] ---

[2073] As described above, the present invention is a system that provides a series of functions to efficiently and effectively advance the user's learning process. Through analysis and feedback by generative AI, emotion recognition and adjustment by an emotion engine, and automatic creation of review tests, the system can deepen the user's understanding and solidify their knowledge.

[2074] The processing flow will be explained below.

[2075] ---

[2076] Step 1:

[2077] A user logs in to the system.

[2078] The terminal displays the login interface.

[2079] The user enters their login information and goes through the authentication process.

[2080] Step 2:

[2081] A learning plan and goal setting screen is displayed to the user whose device has been authenticated.

[2082] The user inputs their study plan and goals (e.g., aiming for a TOEIC score of 900).

[2083] The terminal transmits the input information to the server.

[2084] Step 3:

[2085] The server stores the user's learning plan and goals in a database.

[2086] The server generates an initial learning plan and schedule based on the received information.

[2087] The server sends the generated learning plan to the terminal.

[2088] Step 4:

[2089] The user progresses through the learning and is ready to explain what they learned.

[2090] The device prompts the user to "Describe what you learned."

[2091] Audibly describe what the user learned.

[2092] Step 5:

[2093] The terminal converts the user's voice input into text data.

[2094] The terminal transmits the converted text data to the server.

[2095] Step 6:

[2096] The server sends the text data to the emotion engine.

[2097] The emotion engine recognizes the user's emotional state from their voice.

[2098] The emotion engine sends the emotion analysis results to the server.

[2099] Step 7:

[2100] The server inputs the text data into the generation AI and begins analysis.

[2101] The generative AI evaluates the learning content based on text data and sentiment analysis results.

[2102] Step 8:

[2103] The generative AI analyzes the user's explanation and generates evaluation results and feedback.

[2104] The generative AI takes into account the user's emotional state and adjusts the tone and content of the feedback.

[2105] The server formats the generated evaluation results and feedback and sends them to the device.

[2106] Step 9:

[2107] The device displays the feedback to the user.

[2108] The user reviews the feedback and makes any necessary corrections.

[2109] Step 10:

[2110] The server instructs the AI ​​to generate review tests based on the feedback.

[2111] Generative AI determines the content of the review test and automatically creates the test.

[2112] Step 11:

[2113] The server customizes the generated tests and configures them as the user progresses.

[2114] The server sends the test to the device.

[2115] Step 12:

[2116] The device displays a test notification to the user, prompting them to take the test.

[2117] The user performs the test on the device.

[2118] The user enters the answers to the test and the terminal sends the answers to the server.

[2119] Step 13:

[2120] The server evaluates the test results and stores the scores in a database.

[2121] The server generates additional feedback based on the test results.

[2122] Step 14:

[2123] The device displays additional feedback to the user, prompting them to relearn.

[2124] The user re-learns and explains the new learning content to the terminal.

[2125] Step 15:

[2126] The server periodically evaluates the user's learning progress.

[2127] The server automatically updates the learning plan based on progress.

[2128] The server sends recommendations to the device on what to learn next and how to learn it.

[2129] Step 16:

[2130] The device displays recommended information to the user, supporting continuous learning.

[2131] ---

[2132] The above is the specific processing flow of the system, which effectively supports the user's learning process and aims to solidify knowledge through evaluation and feedback by the generative AI, as well as emotion recognition and adjustment by the emotion engine.

[2133] Example 2

[2134] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2135] Conventional learning support systems provide uniform feedback without considering the user's emotional state, making it difficult to provide appropriate instruction tailored to individual needs. Furthermore, evaluation of the user's learning content and recommendations for the next lesson are often done manually, resulting in reduced learning efficiency. Furthermore, in many cases, only standard test questions are provided, resulting in a lack of customized tests tailored to the user's level of understanding. A system that can solve these issues and provide a more personalized learning experience is needed.

[2136] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[2137] In this invention, the server includes means for a user to set a study plan and goals, means for audibly explaining what the user has learned, means for converting the speech into text data, means for inputting the text data into a generative artificial intelligence model and analyzing it, means for evaluating the study content and providing feedback based on the analysis results, means for recognizing the user's emotional state, means for adjusting the feedback taking the user's emotional state into account, means for automatically creating a review test based on the feedback, means for evaluating the test results and recommending the next study content, and means for updating the study plan according to the user's progress. This makes it possible to provide feedback that takes the user's emotional state into account and tests that are customized according to the user's level of understanding.

[2138] A "user" is an individual or group who uses the system to set a learning plan and goals and progress through the learning content.

[2139] "Study planning" is the process of setting specific study content and schedules to be achieved within a specific period based on the study goals that the user wants to achieve.

[2140] A "goal" is a specific outcome or performance that a user wants to achieve through the system.

[2141] "Audio explanation means" is a method by which a user verbally conveys what they have learned to the system.

[2142] "Means for converting into text data" refers to a technology for converting a user's voice input into text format data.

[2143] A "generative artificial intelligence model" is an artificial intelligence algorithm that analyzes data entered by the user and evaluates learning content and generates feedback.

[2144] "Means of analysis" refers to the process of using a generative artificial intelligence model to understand and evaluate the learning content explained by the user.

[2145] "Means for providing feedback" refers to a method for informing users of their learning progress and areas for improvement based on the analysis results.

[2146] "Means for recognizing emotional states" refers to technology for identifying emotions from a user's voice or text data and detecting changes in the user's emotions.

[2147] "Feedback tailoring" is the process of optimizing the content and tone of the feedback provided, taking into account the user's emotional state.

[2148] The "means for automatically creating review tests" is a technology that automatically generates review test questions based on the user's learning content and level of understanding.

[2149] "Means for evaluating test results" refers to the process of analyzing the results of the test taken by the user and evaluating their performance.

[2150] "Means for recommending next learning content" is a process that recommends the next learning content or points that need to be re-learned based on test results and the user's progress.

[2151] The "means for updating the study plan" is a method for appropriately modifying an existing study plan in accordance with the user's progress and updating it to match the latest study goals.

[2152] The present invention provides a learning support system that helps users study efficiently and effectively, and in particular has a feedback function that takes into account the user's emotional state. This system is composed of the following main hardware and software components:

[2153] Hardware and software used

[2154] 1. User Device

[2155] A device that a user uses as an interface to set up a study plan, input data by voice, take tests, receive feedback, etc. Examples include personal computers, smartphones, and tablets. These devices use voice recognition software (e.g., Google Cloud Speech-to-Text) to convert the user's speech into text.

[2156] 2. Server

[2157] As a central processing unit, it receives data from users and performs analysis, creates feedback, automatically generates tests, etc. On the server, generative AI models (e.g., GPT-4) and emotion engines (e.g., Affectiva) run to analyze the user's learning content and emotional state.

[2158] 3. Generative AI Models

[2159] The generative AI model, which runs on a server and includes algorithms that analyze the text data submitted by users, evaluate their learning, generate feedback, and create review tests, is known as a large-scale language model.

[2160] 4. Emotion Engine

[2161] The emotion engine is a system for recognizing the emotional state of a user from their voice input and providing the analysis results to a generative AI model, which then adjusts the feedback according to the user's emotions.

[2162] Specific processing flow and functions

[2163] Study plan and goal setting

[2164] When a user logs in to the system, the terminal displays the message "Please set your study plan and goals." The user enters their study goal (e.g., get a high score on a test) and the terminal sends this information to the server, which receives it and stores it in a database.

[2165] Specific behavior:

[2166] The user inputs "I want to make a study plan with the goal of passing a qualification exam" into the terminal, and the server receives the information and displays a confirmation message to the user.

[2167] Explanation of what was learned

[2168] After the user has progressed through the learning process, the device will prompt them to "explain what you learned." The user will explain aloud, and the device will use speech recognition software to convert the speech into text.

[2169] Specific behavior:

[2170] When a user speaks to the device, "I learned about differential and integral calculus this weekend," the device uses voice recognition software to convert this into text data, "I learned about differential and integral calculus this weekend," and sends it to the server.

[2171] Analysis of learning content and generation of feedback

[2172] The generative AI model on the server analyzes the text data sent by the user and evaluates the learning content. The emotion engine recognizes emotions from the user's voice and provides the results to the generative AI model. The generative AI model adjusts the feedback taking the emotional state into account.

[2173] Specific behavior:

[2174] If the server assesses that the learning content is fully understood and the emotion engine recognizes that the user is showing signs of impatience during the explanation, the generative AI model will provide gentle feedback such as, "You have a solid understanding of the basics of calculus. Let's try some applied problems next."

[2175] Automatic creation of review tests

[2176] Based on the analysis results, the generative AI model automatically generates review tests to improve the user's understanding. The server selects appropriate test questions and sends them to the device.

[2177] Specific behavior:

[2178] The server generates five questions on the basic concepts of calculus, which the terminal displays as a test to the user. The user takes the test and sends the answers to the server.

[2179] Test result evaluation and study recommendations

[2180] The server evaluates the test results and recommends the next study content to the user based on the results, allowing the user to develop an efficient study plan that is in line with their own learning progress.

[2181] Specific behavior:

[2182] After the user completes the test and the server evaluates it, a recommendation is...

Claims

1. a means for users to set their learning plans and goals; a means of providing an audio explanation of what the user has learned; means for converting the voice into text data; A means for inputting the text data into a generating artificial intelligence and performing an analysis; a means for providing evaluation and feedback of the learning content based on the analysis results; means for automatically creating a review test based on the feedback; A means for evaluating the test results and recommending next study content; means for updating the study plan according to the user's progress; A system including:

2. The feedback to the user includes a probabilistic evaluation. The system of claim 1 .

3. The test is customized based on the user's understanding. The system of claim 1 .

Images