system

Abstract

We provide the system. [Solution] Information gathering means for collecting and analyzing student learning information, A means for generating teaching materials that are customized for each student based on the analysis results, A learning support system that provides the generated teaching materials to students and provides real-time feedback to students' questions, A review planning method that monitors students' learning progress and calculates the timing of review based on the forgetting curve, A means for promoting review that provides a notification to encourage students to review based on the aforementioned review plan, To provide emotional support to students and to increase their motivation to learn, A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a persona chatbot control method performed by at least one processor, the method including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In a conventional educational system, it is difficult to individualize according to the learning speed and understanding of each student, and it is particularly difficult to provide effective guidance for children with different monthly age differences and developmental stages. As a result, problems such as students feeling behind in learning and their self-affirmation decreasing occur. In addition, due to the lack of an efficient review plan and emotional support, the fixation of learning content is insufficient. It is necessary to solve these problems.

Means for Solving the Problems

[0005] This invention solves the above problems by developing a system that analyzes students' learning history and abilities in detail and provides individually customized learning materials. This system monitors students' learning activities and automatically creates a review plan based on the forgetting curve using the obtained data. Furthermore, it responds to students' questions in real time, provides appropriate feedback, and actively provides emotional support to improve motivation and promote the retention of learned content.

[0006] "Information gathering means" refers to a device or software that has the function of recording data related to students' learning activities and transmitting it to a server.

[0007] "Method for generating learning materials" refers to a device or software that performs the process of generating customized learning materials based on students' abilities and level of understanding.

[0008] "Learning support tools" refer to devices or software that provide students with learning materials, respond to questions and inquiries in real time, and have functions to assist in learning.

[0009] A "review planning tool" is a device or software that calculates the optimal timing for review based on a student's learning history and then develops a review plan.

[0010] A "review promotion tool" is a device or software that notifies students of the need for review and has the function of effectively reinforcing the learning content.

[0011] "Motivation support tools" are devices or software that provide emotional support to students and generate messages and feedback to maintain and improve their motivation to learn. [Brief explanation of the drawing]

[0012] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2]This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11] This is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] This is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] This is a sequence diagram showing the processing flow of the data processing system in Example 2, when an emotion engine is combined. [Figure 14] This is a sequence diagram showing the processing flow of the data processing system in Application Example 2, which combines an emotion engine. [Modes for carrying out the invention]

[0013] Hereinafter, an example of an embodiment of the system relating to the technology of this disclosure will be described with reference to the attached drawings.

[0014] First, the terms used in the following description will be explained.

[0015] In the following embodiments, the numbered processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0016] In the following embodiments, the numbered RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0017] In the following embodiments, the numbered storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, and the like.

[0018] In the following embodiments, the numbered communication I / F (Interface) is an interface including a communication processor and an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark), and the like.

[0019] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0020] [First Embodiment]

[0021] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0022] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0023] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0024] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0025] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0026] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0027] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0028] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0029] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0030] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0031] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0032] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0033] This invention is a system implemented using a server and terminals to support individualized learning for students. This system consists of information gathering means, material generation means, learning support means, review planning means, review promotion means, and motivation support means. A specific embodiment of this system is described below.

[0034] The server receives data about students' learning activities from their devices. This data includes learning history, correctness of answers, and response time. The server uses this information to analyze students' comprehension and learning trends. Based on the analysis results, the server generates personalized learning materials for each student. These materials include text, images, and audio adjusted to the student's level.

[0035] The terminal displays learning materials provided by the server to the students. As students use the materials and progress through their learning, the terminal provides real-time feedback. For example, if a student enters a question such as, "I don't know how to solve this problem," the terminal, with the assistance of the server, provides an immediate answer.

[0036] The server also continuously monitors learning progress and calculates effective review timings based on Ebbinghaus's forgetting curve. The server notifies the terminal of this review plan, and the terminal prompts students to review at the appropriate time. For example, if the server determines that "it is worth reviewing this unit in one week," it sends a reminder to the student through the terminal.

[0037] Furthermore, the device provides emotional support to students during their studies. This includes displaying messages such as "You did a great job today!" when they finish their studies, thereby boosting their self-esteem and maintaining their motivation to learn.

[0038] Thus, the present invention provides an efficient learning environment tailored to the individual needs of each student, enabling a deep understanding of the learning content and long-term memory retention.

[0039] The following describes the processing flow.

[0040] Step 1:

[0041] The user (student) begins learning using a device. The device records the student's response status in real time, accumulating data such as the correctness of the answer, the time taken to answer, and comments on the problem.

[0042] Step 2:

[0043] Once a certain learning session is complete, the device sends the collected learning data to the server. This data includes the correct answer rate and answer history for each individual question.

[0044] Step 3:

[0045] The server analyzes the received data to determine students' comprehension levels, learning speed, and error patterns. This allows for a detailed understanding of each student's learning progress.

[0046] Step 4:

[0047] Based on the analysis results, the server creates customized learning materials tailored to each student's level. These materials include text, images, and audio, and focus on addressing students' weaknesses.

[0048] Step 5:

[0049] The server sends the generated customized learning materials to the terminals. The terminals display these materials to the students, incorporating interactive elements as needed to provide an environment where students can learn with interest.

[0050] Step 6:

[0051] Users (students) proceed with their learning using the customized learning materials displayed. They can input questions about anything they are unsure of through their device.

[0052] Step 7:

[0053] The device sends a question from the student to the server. The server analyzes the question, generates an appropriate answer, and sends it back to the device. The device then displays this answer to the student.

[0054] Step 8:

[0055] The server continuously monitors students' progress and understanding of the material, and calculates the optimal timing for review based on Ebbinghaus's forgetting curve.

[0056] Step 9:

[0057] The server sends a notification to the device based on the calculated review timing. The device displays a reminder to the student, such as "You need to review a specific unit."

[0058] Step 10:

[0059] The device automatically displays messages such as, "You made great progress today!" to provide emotional support to students during and after learning sessions.

[0060] (Example 1)

[0061] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0062] In modern education, providing learning support tailored to the individual needs of each student is crucial, but in practice, it is difficult to achieve. Existing systems often provide uniform teaching materials to all students, failing to realize effective education based on individual levels of understanding and learning styles. Furthermore, there is insufficient management of review timing and support to sustain students' motivation to learn. In addition, the inability to respond to questions during learning in real time hinders students' understanding.

[0063] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0064] In this invention, the server includes data processing means, content provision means, and information support means. This makes it possible to generate and provide learning materials optimized for each individual student, calculate the timing of review according to the student's learning progress, and encourage review at the appropriate time. Furthermore, by immediately responding to students' questions during learning, it is possible to promote deeper understanding and increase students' motivation to learn.

[0065] "Data processing means" refers to devices or programs for acquiring information about students' learning activities and analyzing that data.

[0066] "Content delivery means" refers to devices or programs that generate and deliver educational materials optimized to the learning needs of individual students.

[0067] "Information support tools" are devices or programs that provide immediate responses to students' questions and doubts during learning, and offer additional information.

[0068] A "schedule management tool" is a device or program used to manage learning progress and calculate the optimal timing for review.

[0069] "Action-promoting measures" refer to devices or programs that send notifications to students encouraging them to review based on calculated review timings.

[0070] "Mental support measures" refer to devices and programs designed to provide students with positive feedback and improve their motivation to learn.

[0071] This invention is a system built by linking a server and terminals to support individualized learning for students. Specific embodiments are shown below.

[0072] The server collects data on students' learning progress and activities. To this end, it receives data from each student's terminal, including their answers, correct / incorrect answers, and the time taken to complete the tasks. The server stores this information in a database and then performs statistical data analysis using the Python pandas library. This allows for the analysis of each student's learning tendencies and level of understanding.

[0073] The server uses a generative AI model to generate learning materials tailored to the student's learning needs. This involves utilizing natural language processing technology and a generative AI model (e.g., GPT-3®) to generate text based on prompts. For example, a prompt could be: "Based on this student's recent learning data, please suggest assignments for the next week." The generated materials are provided as documents, images, and audio, containing content appropriate to the student's progress and level.

[0074] The device displays learning materials provided by the server to the students. If a student has a question during learning, the device receives the question and provides an immediate answer by querying the server. Using Dialogflow or similar chatbot technology would be effective for this.

[0075] The server further manages the review schedule based on Ebbinghaus's forgetting curve theory. It uses scikit-learn algorithms to calculate the optimal review timing and create a plan. The terminal then uses this plan to notify students at the appropriate time, sending messages such as, "Please review the math practice problems tomorrow."

[0076] Finally, the device plays a role in providing positive feedback to students at the end of their learning session, thereby boosting their motivation. Feedback such as, "You did very well on today's lesson!" helps to increase students' self-esteem and improve their motivation to learn.

[0077] Through this system, personalized learning support is provided to each student, enabling effective learning.

[0078] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0079] Step 1:

[0080] The server receives student learning information from terminals. Specifically, the input data includes student answers, correct / incorrect results, and completion time. This data is stored in a database and processed into the format necessary for statistical analysis. The stored data serves as foundational information for future analysis and material generation.

[0081] Step 2:

[0082] The server analyzes students' comprehension using stored data. Input includes past answer data and learning history. A data frame is constructed using the Python pandas library, and average scores and correct answer rates are calculated. The output is a trend analysis report showing each student's strengths and weaknesses. This result helps understand students' learning trends and is used to generate subsequent learning materials.

[0083] Step 3:

[0084] The server uses a generative AI model to create prompts and then generates customized learning materials based on them. The input data includes prompts such as "Please suggest the given math problem," based on a trend analysis report. The generated text, diagrams, and audio files are output and packaged as learning materials displayed to students. These materials are tailored to the individual learning needs of each student.

[0085] Step 4:

[0086] The terminal displays learning materials generated from the server to the students. Students use these materials to progress through their studies. The input consists of the learning material data to be displayed, and if a student asks a question during their study, that question is sent to the server via the terminal. The output consists of replies and additional information received from the server, which are displayed on the student's screen in real time. For example, if a student types "Please explain this problem," the terminal immediately displays the explanation received from the server.

[0087] Step 5:

[0088] The server calculates a review schedule based on Ebbinghaus's forgetting curve. Input includes the student's overall learning progress and past review history, and uses the scikit-learn algorithm to determine the optimal review timing. The output is a schedule specifying the timing and content of reviews, which is generated individually for each student.

[0089] Step 6:

[0090] The device notifies students based on a review schedule sent from the server. The input for the notification includes data on what to review and when, and the device sends reminders to students based on this data. The output is a specific notification displayed on the device, such as "Let's review the math practice problems next Monday."

[0091] Step 7:

[0092] The device provides emotional support when students finish their learning. The input is the student's daily learning performance, and the output displays a motivational message on the screen such as, "You did very well in your learning today!" This helps to sustain students' motivation to learn.

[0093] (Application Example 1)

[0094] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0095] To provide individualized learning support tailored to each student's learning progress and level of understanding, it is necessary to collect diverse information, analyze it in real time, and generate learning materials and provide effective feedback based on that analysis. However, conventional learning support systems have problems such as insufficient individual support and difficulty in optimizing student motivation and review timing. Furthermore, there is a lack of systems for continuously monitoring student learning and maintaining long-term learning effectiveness.

[0096] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0097] In this invention, the server includes data collection means, content generation means, and educational support means. This makes it possible to grasp students' learning progress in real time, provide individually customized learning materials, and notify them of effective review timings and provide emotional support.

[0098] A "data collection method" is a function for collecting students' learning information and recording learning data such as learning progress, answer time, and correct answer rate.

[0099] The "content generation means" is a function for automatically generating individualized learning materials for students based on the results of the analysis of collected data.

[0100] "Educational support tools" refer to functions that provide generated learning materials to students and offer real-time feedback to students' questions during their learning.

[0101] A "review planning tool" is a function that monitors learning progress and calculates the optimal timing for review based on the forgetting curve.

[0102] A "review promotion tool" is a function that provides notifications to students to encourage review based on calculated review timings.

[0103] "Motivation support measures" refer to functions that provide emotional support to students and deliver messages and feedback to enhance their motivation to learn.

[0104] "Communication means" refers to the function that provides the connection for educational equipment to communicate with a server and transmit students' learning activities to the server in real time.

[0105] "Communication control means" refers to a function that manages the communication process for sending learning data from an educational machine to a server and receiving instructions from the server.

[0106] This invention is a system in which a server and a terminal work together to support individualized learning for students. The server collects and analyzes students' learning information and generates individually customized learning materials. This utilizes data collection means and content generation means. Specifically, learning data such as students' response time and correct answer rate are periodically sent to the server. Based on this information, the server grasps the progress of learning in real time and generates multimodal learning materials as needed.

[0107] The terminal displays learning materials provided by the server to the students. Using educational support tools, it provides immediate feedback if students have questions during their studies. In addition, in learning support, it uses a review planning tool to calculate the most effective review timing based on the student's forgetting curve, and sends a notification from the terminal to the student prompting them to review at that time.

[0108] Furthermore, the device is equipped with motivational support features to provide emotional support to students. When a lesson is completed, the device displays a message that affirms the student's daily efforts, maintaining their motivation to learn. For example, if an elementary school student is learning math and gets stuck on a particular problem, the device can immediately answer questions such as, "Can you tell me how to solve this problem?"

[0109] An example of a prompt is a sentence like, "Generate appropriate feedback for students on a specific math problem," and the accuracy of the answer can be improved through a generative AI model.

[0110] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0111] Step 1:

[0112] The server receives student learning information from the terminal. This data includes answer time, accuracy rate, and learning history. The server stores this information in a database for later analysis. This process allows the server to understand each student's current learning status.

[0113] Step 2:

[0114] The server analyzes the collected learning data and uses a generative AI model to evaluate each student's learning tendencies. This analysis determines learning speed and comprehension level, and obtains basic information for generating personalized educational content accordingly. Specific data processing includes evaluating learning speed and extracting patterns from incorrect answers.

[0115] Step 3:

[0116] The server uses content generation tools to create individually optimized learning materials for each student. This includes a process that combines text, images, and audio according to the student's learning level to generate the materials. The generated materials are customized to deepen the student's understanding. The generated materials are then ready for transmission to the terminal.

[0117] Step 4:

[0118] The terminal receives learning materials from the server and presents them to the students. Through its user interface, the terminal enhances learning effectiveness using visual and auditory stimuli. If students have questions, the terminal uses educational support tools to send queries to the server and generate feedback. Because this feedback is provided in real time, it is expected to improve students' learning efficiency.

[0119] Step 5:

[0120] The server uses review planning tools to monitor students' learning progress and calculate review timing based on the forgetting curve. This calculation identifies when it would be most effective for students to review. This plan is then ready to be communicated to students via their devices.

[0121] Step 6:

[0122] The device displays reminders to students when it's time for review. These reminders are designed to encourage repetition of learned material and aid in long-term retention. Furthermore, the device utilizes motivational support features, displaying messages praising students' efforts at the end of their learning sessions to maintain their enthusiasm.

[0123] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0124] This invention provides individualized learning tailored to each student's learning progress and level of understanding, and further combines this with an emotion recognition engine to create a more effective learning environment. Specific embodiments are described below.

[0125] The server receives data from the terminal regarding the student's learning progress. This data includes the student's answers to problems, the time it took to answer them, and changes in the student's facial expressions and voice obtained from the emotion recognition engine. Based on this data, the server analyzes the student's current level of learning comprehension and emotional state.

[0126] The emotion recognition engine is built into the device and performs facial and voice analysis on students to detect their emotional state in real time. This information is sent to a server and used to understand students' stress levels and analyze their motivation in the current learning content.

[0127] The server considers the information obtained through emotion recognition and customizes the learning materials. For example, if a student is feeling stressed, it adjusts the difficulty level or adds interactive elements to help them relax. It also generates motivational messages based on emotional information and delivers them to students through their devices to improve their motivation to learn.

[0128] The device displays learning materials provided to students and supports real-time interaction. When students input questions or comments during learning, the device transmits them to the server, which immediately generates answers and provides feedback. It also periodically updates sentiment data, which is used to optimize the students' learning experience.

[0129] As users (students) learn using their devices, they become aware that their emotional state is being taken into consideration, leading to a more positive learning experience. For example, when they are tired, the device sends a reminder to pause their studies and take a break.

[0130] Thus, the present invention is a system that provides flexible learning support that takes into account the emotional state of learners, and can improve students' motivation and learning efficiency.

[0131] The following describes the processing flow.

[0132] Step 1:

[0133] The user (student) begins learning using the device. The device collects emotional data in real time by capturing the student's facial expressions with its camera and recording their voice with its microphone.

[0134] Step 2:

[0135] The device inputs collected video and audio data into an emotion recognition engine to analyze the students' emotional state. The analysis results quantify the students' stress levels, motivation levels, and other factors.

[0136] Step 3:

[0137] The device sends the results of the emotion recognition engine along with the training data to the server. The server comprehensively analyzes the student's learning progress and emotional information.

[0138] Step 4:

[0139] The server customizes learning materials to suit each student, taking their emotional state into consideration. For example, if a student is feeling stressed, it prioritizes generating content that promotes relaxation.

[0140] Step 5:

[0141] The server sends the generated customized learning materials to the terminal. The terminal then presents these materials to the student and continuously adjusts the content according to the student's learning progress.

[0142] Step 6:

[0143] The user (student) continues their learning activities using the learning materials displayed on the device. The device provides an interface that accepts input in response to the student's questions.

[0144] Step 7:

[0145] The terminal sends questions from students to the server. The server analyzes the questions, generates appropriate answers and additional materials, and then sends them back to the terminal.

[0146] Step 8:

[0147] The server uses Ebbinghaus's forgetting curve to calculate the most effective timing for review, based on students' learning history and emotional data.

[0148] Step 9:

[0149] The server sends a notification to the device indicating when it's time to review. The device then displays a review reminder to the student at the appropriate time.

[0150] Step 10:

[0151] The device displays motivational messages based on the student's emotional state during learning or when they need a break. This helps maintain the student's motivation to learn and reduces mental stress.

[0152] (Example 2)

[0153] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0154] In today's educational environment, the importance of individualized learning support that takes into account each student's level of understanding and emotional state is increasing. However, conventional systems have struggled to appropriately grasp students' emotional states and effectively customize learning materials and provide motivational support accordingly. To address this problem, a new system is needed that can analyze students' emotions in more detail and optimize their learning experience in real time.

[0155] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0156] In this invention, the server includes an information gathering means for collecting and analyzing student learning information, a material generation means for generating individually customized learning materials for each student based on the analysis results, and a motivation enhancement means for generating and providing motivational messages using a generation AI model. This enables flexible and effective learning support and motivation enhancement that is tailored to each student's level of learning comprehension and emotional state.

[0157] "Information gathering means" refers to a device or software used to acquire students' learning information and emotional state, and to transmit that information to a server.

[0158] "Method for generating learning materials" refers to a device or software for creating individually customized learning materials based on students' learning comprehension levels and emotional states.

[0159] A "learning support tool" is a device or software that has the function of responding to students' questions in real time based on the learning materials provided to them.

[0160] A "review planning tool" is a device or software that monitors students' learning progress and calculates and suggests the optimal timing for review.

[0161] A "review promotion tool" is a device or software that notifies students to review based on the timing calculated by the review planning tool.

[0162] An "emotional analysis tool" is a device or software that analyzes students' facial expressions and voices to evaluate their emotional state and allows for adjustments to learning content based on the results.

[0163] A "motivation enhancement tool" is a device or software that uses a generative AI model to create and deliver messages that enhance students' motivation to learn.

[0164] A "generative AI model" is an artificial intelligence computational model used to generate optimal messages and learning materials tailored to the student's situation.

[0165] This invention is a system that provides students with an individualized learning experience by taking into account their learning progress and level of understanding, and by combining this with emotion recognition technology. The system mainly consists of three components: a server, a terminal, and a user.

[0166] The server plays a role in aggregating and analyzing student learning information. It receives student answer results and answer times transmitted from terminals, as well as sentiment data acquired via an emotion recognition engine. Cloud-based computing technology and database management systems are used for analysis. Specifically, AI models are used to evaluate the accuracy and comprehension of answers, and learning content is adjusted based on sentiment data. For example, if a student shows anxiety about a math problem, the learning materials are adjusted to present easier problems to that student.

[0167] The terminal is a device directly operated by students, supporting the display of learning materials and real-time interaction. The terminal has a built-in camera and microphone, which analyze students' facial expressions and voices, sending emotion recognition data to the server. The terminal itself uses a local program and a simple data analysis module for processing. Furthermore, the terminal can instantly display feedback provided by the server, and can include prompts such as, "Please suggest ways to adjust the learning materials when the student is tired."

[0168] Users (students) use their devices to progress through personalized learning materials. As students learn, they receive real-time feedback from their devices, which helps them adjust their learning plan. They can also input questions and requests into their devices to receive quick feedback from the server. This allows users to learn at their own pace and according to their emotions, providing an efficient and comfortable learning environment.

[0169] This system aims to maximize students' personalized learning experiences by enabling real-time content adjustments based on detailed data, including emotional states. Utilizing generative AI models, it consistently provides optimal support, thereby improving learning motivation and optimizing learning efficiency.

[0170] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0171] Step 1:

[0172] The device collects user (student) information at the start of learning. Specifically, it analyzes the student's facial expressions and voice through the camera and microphone. This allows for real-time understanding of their emotional state, and also records the time taken to answer questions and the results of those questions during learning. This data is sent to the server. The input is data on the student's behavior during learning, and the output is the data sent to the server.

[0173] Step 2:

[0174] The server receives learning information and emotional state data transmitted from the terminal. Cloud computing technology is used to analyze the received data. An AI model analyzes this data and evaluates learning comprehension and emotional state. The data input is student information from the terminal, and the output is a report of the analyzed learning comprehension and emotional state.

[0175] Step 3:

[0176] The server customizes the learning materials based on the analysis results obtained. Specifically, it uses a generative AI model to generate materials suitable for the learning progress. Here, difficulty levels can be adjusted and interactive elements can be introduced. The input is the analysis results, and the output is the customized learning materials.

[0177] Step 4:

[0178] The server generates motivational messages along with the learning materials. This also utilizes a generative AI model to select appropriate words and construct the messages. The input consists of customized learning materials and emotional information, while the output is the learning materials sent along with the motivational messages.

[0179] Step 5:

[0180] The terminal displays customized learning materials and motivational messages sent from the server to the user (student). As the user progresses through the materials, they can input further questions and comments. Input consists of learning materials from the server and user feedback, while output consists of the display to the user and feedback data sent to the server.

[0181] Step 6:

[0182] The device constantly monitors the user's facial expressions and learning progress, and sends any new data obtained to the server. This allows the server to dynamically adjust the learning progress. The input is new reaction data during learning, and the output is the transmission of updated data to the server.

[0183] (Application Example 2)

[0184] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0185] Traditional learning support systems can provide customized learning materials based on students' learning progress and understanding, but they lack the ability to assess students' emotions and motivation in real time. As a result, learners may experience stress or decreased motivation. To maximize learning efficiency, a system is needed that provides appropriate learning materials and feedback while taking into account the learner's emotional state.

[0186] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0187] In this invention, the server includes information gathering means, material generation means, learning support means, review planning means, review promotion means, motivation support means, and adjustment means that evaluate the student's emotional state using facial recognition technology and dynamically adjust the learning content. This enables comprehensive learning optimization that takes into account not only the student's learning progress but also their emotional state.

[0188] "Information gathering methods" refer to systems that collect answer results, answer times, facial expression data, and audio data in order to understand students' learning progress and emotional state.

[0189] The "teaching material generation method" is a function that generates customized teaching materials tailored to each student's level of understanding and emotional state, based on collected data.

[0190] "Learning support methods" refer to the process of presenting generated learning materials to students and supporting their learning through real-time question-and-answer sessions and feedback.

[0191] A "review planning tool" is a function that analyzes students' learning progress and calculates the appropriate timing for review based on the forgetting curve.

[0192] A "review promotion tool" is a system that provides notifications to encourage students to review appropriately based on their review plan.

[0193] "Motivation support methods" refer to the process of generating messages that take students' emotions into consideration and providing support to enhance their motivation to learn.

[0194] The "adjustment mechanism" is a function that uses facial recognition technology to analyze students' emotional states in real time and dynamically adjust the difficulty level of the learning content.

[0195] In the system that realizes this invention, a program is embedded in a consumer robot for home use. The server analyzes students' learning data and emotional information in real time and generates individually customized learning plans. As a means of information gathering, the robot uses its built-in camera and microphone, analyzes facial expressions using the Google® Cloud Vision API, and converts speech to text using Google Cloud Speech-to-Text. This allows the robot to understand the emotional state of the students.

[0196] The terminal records the results and response time of the questions answered by the student and sends this data to the server. Based on this data, the server provides appropriate learning materials to the student using a material generation system. The materials are in a multimodal format, including text, still images, and audio. The learning support system enables real-time question answering, allowing students to receive immediate feedback whenever they ask a question.

[0197] Furthermore, by providing emotional support through motivational support mechanisms, students' motivation to learn can be enhanced. When students are tired or stressed, the robot will suggest taking a break in a gentle voice or deliver an encouraging message. For example, if a third-grade elementary school student gets stuck on a difficult math problem, the robot might suggest, "How about taking a short break? Or would you like a hint?"

[0198] An example of a prompt is: "The robot will analyze the child's facial expressions and voice and flexibly adjust the learning content. If the child is having trouble with a difficult problem, please think about what to do." Using this prompt, a generative AI model can suggest an appropriate intervention method.

[0199] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0200] Step 1:

[0201] The device collects student facial expression data captured by its built-in camera and audio data captured by its microphone in real time. This data is sent as input to the Google Cloud Vision API and the Google Cloud Speech-to-Text API. This process converts the image data into facial expression information and the audio data into text information.

[0202] Step 2:

[0203] The server analyzes the facial expression information and audio text information obtained in Step 1 to evaluate the student's emotional state. This analysis uses a generative AI model to predict the student's emotional state using prompt sentences. For example, under the condition "the student is confused," the prediction result is output.

[0204] Step 3:

[0205] Based on the analysis results from Step 2, the server generates learning materials tailored to each student using a material generation method. In this process, a generation AI model is used to create multimodal materials including text, images, and audio with adjusted difficulty levels. The inputs used are the student's emotional assessment data and previous learning history, and the output material content is based on this information.

[0206] Step 4:

[0207] The terminal displays customized learning materials sent from the server to the student. It records the student's response time and results when solving problems and sends this data back to the server. This data is processed by the server as input data necessary for generating the next set of learning materials.

[0208] Step 5:

[0209] The server aggregates answer data and emotional data, and uses a review planning tool to calculate review timing based on the forgetting curve. Based on this calculation result, information is output to suggest the optimal review date.

[0210] Step 6:

[0211] The device provides students with review reminders based on the review information calculated in step 5. Students, as users, can then receive these notifications and perform their reviews at the appropriate time.

[0212] Step 7:

[0213] With the aim of providing emotional support to users, the server generates feedback through motivational support methods and delivers it via the terminal. Based on the data (input) obtained through emotion recognition, it optimizes and outputs emotional support messages, such as "How about taking a short break?"

[0214] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0215] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0216] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0217] [Second Embodiment]

[0218] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0219] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0220] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0221] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0222] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0223] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0224] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0225] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0226] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0227] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0228] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0229] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0230] This invention is a system implemented using a server and terminals to support individualized learning for students. This system consists of information gathering means, material generation means, learning support means, review planning means, review promotion means, and motivation support means. A specific embodiment of this system is described below.

[0231] The server receives data about students' learning activities from their devices. This data includes learning history, correctness of answers, and response time. The server uses this information to analyze students' comprehension and learning trends. Based on the analysis results, the server generates personalized learning materials for each student. These materials include text, images, and audio adjusted to the student's level.

[0232] The terminal displays learning materials provided by the server to the students. As students use the materials and progress through their learning, the terminal provides real-time feedback. For example, if a student enters a question such as, "I don't know how to solve this problem," the terminal, with the assistance of the server, provides an immediate answer.

[0233] The server also continuously monitors learning progress and calculates effective review timings based on Ebbinghaus's forgetting curve. The server notifies the terminal of this review plan, and the terminal prompts students to review at the appropriate time. For example, if the server determines that "it is worth reviewing this unit in one week," it sends a reminder to the student through the terminal.

[0234] Furthermore, the device provides emotional support to students during their studies. This includes displaying messages such as "You did a great job today!" when they finish their studies, thereby boosting their self-esteem and maintaining their motivation to learn.

[0235] Thus, the present invention provides an efficient learning environment tailored to the individual needs of each student, enabling a deep understanding of the learning content and long-term memory retention.

[0236] The following describes the processing flow.

[0237] Step 1:

[0238] The user (student) begins learning using a device. The device records the student's response status in real time, accumulating data such as the correctness of the answer, the time taken to answer, and comments on the problem.

[0239] Step 2:

[0240] Once a certain learning session is complete, the device sends the collected learning data to the server. This data includes the correct answer rate and answer history for each individual question.

[0241] Step 3:

[0242] The server analyzes the received data to determine students' comprehension levels, learning speed, and error patterns. This allows for a detailed understanding of each student's learning progress.

[0243] Step 4:

[0244] Based on the analysis results, the server creates customized learning materials tailored to each student's level. These materials include text, images, and audio, and focus on addressing students' weaknesses.

[0245] Step 5:

[0246] The server sends the generated customized learning materials to the terminals. The terminals display these materials to the students, incorporating interactive elements as needed to provide an environment where students can learn with interest.

[0247] Step 6:

[0248] Users (students) proceed with their learning using the customized learning materials displayed. They can input questions about anything they are unsure of through their device.

[0249] Step 7:

[0250] The device sends a question from the student to the server. The server analyzes the question, generates an appropriate answer, and sends it back to the device. The device then displays this answer to the student.

[0251] Step 8:

[0252] The server continuously monitors students' progress and understanding of the material, and calculates the optimal timing for review based on Ebbinghaus's forgetting curve.

[0253] Step 9:

[0254] The server sends a notification to the device based on the calculated review timing. The device displays a reminder to the student, such as "You need to review a specific unit."

[0255] Step 10:

[0256] The device automatically displays messages such as, "You made great progress today!" to provide emotional support to students during and after learning sessions.

[0257] (Example 1)

[0258] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0259] In modern education, providing learning support tailored to the individual needs of each student is crucial, but in practice, it is difficult to achieve. Existing systems often provide uniform teaching materials to all students, failing to realize effective education based on individual levels of understanding and learning styles. Furthermore, there is insufficient management of review timing and support to sustain students' motivation to learn. In addition, the inability to respond to questions during learning in real time hinders students' understanding.

[0260] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0261] In this invention, the server includes data processing means, content provision means, and information support means. This makes it possible to generate and provide learning materials optimized for each individual student, calculate the timing of review according to the student's learning progress, and encourage review at the appropriate time. Furthermore, by immediately responding to students' questions during learning, it is possible to promote deeper understanding and increase students' motivation to learn.

[0262] "Data processing means" refers to devices or programs for acquiring information about students' learning activities and analyzing that data.

[0263] "Content delivery means" refers to devices or programs that generate and deliver educational materials optimized to the learning needs of individual students.

[0264] "Information support tools" are devices or programs that provide immediate responses to students' questions and doubts during learning, and offer additional information.

[0265] A "schedule management tool" is a device or program used to manage learning progress and calculate the optimal timing for review.

[0266] "Action-promoting measures" refer to devices or programs that send notifications to students encouraging them to review based on calculated review timings.

[0267] "Mental support measures" refer to devices and programs designed to provide students with positive feedback and improve their motivation to learn.

[0268] This invention is a system built by linking a server and terminals to support individualized learning for students. Specific embodiments are shown below.

[0269] The server collects data on students' learning progress and activities. To this end, it receives data from each student's terminal, including their answers, correct / incorrect answers, and the time taken to complete the tasks. The server stores this information in a database and then performs statistical data analysis using the Python pandas library. This allows for the analysis of each student's learning tendencies and level of understanding.

[0270] The server uses a generative AI model to generate learning materials tailored to the student's learning needs. This involves utilizing natural language processing technology and a generative AI model (e.g., GPT-3) to generate text based on prompts. For example, a prompt could be: "Based on this student's recent learning data, please suggest assignments for the next week." The generated materials are provided as documents, images, and audio, containing content appropriate to the student's progress and level.

[0271] The device displays learning materials provided by the server to the students. If a student has a question during learning, the device receives the question and provides an immediate answer by querying the server. Using Dialogflow or similar chatbot technology would be effective for this.

[0272] The server further manages the review schedule based on Ebbinghaus's forgetting curve theory. It uses scikit-learn algorithms to calculate the optimal review timing and create a plan. The terminal then uses this plan to notify students at the appropriate time, sending messages such as, "Please review the math practice problems tomorrow."

[0273] Finally, the device plays a role in providing positive feedback to students at the end of their learning session, thereby boosting their motivation. Feedback such as, "You did very well on today's lesson!" helps to increase students' self-esteem and improve their motivation to learn.

[0274] Through this system, personalized learning support is provided to each student, enabling effective learning.

[0275] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0276] Step 1:

[0277] The server receives student learning information from terminals. Specifically, the input data includes student answers, correct / incorrect results, and completion time. This data is stored in a database and processed into the format necessary for statistical analysis. The stored data serves as foundational information for future analysis and material generation.

[0278] Step 2:

[0279] The server analyzes students' comprehension using stored data. Input includes past answer data and learning history. A dataframe is constructed using the Python pandas library, and average scores and correct answer rates are calculated. The output is a trend analysis report showing each student's strengths and weaknesses. This result helps understand students' learning trends and is used to generate subsequent learning materials.

[0280] Step 3:

[0281] The server uses a generative AI model to create prompts and then generates customized learning materials based on them. The input data includes prompts such as "Please suggest the given math problem," based on a trend analysis report. The generated text, diagrams, and audio files are output and packaged as learning materials displayed to students. These materials are tailored to the individual learning needs of each student.

[0282] Step 4:

[0283] The terminal displays the teaching materials generated by the server to the students. The students use these to proceed with their learning. As input, there is the teaching material data to be displayed. When a student asks a question during learning, the question is sent to the server via the terminal. As output, the replies and additional information obtained from the server are presented to the student's screen in real time. As a specific example, when a student inputs "Please explain this problem", the terminal immediately displays the explanation received from the server.

[0284] Step 5:

[0285] The server calculates a review schedule based on Ebbinghaus' forgetting curve. As input quantities, there is the overall learning progress information of the students and their past review history, and the optimal review time is calculated using the algorithms of scikit-learn. The output is a schedule indicating the timing and content of the review, which is generated individually for each student.

[0286] Step 6:

[0287] The terminal notifies the students based on the review schedule sent from the server. As input for the notification, there is data regarding the content and timing to be reviewed, and based on this, a reminder is sent to the students. The output is a specific notification such as "Let's review the math practice problems on Monday next week." displayed on the terminal.

[0288] Step 7:

[0289] The terminal provides emotional support when the student finishes learning. The input is the student's one-day learning performance, and as output, a message that boosts motivation such as "You did very well in today's learning!" is displayed on the screen. This makes it possible to continuously stimulate the student's learning motivation.

[0290] (Application Example 1)

[0291] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0292] To provide individualized learning support tailored to each student's learning progress and level of understanding, it is necessary to collect diverse information, analyze it in real time, and generate learning materials and provide effective feedback based on that analysis. However, conventional learning support systems have problems such as insufficient individual support and difficulty in optimizing student motivation and review timing. Furthermore, there is a lack of systems for continuously monitoring student learning and maintaining long-term learning effectiveness.

[0293] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0294] In this invention, the server includes data collection means, content generation means, and educational support means. This makes it possible to grasp students' learning progress in real time, provide individually customized learning materials, and notify them of effective review timings and provide emotional support.

[0295] A "data collection method" is a function for collecting students' learning information and recording learning data such as learning progress, answer time, and correct answer rate.

[0296] The "content generation means" is a function for automatically generating individualized learning materials for students based on the results of the analysis of collected data.

[0297] "Educational support tools" refer to functions that provide generated learning materials to students and offer real-time feedback to students' questions during their learning.

[0298] A "review planning tool" is a function that monitors learning progress and calculates the optimal timing for review based on the forgetting curve.

[0299] A "review promotion tool" is a function that provides notifications to students to encourage review based on calculated review timings.

[0300] "Motivation support measures" refer to functions that provide emotional support to students and deliver messages and feedback to enhance their motivation to learn.

[0301] "Communication means" refers to the function that provides the connection for educational equipment to communicate with a server and transmit students' learning activities to the server in real time.

[0302] "Communication control means" refers to a function that manages the communication process for sending learning data from an educational machine to a server and receiving instructions from the server.

[0303] This invention is a system in which a server and a terminal work together to support individualized learning for students. The server collects and analyzes students' learning information and generates individually customized learning materials. This utilizes data collection means and content generation means. Specifically, learning data such as students' response time and correct answer rate are periodically sent to the server. Based on this information, the server grasps the progress of learning in real time and generates multimodal learning materials as needed.

[0304] The terminal displays learning materials provided by the server to the students. Using educational support tools, it provides immediate feedback if students have questions during their studies. In addition, in learning support, it uses a review planning tool to calculate the most effective review timing based on the student's forgetting curve, and sends a notification from the terminal to the student prompting them to review at that time.

[0305] Furthermore, the terminal is equipped with motivation support means to provide emotional support to students. When learning ends, the terminal displays a message that affirms the students' daily efforts, maintaining their learning motivation. As a specific example, when an elementary school student is learning arithmetic and encounters difficulties with a particular problem, the terminal can immediately answer questions such as "Please teach me the solution to this problem."

[0306] As an example of a prompt sentence, there is a sentence like "Generate appropriate feedback for students on a specific math problem," and the accuracy of the answer can be improved through the generative AI model.

[0307] The flow of the specific process in Application Example 1 will be described using FIG. 12.

[0308] Step 1:

[0309] The server receives the learning information of the students from the terminal. This data includes the answering time, correct answer rate, and learning history. The server stores this information in a database for later analysis. Through this process, the latest learning status of each student can be grasped.

[0310] Step 2:

[0311] The server analyzes the collected learning data and uses the generative AI model to evaluate the learning tendencies of each student. In this analysis, the learning speed and comprehension are determined to obtain basic information for generating individualized educational content accordingly. Specific data processing includes evaluating the learning speed and extracting the patterns of incorrect questions.

[0312] Step 3:

[0313] The server uses content generation tools to create individually optimized learning materials for each student. This includes a process that combines text, images, and audio according to the student's learning level to generate the materials. The generated materials are customized to deepen the student's understanding. The generated materials are then ready for transmission to the terminal.

[0314] Step 4:

[0315] The terminal receives learning materials from the server and presents them to the students. Through its user interface, the terminal enhances learning effectiveness using visual and auditory stimuli. If students have questions, the terminal uses educational support tools to send queries to the server and generate feedback. Because this feedback is provided in real time, it is expected to improve students' learning efficiency.

[0316] Step 5:

[0317] The server uses a review planning system to monitor students' learning progress and calculate review timing based on the forgetting curve. This calculation identifies when it would be most effective for students to review. This plan is then ready to be communicated to students via their devices.

[0318] Step 6:

[0319] The device displays reminders to students when it's time for review. These reminders are designed to encourage repetition of learned material and aid in long-term retention. Furthermore, the device utilizes motivational support features, displaying messages praising students' efforts at the end of their learning sessions to maintain their enthusiasm.

[0320] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0321] This invention provides individualized learning tailored to each student's learning progress and level of understanding, and further combines this with an emotion recognition engine to create a more effective learning environment. Specific embodiments are described below.

[0322] The server receives data from the terminal regarding the student's learning progress. This data includes the student's answers to problems, the time it takes to answer them, and changes in the student's facial expressions and voice obtained from the emotion recognition engine. Based on this data, the server analyzes the student's current level of learning comprehension and emotional state.

[0323] The emotion recognition engine is built into the device and performs facial and voice analysis on students to detect their emotional state in real time. This information is sent to a server and used to understand students' stress levels and analyze their motivation in the current learning content.

[0324] The server considers the information obtained through emotion recognition and customizes the learning materials. For example, if a student is feeling stressed, it adjusts the difficulty level or adds interactive elements to help them relax. It also generates motivational messages based on emotional information and delivers them to students through their devices to improve their motivation to learn.

[0325] The device displays learning materials provided to students and supports real-time interaction. When students input questions or comments during learning, the device transmits them to the server, which immediately generates answers and provides feedback. It also periodically updates sentiment data, which is used to optimize the students' learning experience.

[0326] As users (students) learn using their devices, they become aware that their emotional state is being taken into consideration, leading to a more positive learning experience. For example, when they are tired, the device sends a reminder to pause their studies and take a break.

[0327] Thus, the present invention is a system that provides flexible learning support that takes into account the emotional state of learners, and can improve students' motivation and learning efficiency.

[0328] The following describes the processing flow.

[0329] Step 1:

[0330] The user (student) begins learning using the device. The device collects emotional data in real time by capturing the student's facial expressions with its camera and recording their voice with its microphone.

[0331] Step 2:

[0332] The device inputs the collected video and audio data into an emotion recognition engine to analyze the students' emotional state. The analysis results quantify the students' stress levels, motivation levels, and other factors.

[0333] Step 3:

[0334] The device sends the results of the emotion recognition engine along with the training data to the server. The server comprehensively analyzes the student's learning progress and emotional information.

[0335] Step 4:

[0336] The server customizes learning materials to suit each student, taking their emotional state into consideration. For example, if a student is feeling stressed, it prioritizes generating content that promotes relaxation.

[0337] Step 5:

[0338] The server sends the generated customized learning materials to the terminal. The terminal then presents these materials to the student and continuously adjusts the content according to the student's learning progress.

[0339] Step 6:

[0340] The user (student) continues their learning activities using the learning materials displayed on the device. The device provides an interface that accepts input in response to the student's questions.

[0341] Step 7:

[0342] The terminal sends questions from students to the server. The server analyzes the questions, generates appropriate answers and additional materials, and then sends them back to the terminal.

[0343] Step 8:

[0344] The server uses Ebbinghaus's forgetting curve to calculate the most effective timing for review, based on students' learning history and emotional data.

[0345] Step 9:

[0346] The server sends a notification to the device indicating when it's time to review. The device then displays a review reminder to the student at the appropriate time.

[0347] Step 10:

[0348] The device displays motivational messages based on the student's emotional state during study sessions or when breaks are needed. This helps maintain the student's motivation to learn and reduces mental stress.

[0349] (Example 2)

[0350] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0351] In today's educational environment, the importance of individualized learning support that takes into account each student's level of understanding and emotional state is increasing. However, conventional systems have struggled to appropriately grasp students' emotional states and effectively customize learning materials and provide motivational support accordingly. To address this problem, a new system is needed that can analyze students' emotions in more detail and optimize their learning experience in real time.

[0352] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0353] In this invention, the server includes an information gathering means for collecting and analyzing student learning information, a material generation means for generating individually customized learning materials for each student based on the analysis results, and a motivation enhancement means for generating and providing motivational messages using a generation AI model. This enables flexible and effective learning support and motivation enhancement that is tailored to each student's level of learning comprehension and emotional state.

[0354] "Information gathering means" refers to devices or software used to acquire students' learning information and emotional states and transmit them to a server.

[0355] "Method for generating learning materials" refers to a device or software for creating individually customized learning materials based on students' learning comprehension levels and emotional states.

[0356] A "learning support tool" is a device or software that has the function of responding to students' questions in real time based on the learning materials provided to them.

[0357] A "review planning tool" is a device or software that monitors students' learning progress and calculates and suggests the optimal timing for review.

[0358] A "review promotion tool" is a device or software that notifies students to review based on the timing calculated by the review planning tool.

[0359] An "emotional analysis tool" is a device or software that analyzes students' facial expressions and voices to evaluate their emotional state and allows for adjustments to learning content based on the results.

[0360] A "motivation enhancement tool" is a device or software that uses a generative AI model to create and deliver messages that enhance students' motivation to learn.

[0361] A "generative AI model" is an artificial intelligence computational model used to generate optimal messages and learning materials tailored to the student's situation.

[0362] This invention is a system that provides students with an individualized learning experience by taking into account their learning progress and level of understanding, and by combining this with emotion recognition technology. The system mainly consists of three components: a server, a terminal, and a user.

[0363] The server plays a role in aggregating and analyzing student learning information. It receives student answer results and answer times transmitted from terminals, as well as sentiment data acquired via an emotion recognition engine. Cloud-based computing technology and database management systems are used for analysis. Specifically, AI models are used to evaluate the accuracy and comprehension of answers, and learning content is adjusted based on sentiment data. For example, if a student shows anxiety about a math problem, the learning materials are adjusted to present easier problems to that student.

[0364] The terminal is a device directly operated by students, supporting the display of learning materials and real-time interaction. The terminal has a built-in camera and microphone, which analyze students' facial expressions and voices, sending emotion recognition data to the server. The terminal itself uses a local program and a simple data analysis module for processing. Furthermore, the terminal can instantly display feedback provided by the server, and can include prompts such as, "Please suggest ways to adjust the learning materials when the student is tired."

[0365] Users (students) use their devices to progress through personalized learning materials. As students learn, they receive real-time feedback from their devices, which helps them adjust their learning plan. They can also input questions and requests into their devices to receive quick feedback from the server. This allows users to learn at their own pace and according to their emotions, providing an efficient and comfortable learning environment.

[0366] This system aims to maximize students' personalized learning experiences by enabling instant content adjustments based on detailed data, including emotional states. Utilizing generative AI models, it consistently provides optimal support, thereby improving learning motivation and optimizing learning efficiency.

[0367] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0368] Step 1:

[0369] The device collects user (student) information at the start of learning. Specifically, it analyzes the student's facial expressions and voice through the camera and microphone. This allows for real-time understanding of their emotional state, and also records the time taken to answer questions and the results of those questions during learning. This data is sent to the server. The input is data on the student's behavior during learning, and the output is the data sent to the server.

[0370] Step 2:

[0371] The server receives learning information and emotional state data transmitted from the terminal. Cloud computing technology is used to analyze the received data. An AI model analyzes this data and evaluates learning comprehension and emotional state. The data input is student information from the terminal, and the output is a report of the analyzed learning comprehension and emotional state.

[0372] Step 3:

[0373] The server customizes the learning materials based on the analysis results obtained. Specifically, it uses a generative AI model to generate materials suitable for the learning progress. Here, difficulty levels can be adjusted and interactive elements can be introduced. The input is the analysis results, and the output is the customized learning materials.

[0374] Step 4:

[0375] The server generates motivational messages along with the learning materials. This also utilizes a generative AI model to select appropriate words and construct the messages. The input consists of customized learning materials and emotional information, while the output is the learning materials sent along with the motivational messages.

[0376] Step 5:

[0377] The terminal displays customized learning materials and motivational messages sent from the server to the user (student). As the user progresses through the materials, they can input further questions and comments. Input consists of learning materials from the server and user feedback, while output consists of the display to the user and feedback data sent to the server.

[0378] Step 6:

[0379] The device constantly monitors the user's facial expressions and learning progress, and sends any new data obtained to the server. This allows the server to dynamically adjust the learning progress. The input is new reaction data during learning, and the output is the transmission of updated data to the server.

[0380] (Application Example 2)

[0381] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0382] Traditional learning support systems can provide customized learning materials based on students' learning progress and understanding, but they lack the ability to assess students' emotions and motivation in real time. As a result, learners may experience stress or decreased motivation. To maximize learning efficiency, a system is needed that provides appropriate learning materials and feedback while taking into account the learner's emotional state.

[0383] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0384] In this invention, the server includes information gathering means, material generation means, learning support means, review planning means, review promotion means, motivation support means, and adjustment means that evaluate the student's emotional state using facial recognition technology and dynamically adjust the learning content. This enables comprehensive learning optimization that takes into account not only the student's learning progress but also their emotional state.

[0385] "Information gathering methods" refer to systems that collect answer results, answer times, facial expression data, and audio data in order to understand students' learning progress and emotional state.

[0386] The "teaching material generation method" is a function that generates customized teaching materials tailored to each student's level of understanding and emotional state, based on collected data.

[0387] "Learning support methods" refer to the process of presenting generated learning materials to students and supporting their learning through real-time question-and-answer sessions and feedback.

[0388] A "review planning tool" is a function that analyzes students' learning progress and calculates the appropriate timing for review based on the forgetting curve.

[0389] A "review promotion tool" is a system that provides notifications to encourage students to review appropriately based on their review plan.

[0390] "Motivation support methods" refer to the process of generating messages that take students' emotions into consideration and providing support to enhance their motivation to learn.

[0391] The "adjustment mechanism" is a function that uses facial recognition technology to analyze students' emotional states in real time and dynamically adjust the difficulty level of the learning content.

[0392] In the system that realizes this invention, a program is embedded in a consumer robot for home use. The server analyzes students' learning data and emotional information in real time and generates individually customized learning plans. As a means of information gathering, the robot uses its built-in camera and microphone, analyzes facial expressions using the Google Cloud Vision API, and converts speech to text using Google Cloud Speech-to-Text. This allows the robot to understand the emotional state of the students.

[0393] The terminal records the results and response time of the questions answered by the student and sends this data to the server. Based on this data, the server provides appropriate learning materials to the student using a material generation system. The materials are in a multimodal format, including text, still images, and audio. The learning support system enables real-time question answering, allowing students to receive immediate feedback whenever they ask a question.

[0394] Furthermore, by providing emotional support through motivational support mechanisms, students' motivation to learn can be enhanced. When students are tired or stressed, the robot will suggest taking a break in a gentle voice or deliver an encouraging message. For example, if a third-grade elementary school student gets stuck on a difficult math problem, the robot might suggest, "How about taking a short break? Or would you like a hint?"

[0395] An example of a prompt is: "The robot will analyze the child's facial expressions and voice and flexibly adjust the learning content. If the child is having trouble with a difficult problem, please think about what to do." Using this prompt, a generative AI model can suggest an appropriate intervention method.

[0396] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0397] Step 1:

[0398] The device collects student facial expression data captured by its built-in camera and audio data captured by its microphone in real time. This data is sent as input to the Google Cloud Vision API and the Google Cloud Speech-to-Text API. This process converts the image data into facial expression information and the audio data into text information.

[0399] Step 2:

[0400] The server analyzes the facial expression information and audio text information obtained in Step 1 to evaluate the student's emotional state. This analysis uses a generative AI model to predict the student's emotional state using prompt sentences. For example, under the condition "the student is confused," the prediction result is output.

[0401] Step 3:

[0402] Based on the analysis results from Step 2, the server generates learning materials tailored to each student using a material generation method. In this process, a generation AI model is used to create multimodal materials including text, images, and audio with adjusted difficulty levels. The inputs used are the student's emotional assessment data and previous learning history, and the output material content is based on this information.

[0403] Step 4:

[0404] The terminal displays customized learning materials sent from the server to the student. It records the student's response time and results when solving problems and sends this data back to the server. This data is processed by the server as input data necessary for generating the next set of learning materials.

[0405] Step 5:

[0406] The server aggregates answer data and emotional data, and uses a review planning tool to calculate review timing based on the forgetting curve. Based on this calculation result, information is output to suggest the optimal review date.

[0407] Step 6:

[0408] The device provides students with review reminders based on the review information calculated in step 5. Students, as users, can then receive these notifications and perform their reviews at the appropriate time.

[0409] Step 7:

[0410] With the aim of providing emotional support to users, the server generates feedback through motivational support methods and delivers it via the terminal. Based on the data (input) obtained through emotion recognition, it optimizes and outputs emotional support messages, such as "How about taking a short break?"

[0411] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0412] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0413] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0414] [Third Embodiment]

[0415] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0416] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0417] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0418] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0419] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0420] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0421] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0422] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0423] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0424] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0425] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0426] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0427] This invention is a system implemented using a server and terminals to support individualized learning for students. This system consists of information gathering means, material generation means, learning support means, review planning means, review promotion means, and motivation support means. A specific embodiment of this system is described below.

[0428] The server receives data about students' learning activities from their devices. This data includes learning history, correctness of answers, and response time. The server uses this information to analyze students' comprehension and learning trends. Based on the analysis results, the server generates personalized learning materials for each student. These materials include text, images, and audio adjusted to the student's level.

[0429] The terminal displays learning materials provided by the server to the students. As students use the materials and progress through their learning, the terminal provides real-time feedback. For example, if a student enters a question such as, "I don't know how to solve this problem," the terminal, with the assistance of the server, provides an immediate answer.

[0430] The server also continuously monitors learning progress and calculates effective review timings based on Ebbinghaus's forgetting curve. The server notifies the terminal of this review plan, and the terminal prompts students to review at the appropriate time. For example, if the server determines that "it is worth reviewing this unit in one week," it sends a reminder to the student through the terminal.

[0431] Furthermore, the device provides emotional support to students during their studies. This includes displaying messages such as "You did a great job today!" when they finish their studies, thereby boosting their self-esteem and maintaining their motivation to learn.

[0432] Thus, the present invention provides an efficient learning environment tailored to the individual needs of each student, enabling a deep understanding of the learning content and long-term memory retention.

[0433] The following describes the processing flow.

[0434] Step 1:

[0435] The user (student) begins learning using a device. The device records the student's response status in real time, accumulating data such as the correctness of the answer, the time taken to answer, and comments on the problem.

[0436] Step 2:

[0437] Once a certain learning session is complete, the device sends the collected learning data to the server. This data includes the correct answer rate and answer history for each individual question.

[0438] Step 3:

[0439] The server analyzes the received data to determine students' comprehension levels, learning speed, and error patterns. This allows for a detailed understanding of each student's learning progress.

[0440] Step 4:

[0441] Based on the analysis results, the server creates customized learning materials tailored to each student's level. These materials include text, images, and audio, and focus on addressing students' weaknesses.

[0442] Step 5:

[0443] The server sends the generated customized learning materials to the terminals. The terminals display these materials to the students, incorporating interactive elements as needed to provide an environment where students can learn with interest.

[0444] Step 6:

[0445] Users (students) proceed with their learning using the customized learning materials displayed. They can input questions about anything they are unsure of through their device.

[0446] Step 7:

[0447] The device sends a question from the student to the server. The server analyzes the question, generates an appropriate answer, and sends it back to the device. The device then displays this answer to the student.

[0448] Step 8:

[0449] The server continuously monitors students' progress and understanding of the material, and calculates the optimal timing for review based on Ebbinghaus's forgetting curve.

[0450] Step 9:

[0451] The server sends a notification to the device based on the calculated review timing. The device displays a reminder to the student, such as "You need to review a specific unit."

[0452] Step 10:

[0453] The device automatically displays messages such as, "You made great progress today!" to provide emotional support to students during and after learning sessions.

[0454] (Example 1)

[0455] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0456] In modern education, providing learning support tailored to the individual needs of each student is crucial, but in practice, it is difficult to achieve. Existing systems often provide uniform teaching materials to all students, failing to realize effective education based on individual levels of understanding and learning styles. Furthermore, there is insufficient management of review timing and support to sustain students' motivation to learn. In addition, the inability to respond to questions during learning in real time hinders students' understanding.

[0457] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0458] In this invention, the server includes data processing means, content provision means, and information support means. This makes it possible to generate and provide learning materials optimized for each individual student, calculate the timing of review according to the student's learning progress, and encourage review at the appropriate time. Furthermore, by immediately responding to students' questions during learning, it is possible to promote deeper understanding and increase students' motivation to learn.

[0459] "Data processing means" refers to devices or programs for acquiring information about students' learning activities and analyzing that data.

[0460] "Content delivery means" refers to devices or programs that generate and deliver educational materials optimized to the learning needs of individual students.

[0461] "Information support tools" are devices or programs that provide immediate responses to students' questions and doubts during learning, and offer additional information.

[0462] A "schedule management tool" is a device or program used to manage learning progress and calculate the optimal timing for review.

[0463] "Action-promoting measures" refer to devices or programs that send notifications to students encouraging them to review based on calculated review timings.

[0464] "Mental support measures" refer to devices and programs designed to provide students with positive feedback and improve their motivation to learn.

[0465] This invention is a system built by linking a server and terminals to support individualized learning for students. Specific embodiments are shown below.

[0466] The server collects data on students' learning progress and activities. To this end, it receives data from each student's terminal, including their answers, correct / incorrect answers, and the time taken to complete the tasks. The server stores this information in a database and then performs statistical data analysis using the Python pandas library. This allows for the analysis of each student's learning tendencies and level of understanding.

[0467] The server uses a generative AI model to generate learning materials tailored to the student's learning needs. This involves utilizing natural language processing technology and a generative AI model (e.g., GPT-3) to generate text based on prompts. For example, a prompt could be: "Based on this student's recent learning data, please suggest assignments for the next week." The generated materials are provided as documents, images, and audio, containing content appropriate to the student's progress and level.

[0468] The device displays learning materials provided by the server to the students. If a student has a question during learning, the device receives the question and provides an immediate answer by querying the server. Using Dialogflow or similar chatbot technology would be effective for this.

[0469] The server further manages the review schedule based on Ebbinghaus's forgetting curve theory. It uses scikit-learn algorithms to calculate the optimal review timing and create a plan. The terminal then uses this plan to notify students at the appropriate time, sending messages such as, "Please review the math practice problems tomorrow."

[0470] Finally, the device plays a role in providing positive feedback to students at the end of their learning session, thereby boosting their motivation. Feedback such as, "You did very well on today's lesson!" helps to increase students' self-esteem and improve their motivation to learn.

[0471] Through this system, personalized learning support is provided to each student, enabling effective learning.

[0472] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0473] Step 1:

[0474] The server receives student learning information from terminals. Specifically, the input data includes student answers, correct / incorrect results, and completion time. This data is stored in a database and processed into the format necessary for statistical analysis. The stored data serves as foundational information for future analysis and material generation.

[0475] Step 2:

[0476] The server analyzes students' comprehension using stored data. Input includes past answer data and learning history. A dataframe is constructed using the Python pandas library, and average scores and correct answer rates are calculated. The output is a trend analysis report showing each student's strengths and weaknesses. This result helps understand students' learning trends and is used to generate subsequent learning materials.

[0477] Step 3:

[0478] The server uses a generative AI model to create prompts and then generates customized learning materials based on them. The input data includes prompts such as "Please suggest the given math problem," based on a trend analysis report. The generated text, diagrams, and audio files are output and packaged as learning materials displayed to students. These materials are tailored to the individual learning needs of each student.

[0479] Step 4:

[0480] The terminal displays learning materials generated from the server to the students. Students use these materials to progress through their studies. The input consists of the learning material data to be displayed, and if a student asks a question during their study, that question is sent to the server via the terminal. The output consists of replies and additional information received from the server, which are displayed on the student's screen in real time. For example, if a student types "Please explain this problem," the terminal immediately displays the explanation received from the server.

[0481] Step 5:

[0482] The server calculates a review schedule based on Ebbinghaus's forgetting curve. Input includes the student's overall learning progress and past review history, and uses the scikit-learn algorithm to determine the optimal review timing. The output is a schedule specifying the timing and content of reviews, which is generated individually for each student.

[0483] Step 6:

[0484] The device notifies students based on a review schedule sent from the server. The input for the notification includes data on what to review and when, and the device sends reminders to students based on this data. The output is a specific notification displayed on the device, such as "Let's review the math practice problems next Monday."

[0485] Step 7:

[0486] The device provides emotional support when students finish their learning. The input is the student's daily learning performance, and the output displays a motivational message on the screen such as, "You did very well in your learning today!" This helps to sustain students' motivation to learn.

[0487] (Application Example 1)

[0488] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0489] To provide individualized learning support tailored to each student's learning progress and level of understanding, it is necessary to collect diverse information, analyze it in real time, and generate learning materials and provide effective feedback based on that analysis. However, conventional learning support systems have problems such as insufficient individual support and difficulty in optimizing student motivation and review timing. Furthermore, there is a lack of systems for continuously monitoring student learning and maintaining long-term learning effectiveness.

[0490] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0491] In this invention, the server includes data collection means, content generation means, and educational support means. This makes it possible to grasp students' learning progress in real time, provide individually customized learning materials, and notify them of effective review timings and provide emotional support.

[0492] A "data collection method" is a function for collecting students' learning information and recording learning data such as learning progress, answer time, and correct answer rate.

[0493] The "content generation means" is a function for automatically generating individualized learning materials for students based on the results of the analysis of collected data.

[0494] "Educational support tools" refer to functions that provide generated learning materials to students and offer real-time feedback to students' questions during their learning.

[0495] A "review planning tool" is a function that monitors learning progress and calculates the optimal timing for review based on the forgetting curve.

[0496] A "review promotion tool" is a function that provides notifications to students to encourage review based on calculated review timings.

[0497] "Motivation support measures" refer to functions that provide emotional support to students and deliver messages and feedback to enhance their motivation to learn.

[0498] "Communication means" refers to the function that provides the connection for educational equipment to communicate with a server and transmit students' learning activities to the server in real time.

[0499] "Communication control means" refers to a function that manages the communication process for sending learning data from an educational machine to a server and receiving instructions from the server.

[0500] This invention is a system in which a server and a terminal work together to support individualized learning for students. The server collects and analyzes students' learning information and generates individually customized learning materials. This utilizes data collection means and content generation means. Specifically, learning data such as students' response time and correct answer rate are periodically sent to the server. Based on this information, the server grasps the progress of learning in real time and generates multimodal learning materials as needed.

[0501] The terminal displays learning materials provided by the server to the students. Using educational support tools, it provides immediate feedback if students have questions during their studies. In addition, in learning support, it uses a review planning tool to calculate the most effective review timing based on the student's forgetting curve, and sends a notification from the terminal to the student prompting them to review at that time.

[0502] Furthermore, the device is equipped with motivational support features to provide emotional support to students. When a lesson is completed, the device displays a message that affirms the student's daily efforts, maintaining their motivation to learn. For example, if an elementary school student is learning math and gets stuck on a particular problem, the device can immediately answer questions such as, "Can you tell me how to solve this problem?"

[0503] An example of a prompt is a sentence like, "Generate appropriate feedback for students on a specific math problem," and the accuracy of the answer can be improved through a generative AI model.

[0504] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0505] Step 1:

[0506] The server receives student learning information from the terminal. This data includes answer time, accuracy rate, and learning history. The server stores this information in a database for later analysis. This process allows the server to understand each student's current learning status.

[0507] Step 2:

[0508] The server analyzes the collected learning data and uses a generative AI model to evaluate each student's learning tendencies. This analysis determines learning speed and comprehension level, and obtains basic information for generating personalized educational content accordingly. Specific data processing includes evaluating learning speed and extracting patterns from incorrect answers.

[0509] Step 3:

[0510] The server uses content generation tools to create individually optimized learning materials for each student. This includes a process that combines text, images, and audio according to the student's learning level to generate the materials. The generated materials are customized to deepen the student's understanding. The generated materials are then ready for transmission to the terminal.

[0511] Step 4:

[0512] The terminal receives learning materials from the server and presents them to the students. Through its user interface, the terminal enhances learning effectiveness using visual and auditory stimuli. If students have questions, the terminal uses educational support tools to send queries to the server and generate feedback. Because this feedback is provided in real time, it is expected to improve students' learning efficiency.

[0513] Step 5:

[0514] The server uses a review planning system to monitor students' learning progress and calculate review timing based on the forgetting curve. This calculation identifies when it would be most effective for students to review. This plan is then ready to be communicated to students via their devices.

[0515] Step 6:

[0516] The device displays reminders to students when it's time for review. These reminders are designed to encourage repetition of learned material and aid in long-term retention. Furthermore, the device utilizes motivational support features, displaying messages praising students' efforts at the end of their learning sessions to maintain their enthusiasm.

[0517] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0518] This invention provides individualized learning tailored to each student's learning progress and level of understanding, and further combines this with an emotion recognition engine to create a more effective learning environment. Specific embodiments are described below.

[0519] The server receives data from the terminal regarding the student's learning progress. This data includes the student's answers to problems, the time it takes to answer them, and changes in the student's facial expressions and voice obtained from the emotion recognition engine. Based on this data, the server analyzes the student's current level of learning comprehension and emotional state.

[0520] The emotion recognition engine is built into the device and performs facial and voice analysis on students to detect their emotional state in real time. This information is sent to a server and used to understand students' stress levels and analyze their motivation in the current learning content.

[0521] The server considers the information obtained through emotion recognition and customizes the learning materials. For example, if a student is feeling stressed, it adjusts the difficulty level or adds interactive elements to help them relax. It also generates motivational messages based on emotional information and delivers them to students through their devices to improve their motivation to learn.

[0522] The device displays learning materials provided to students and supports real-time interaction. When students input questions or comments during learning, the device transmits them to the server, which immediately generates answers and provides feedback. It also periodically updates sentiment data, which is used to optimize the students' learning experience.

[0523] As users (students) learn using their devices, they become aware that their emotional state is being taken into consideration, leading to a more positive learning experience. For example, when they are tired, the device sends a reminder to pause their studies and take a break.

[0524] Thus, the present invention is a system that provides flexible learning support that takes into account the emotional state of learners, and can improve students' motivation and learning efficiency.

[0525] The following describes the processing flow.

[0526] Step 1:

[0527] The user (student) begins learning using the device. The device collects emotional data in real time by capturing the student's facial expressions with its camera and recording their voice with its microphone.

[0528] Step 2:

[0529] The device inputs the collected video and audio data into an emotion recognition engine to analyze the students' emotional state. The analysis results quantify the students' stress levels, motivation levels, and other factors.

[0530] Step 3:

[0531] The device sends the results of the emotion recognition engine along with the training data to the server. The server comprehensively analyzes the student's learning progress and emotional information.

[0532] Step 4:

[0533] The server customizes learning materials to suit each student, taking their emotional state into consideration. For example, if a student is feeling stressed, it prioritizes generating content that promotes relaxation.

[0534] Step 5:

[0535] The server sends the generated customized learning materials to the terminal. The terminal then presents these materials to the student and continuously adjusts the content according to the student's learning progress.

[0536] Step 6:

[0537] The user (student) continues their learning activities using the learning materials displayed on the device. The device provides an interface that accepts input in response to the student's questions.

[0538] Step 7:

[0539] The terminal sends questions from students to the server. The server analyzes the questions, generates appropriate answers and additional materials, and then sends them back to the terminal.

[0540] Step 8:

[0541] The server uses Ebbinghaus's forgetting curve to calculate the most effective timing for review, based on students' learning history and emotional data.

[0542] Step 9:

[0543] The server sends a notification to the device indicating when it's time to review. The device then displays a review reminder to the student at the appropriate time.

[0544] Step 10:

[0545] The device displays motivational messages based on the student's emotional state during study sessions or when breaks are needed. This helps maintain the student's motivation to learn and reduces mental stress.

[0546] (Example 2)

[0547] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0548] In today's educational environment, the importance of individualized learning support that takes into account each student's level of understanding and emotional state is increasing. However, conventional systems have struggled to appropriately grasp students' emotional states and effectively customize learning materials and provide motivational support accordingly. To address this problem, a new system is needed that can analyze students' emotions in more detail and optimize their learning experience in real time.

[0549] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0550] In this invention, the server includes an information gathering means for collecting and analyzing student learning information, a material generation means for generating individually customized learning materials for each student based on the analysis results, and a motivation enhancement means for generating and providing motivational messages using a generation AI model. This enables flexible and effective learning support and motivation enhancement that is tailored to each student's level of learning comprehension and emotional state.

[0551] "Information gathering means" refers to a device or software used to acquire students' learning information and emotional state, and to transmit that information to a server.

[0552] "Method for generating learning materials" refers to a device or software for creating individually customized learning materials based on students' learning comprehension levels and emotional states.

[0553] A "learning support tool" is a device or software that has the function of responding to students' questions in real time based on the learning materials provided to them.

[0554] A "review planning tool" is a device or software that monitors students' learning progress and calculates and suggests the optimal timing for review.

[0555] A "review promotion tool" is a device or software that notifies students to review based on the timing calculated by the review planning tool.

[0556] An "emotional analysis tool" is a device or software that analyzes students' facial expressions and voices to evaluate their emotional state and allows for adjustments to learning content based on the results.

[0557] A "motivation enhancement tool" is a device or software that uses a generative AI model to create and deliver messages that enhance students' motivation to learn.

[0558] A "generative AI model" is an artificial intelligence computational model used to generate optimal messages and learning materials tailored to the student's situation.

[0559] This invention is a system that provides students with an individualized learning experience by taking into account their learning progress and level of understanding, and by combining this with emotion recognition technology. The system mainly consists of three components: a server, a terminal, and a user.

[0560] The server plays a role in aggregating and analyzing student learning information. It receives student answer results and answer times transmitted from terminals, as well as sentiment data acquired via an emotion recognition engine. Cloud-based computing technology and database management systems are used for analysis. Specifically, AI models are used to evaluate the accuracy and comprehension of answers, and learning content is adjusted based on sentiment data. For example, if a student shows anxiety about a math problem, the learning materials are adjusted to present easier problems to that student.

[0561] The terminal is a device directly operated by students, supporting the display of learning materials and real-time interaction. The terminal has a built-in camera and microphone, which analyze students' facial expressions and voices, sending emotion recognition data to the server. The terminal itself uses a local program and a simple data analysis module for processing. Furthermore, the terminal can instantly display feedback provided by the server, and can include prompts such as, "Please suggest ways to adjust the learning materials when the student is tired."

[0562] Users (students) use their devices to progress through personalized learning materials. As students learn, they receive real-time feedback from their devices, which helps them adjust their learning plan. They can also input questions and requests into their devices to receive quick feedback from the server. This allows users to learn at their own pace and according to their emotions, providing an efficient and comfortable learning environment.

[0563] This system aims to maximize students' personalized learning experiences by enabling instant content adjustments based on detailed data, including emotional states. Utilizing generative AI models, it consistently provides optimal support, thereby improving learning motivation and optimizing learning efficiency.

[0564] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0565] Step 1:

[0566] The device collects user (student) information at the start of learning. Specifically, it analyzes the student's facial expressions and voice through the camera and microphone. This allows for real-time understanding of their emotional state, and also records the time taken to answer questions and the results of those questions during learning. This data is sent to the server. The input is data on the student's behavior during learning, and the output is the data sent to the server.

[0567] Step 2:

[0568] The server receives learning information and emotional state data transmitted from the terminal. Cloud computing technology is used to analyze the received data. An AI model analyzes this data and evaluates learning comprehension and emotional state. The data input is student information from the terminal, and the output is a report of the analyzed learning comprehension and emotional state.

[0569] Step 3:

[0570] The server customizes the learning materials based on the analysis results obtained. Specifically, it uses a generative AI model to generate materials suitable for the learning progress. Here, difficulty levels can be adjusted and interactive elements can be introduced. The input is the analysis results, and the output is the customized learning materials.

[0571] Step 4:

[0572] The server generates motivational messages along with the learning materials. This also utilizes a generative AI model to select appropriate words and construct the messages. The input consists of customized learning materials and emotional information, while the output is the learning materials sent along with the motivational messages.

[0573] Step 5:

[0574] The terminal displays customized learning materials and motivational messages sent from the server to the user (student). As the user progresses through the materials, they can input further questions and comments. Input consists of learning materials from the server and user feedback, while output consists of the display to the user and feedback data sent to the server.

[0575] Step 6:

[0576] The device constantly monitors the user's facial expressions and learning progress, and sends any new data obtained to the server. This allows the server to dynamically adjust the learning progress. The input is new reaction data during learning, and the output is the transmission of updated data to the server.

[0577] (Application Example 2)

[0578] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0579] Traditional learning support systems can provide customized learning materials based on students' learning progress and understanding, but they lack the ability to assess students' emotions and motivation in real time. As a result, learners may experience stress or decreased motivation. To maximize learning efficiency, a system is needed that provides appropriate learning materials and feedback while taking into account the learner's emotional state.

[0580] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0581] In this invention, the server includes information gathering means, material generation means, learning support means, review planning means, review promotion means, motivation support means, and adjustment means that evaluate the student's emotional state using facial recognition technology and dynamically adjust the learning content. This enables comprehensive learning optimization that takes into account not only the student's learning progress but also their emotional state.

[0582] "Information gathering methods" refer to systems that collect answer results, answer times, facial expression data, and audio data in order to understand students' learning progress and emotional state.

[0583] The "teaching material generation method" is a function that generates customized teaching materials tailored to each student's level of understanding and emotional state, based on collected data.

[0584] "Learning support methods" refer to the process of presenting generated learning materials to students and supporting their learning through real-time question-and-answer sessions and feedback.

[0585] A "review planning tool" is a function that analyzes students' learning progress and calculates the appropriate timing for review based on the forgetting curve.

[0586] A "review promotion tool" is a system that provides notifications to encourage students to review appropriately based on their review plan.

[0587] "Motivation support methods" refer to the process of generating messages that take students' emotions into consideration and providing support to enhance their motivation to learn.

[0588] The "adjustment mechanism" is a function that uses facial recognition technology to analyze students' emotional states in real time and dynamically adjust the difficulty level of the learning content.

[0589] In the system that realizes this invention, a program is embedded in a consumer robot for home use. The server analyzes students' learning data and emotional information in real time and generates individually customized learning plans. As a means of information gathering, the robot uses its built-in camera and microphone, analyzes facial expressions using the Google Cloud Vision API, and converts speech to text using Google Cloud Speech-to-Text. This allows the robot to understand the emotional state of the students.

[0590] The terminal records the results and response time of the questions answered by the student and sends this data to the server. Based on this data, the server provides appropriate learning materials to the student using a material generation system. The materials are in a multimodal format, including text, still images, and audio. The learning support system enables real-time question answering, allowing students to receive immediate feedback whenever they ask a question.

[0591] Furthermore, by providing emotional support through motivational support mechanisms, students' motivation to learn can be enhanced. When students are tired or stressed, the robot will suggest taking a break in a gentle voice or deliver an encouraging message. For example, if a third-grade elementary school student gets stuck on a difficult math problem, the robot might suggest, "How about taking a short break? Or would you like a hint?"

[0592] An example of a prompt is: "The robot will analyze the child's facial expressions and voice and flexibly adjust the learning content. If the child is having trouble with a difficult problem, please think about what to do." Using this prompt, a generative AI model can suggest an appropriate intervention method.

[0593] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0594] Step 1:

[0595] The device collects student facial expression data captured by its built-in camera and audio data captured by its microphone in real time. This data is sent as input to the Google Cloud Vision API and the Google Cloud Speech-to-Text API. This process converts the image data into facial expression information and the audio data into text information.

[0596] Step 2:

[0597] The server analyzes the facial expression information and audio text information obtained in Step 1 to evaluate the student's emotional state. This analysis uses a generative AI model to predict the student's emotional state using prompt sentences. For example, under the condition "the student is confused," the prediction result is output.

[0598] Step 3:

[0599] Based on the analysis results from Step 2, the server generates learning materials tailored to each student using a material generation method. In this process, a generation AI model is used to create multimodal materials including text, images, and audio with adjusted difficulty levels. The inputs used are the student's emotional assessment data and previous learning history, and the output material content is based on this information.

[0600] Step 4:

[0601] The terminal displays customized learning materials sent from the server to the student. It records the student's response time and results when solving problems and sends this data back to the server. This data is processed by the server as input data necessary for generating the next set of learning materials.

[0602] Step 5:

[0603] The server aggregates answer data and emotional data, and uses a review planning tool to calculate review timing based on the forgetting curve. Based on this calculation result, information is output to suggest the optimal review date.

[0604] Step 6:

[0605] The device provides students with review reminders based on the review information calculated in step 5. Students, as users, can then receive these notifications and perform their reviews at the appropriate time.

[0606] Step 7:

[0607] With the aim of providing emotional support to users, the server generates feedback through motivational support methods and delivers it via the terminal. Based on the data (input) obtained through emotion recognition, it optimizes and outputs emotional support messages, such as "How about taking a short break?"

[0608] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0609] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0610] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0611] [Fourth Embodiment]

[0612] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0613] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0614] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0615] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0616] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0617] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0618] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0619] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0620] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0621] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0622] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0623] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0624] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0625] This invention is a system implemented using a server and terminals to support individualized learning for students. This system consists of information gathering means, material generation means, learning support means, review planning means, review promotion means, and motivation support means. A specific embodiment of this system is described below.

[0626] The server receives data about students' learning activities from their devices. This data includes learning history, correctness of answers, and response time. The server uses this information to analyze students' comprehension and learning trends. Based on the analysis results, the server generates personalized learning materials for each student. These materials include text, images, and audio adjusted to the student's level.

[0627] The terminal displays learning materials provided by the server to the students. As students use the materials and progress through their learning, the terminal provides real-time feedback. For example, if a student enters a question such as, "I don't know how to solve this problem," the terminal, with the assistance of the server, provides an immediate answer.

[0628] The server also continuously monitors learning progress and calculates effective review timings based on Ebbinghaus's forgetting curve. The server notifies the terminal of this review plan, and the terminal prompts students to review at the appropriate time. For example, if the server determines that "it is worth reviewing this unit in one week," it sends a reminder to the student through the terminal.

[0629] Furthermore, the device provides emotional support to students during their studies. This includes displaying messages such as "You did a great job today!" when they finish their studies, thereby boosting their self-esteem and maintaining their motivation to learn.

[0630] Thus, the present invention provides an efficient learning environment tailored to the individual needs of each student, enabling a deep understanding of the learning content and long-term memory retention.

[0631] The following describes the processing flow.

[0632] Step 1:

[0633] The user (student) begins learning using a device. The device records the student's response status in real time, accumulating data such as the correctness of the answer, the time taken to answer, and comments on the problem.

[0634] Step 2:

[0635] Once a certain learning session is complete, the device sends the collected learning data to the server. This data includes the correct answer rate and answer history for each individual question.

[0636] Step 3:

[0637] The server analyzes the received data to determine students' comprehension levels, learning speed, and error patterns. This allows for a detailed understanding of each student's learning progress.

[0638] Step 4:

[0639] Based on the analysis results, the server creates customized learning materials tailored to each student's level. These materials include text, images, and audio, and focus on addressing students' weaknesses.

[0640] Step 5:

[0641] The server sends the generated customized learning materials to the terminals. The terminals display these materials to the students, incorporating interactive elements as needed to provide an environment where students can learn with interest.

[0642] Step 6:

[0643] Users (students) proceed with their learning using the customized learning materials displayed. They can input questions about anything they are unsure of through their device.

[0644] Step 7:

[0645] The device sends a question from the student to the server. The server analyzes the question, generates an appropriate answer, and sends it back to the device. The device then displays this answer to the student.

[0646] Step 8:

[0647] The server continuously monitors students' progress and understanding of the material, and calculates the optimal timing for review based on Ebbinghaus's forgetting curve.

[0648] Step 9:

[0649] The server sends a notification to the device based on the calculated review timing. The device displays a reminder to the student, such as "You need to review a specific unit."

[0650] Step 10:

[0651] The device automatically displays messages such as, "You made great progress today!" to provide emotional support to students during and after learning sessions.

[0652] (Example 1)

[0653] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0654] In modern education, providing learning support tailored to the individual needs of each student is crucial, but in practice, it is difficult to achieve. Existing systems often provide uniform teaching materials to all students, failing to realize effective education based on individual levels of understanding and learning styles. Furthermore, there is insufficient management of review timing and support to sustain students' motivation to learn. In addition, the inability to respond to questions during learning in real time hinders students' understanding.

[0655] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0656] In this invention, the server includes data processing means, content provision means, and information support means. This makes it possible to generate and provide learning materials optimized for each individual student, calculate the timing of review according to the student's learning progress, and encourage review at the appropriate time. Furthermore, by immediately responding to students' questions during learning, it is possible to promote deeper understanding and increase students' motivation to learn.

[0657] "Data processing means" refers to devices or programs for acquiring information about students' learning activities and analyzing that data.

[0658] "Content delivery means" refers to devices or programs that generate and deliver educational materials optimized to the learning needs of individual students.

[0659] "Information support tools" are devices or programs that provide immediate responses to students' questions and doubts during learning, and offer additional information.

[0660] A "schedule management tool" is a device or program used to manage learning progress and calculate the optimal timing for review.

[0661] "Action-promoting measures" refer to devices or programs that send notifications to students encouraging them to review based on calculated review timings.

[0662] "Mental support measures" refer to devices and programs designed to provide students with positive feedback and improve their motivation to learn.

[0663] This invention is a system built by linking a server and terminals to support individualized learning for students. Specific embodiments are shown below.

[0664] The server collects data on students' learning progress and activities. To this end, it receives data from each student's terminal, including their answers, correct / incorrect answers, and the time taken to complete the tasks. The server stores this information in a database and then performs statistical data analysis using the Python pandas library. This allows for the analysis of each student's learning tendencies and level of understanding.

[0665] The server uses a generative AI model to generate learning materials tailored to the student's learning needs. This involves utilizing natural language processing technology and a generative AI model (e.g., GPT-3) to generate text based on prompts. For example, a prompt could be: "Based on this student's recent learning data, please suggest assignments for the next week." The generated materials are provided as documents, images, and audio, containing content appropriate to the student's progress and level.

[0666] The device displays learning materials provided by the server to the students. If a student has a question during learning, the device receives the question and provides an immediate answer by querying the server. Using Dialogflow or similar chatbot technology would be effective for this.

[0667] The server further manages the review schedule based on Ebbinghaus's forgetting curve theory. It uses scikit-learn algorithms to calculate the optimal review timing and create a plan. The terminal then uses this plan to notify students at the appropriate time, sending messages such as, "Please review the math practice problems tomorrow."

[0668] Finally, the device plays a role in providing positive feedback to students at the end of their learning session, thereby boosting their motivation. Feedback such as, "You did very well on today's lesson!" helps to increase students' self-esteem and improve their motivation to learn.

[0669] Through this system, personalized learning support is provided to each student, enabling effective learning.

[0670] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0671] Step 1:

[0672] The server receives student learning information from terminals. Specifically, the input data includes student answers, correct / incorrect results, and completion time. This data is stored in a database and processed into the format necessary for statistical analysis. The stored data serves as foundational information for future analysis and material generation.

[0673] Step 2:

[0674] The server analyzes students' comprehension using stored data. Input includes past answer data and learning history. A dataframe is constructed using the Python pandas library, and average scores and correct answer rates are calculated. The output is a trend analysis report showing each student's strengths and weaknesses. This result helps understand students' learning trends and is used to generate subsequent learning materials.

[0675] Step 3:

[0676] The server uses a generative AI model to create prompts and then generates customized learning materials based on them. The input data includes prompts such as "Please suggest the given math problem," based on a trend analysis report. The generated text, diagrams, and audio files are output and packaged as learning materials displayed to students. These materials are tailored to the individual learning needs of each student.

[0677] Step 4:

[0678] The terminal displays learning materials generated from the server to the students. Students use these materials to progress through their studies. The input consists of the learning material data to be displayed, and if a student asks a question during their study, that question is sent to the server via the terminal. The output consists of replies and additional information received from the server, which are displayed on the student's screen in real time. For example, if a student types "Please explain this problem," the terminal immediately displays the explanation received from the server.

[0679] Step 5:

[0680] The server calculates a review schedule based on Ebbinghaus's forgetting curve. Input includes the student's overall learning progress and past review history, and uses the scikit-learn algorithm to determine the optimal review timing. The output is a schedule specifying the timing and content of reviews, which is generated individually for each student.

[0681] Step 6:

[0682] The device notifies students based on a review schedule sent from the server. The input for the notification includes data on what to review and when, and the device sends reminders to students based on this data. The output is a specific notification displayed on the device, such as "Let's review the math practice problems next Monday."

[0683] Step 7:

[0684] The device provides emotional support when students finish their learning. The input is the student's daily learning performance, and the output displays a motivational message on the screen such as, "You did very well in your learning today!" This helps to sustain students' motivation to learn.

[0685] (Application Example 1)

[0686] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0687] To provide individualized learning support tailored to each student's learning progress and level of understanding, it is necessary to collect diverse information, analyze it in real time, and generate learning materials and provide effective feedback based on that analysis. However, conventional learning support systems have problems such as insufficient individual support and difficulty in optimizing student motivation and review timing. Furthermore, there is a lack of systems for continuously monitoring student learning and maintaining long-term learning effectiveness.

[0688] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0689] In this invention, the server includes data collection means, content generation means, and educational support means. This makes it possible to grasp students' learning progress in real time, provide individually customized learning materials, and notify them of effective review timings and provide emotional support.

[0690] A "data collection method" is a function for collecting students' learning information and recording learning data such as learning progress, answer time, and correct answer rate.

[0691] The "content generation means" is a function for automatically generating individualized learning materials for students based on the results of the analysis of collected data.

[0692] "Educational support tools" refer to functions that provide generated learning materials to students and offer real-time feedback to students' questions during their learning.

[0693] A "review planning tool" is a function that monitors learning progress and calculates the optimal timing for review based on the forgetting curve.

[0694] A "review promotion tool" is a function that provides notifications to students to encourage review based on calculated review timings.

[0695] "Motivation support measures" refer to functions that provide emotional support to students and deliver messages and feedback to enhance their motivation to learn.

[0696] "Communication means" refers to the function that provides the connection for educational equipment to communicate with a server and transmit students' learning activities to the server in real time.

[0697] "Communication control means" refers to a function that manages the communication process for sending learning data from an educational machine to a server and receiving instructions from the server.

[0698] This invention is a system in which a server and a terminal work together to support individualized learning for students. The server collects and analyzes students' learning information and generates individually customized learning materials. This utilizes data collection means and content generation means. Specifically, learning data such as students' response time and correct answer rate are periodically sent to the server. Based on this information, the server grasps the progress of learning in real time and generates multimodal learning materials as needed.

[0699] The terminal displays learning materials provided by the server to the students. Using educational support tools, it provides immediate feedback if students have questions during their studies. In addition, in learning support, it uses a review planning tool to calculate the most effective review timing based on the student's forgetting curve, and sends a notification from the terminal to the student prompting them to review at that time.

[0700] Furthermore, the device is equipped with motivational support features to provide emotional support to students. When a lesson is completed, the device displays a message that affirms the student's daily efforts, maintaining their motivation to learn. For example, if an elementary school student is learning math and gets stuck on a particular problem, the device can immediately answer questions such as, "Can you tell me how to solve this problem?"

[0701] An example of a prompt is a sentence like, "Generate appropriate feedback for students on a specific math problem," and the accuracy of the answer can be improved through a generative AI model.

[0702] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0703] Step 1:

[0704] The server receives student learning information from the terminal. This data includes answer time, accuracy rate, and learning history. The server stores this information in a database for later analysis. This process allows the server to understand each student's current learning status.

[0705] Step 2:

[0706] The server analyzes the collected learning data and uses a generative AI model to evaluate each student's learning tendencies. This analysis determines learning speed and comprehension level, and obtains basic information for generating personalized educational content accordingly. Specific data processing includes evaluating learning speed and extracting patterns from incorrect answers.

[0707] Step 3:

[0708] The server uses content generation tools to create individually optimized learning materials for each student. This includes a process that combines text, images, and audio according to the student's learning level to generate the materials. The generated materials are customized to deepen the student's understanding. The generated materials are then ready for transmission to the terminal.

[0709] Step 4:

[0710] The terminal receives learning materials from the server and presents them to the students. Through its user interface, the terminal enhances learning effectiveness using visual and auditory stimuli. If students have questions, the terminal uses educational support tools to send queries to the server and generate feedback. Because this feedback is provided in real time, it is expected to improve students' learning efficiency.

[0711] Step 5:

[0712] The server uses a review planning system to monitor students' learning progress and calculate review timing based on the forgetting curve. This calculation identifies when it would be most effective for students to review. This plan is then ready to be communicated to students via their devices.

[0713] Step 6:

[0714] The device displays reminders to students when it's time for review. These reminders are designed to encourage repetition of learned material and aid in long-term retention. Furthermore, the device utilizes motivational support features, displaying messages praising students' efforts at the end of their learning sessions to maintain their enthusiasm.

[0715] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0716] This invention provides individualized learning tailored to each student's learning progress and level of understanding, and further combines this with an emotion recognition engine to create a more effective learning environment. Specific embodiments are described below.

[0717] The server receives data from the terminal regarding the student's learning progress. This data includes the student's answers to problems, the time it takes to answer them, and changes in the student's facial expressions and voice obtained from the emotion recognition engine. Based on this data, the server analyzes the student's current level of learning comprehension and emotional state.

[0718] The emotion recognition engine is built into the device and performs facial and voice analysis on students to detect their emotional state in real time. This information is sent to a server and used to understand students' stress levels and analyze their motivation in the current learning content.

[0719] The server considers the information obtained through emotion recognition and customizes the learning materials. For example, if a student is feeling stressed, it adjusts the difficulty level or adds interactive elements to help them relax. It also generates motivational messages based on emotional information and delivers them to students through their devices to improve their motivation to learn.

[0720] The device displays learning materials provided to students and supports real-time interaction. When students input questions or comments during learning, the device transmits them to the server, which immediately generates answers and provides feedback. It also periodically updates sentiment data, which is used to optimize the students' learning experience.

[0721] As users (students) learn using their devices, they become aware that their emotional state is being taken into consideration, leading to a more positive learning experience. For example, when they are tired, the device sends a reminder to pause their studies and take a break.

[0722] Thus, the present invention is a system that provides flexible learning support that takes into account the emotional state of learners, and can improve students' motivation and learning efficiency.

[0723] The following describes the processing flow.

[0724] Step 1:

[0725] The user (student) begins learning using the device. The device collects emotional data in real time by capturing the student's facial expressions with its camera and recording their voice with its microphone.

[0726] Step 2:

[0727] The device inputs the collected video and audio data into an emotion recognition engine to analyze the students' emotional state. The analysis results quantify the students' stress levels, motivation levels, and other factors.

[0728] Step 3:

[0729] The device sends the results of the emotion recognition engine along with the training data to the server. The server comprehensively analyzes the student's learning progress and emotional information.

[0730] Step 4:

[0731] The server customizes learning materials to suit each student, taking their emotional state into consideration. For example, if a student is feeling stressed, it prioritizes generating content that promotes relaxation.

[0732] Step 5:

[0733] The server sends the generated customized learning materials to the terminal. The terminal then presents these materials to the student and continuously adjusts the content according to the student's learning progress.

[0734] Step 6:

[0735] The user (student) continues their learning activities using the learning materials displayed on the device. The device provides an interface that accepts input in response to the student's questions.

[0736] Step 7:

[0737] The terminal sends questions from students to the server. The server analyzes the questions, generates appropriate answers and additional materials, and then sends them back to the terminal.

[0738] Step 8:

[0739] The server uses Ebbinghaus's forgetting curve to calculate the most effective timing for review, based on students' learning history and emotional data.

[0740] Step 9:

[0741] The server sends a notification to the device indicating when it's time to review. The device then displays a review reminder to the student at the appropriate time.

[0742] Step 10:

[0743] The device displays motivational messages based on the student's emotional state during study sessions or when breaks are needed. This helps maintain the student's motivation to learn and reduces mental stress.

[0744] (Example 2)

[0745] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0746] In today's educational environment, the importance of individualized learning support that takes into account each student's level of understanding and emotional state is increasing. However, conventional systems have struggled to appropriately grasp students' emotional states and effectively customize learning materials and provide motivational support accordingly. To address this problem, a new system is needed that can analyze students' emotions in more detail and optimize their learning experience in real time.

[0747] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0748] In this invention, the server includes an information gathering means for collecting and analyzing student learning information, a material generation means for generating individually customized learning materials for each student based on the analysis results, and a motivation enhancement means for generating and providing motivational messages using a generation AI model. This enables flexible and effective learning support and motivation enhancement that is tailored to each student's level of learning comprehension and emotional state.

[0749] "Information gathering means" refers to a device or software used to acquire students' learning information and emotional state, and to transmit that information to a server.

[0750] "Method for generating learning materials" refers to a device or software for creating individually customized learning materials based on students' learning comprehension levels and emotional states.

[0751] A "learning support tool" is a device or software that has the function of responding to students' questions in real time based on the learning materials provided to them.

[0752] A "review planning tool" is a device or software that monitors students' learning progress and calculates and suggests the optimal timing for review.

[0753] A "review promotion tool" is a device or software that notifies students to review based on the timing calculated by the review planning tool.

[0754] An "emotional analysis tool" is a device or software that analyzes students' facial expressions and voices to evaluate their emotional state and allows for adjustments to learning content based on the results.

[0755] A "motivation enhancement tool" is a device or software that uses a generative AI model to create and deliver messages that enhance students' motivation to learn.

[0756] A "generative AI model" is an artificial intelligence computational model used to generate optimal messages and learning materials tailored to the student's situation.

[0757] This invention is a system that provides students with an individualized learning experience by taking into account their learning progress and level of understanding, and by combining this with emotion recognition technology. The system mainly consists of three components: a server, a terminal, and a user.

[0758] The server plays a role in aggregating and analyzing student learning information. It receives student answer results and answer times transmitted from terminals, as well as sentiment data acquired via an emotion recognition engine. Cloud-based computing technology and database management systems are used for analysis. Specifically, AI models are used to evaluate the accuracy and comprehension of answers, and learning content is adjusted based on sentiment data. For example, if a student shows anxiety about a math problem, the learning materials are adjusted to present easier problems to that student.

[0759] The terminal is a device directly operated by students, supporting the display of learning materials and real-time interaction. The terminal has a built-in camera and microphone, which analyze students' facial expressions and voices, sending emotion recognition data to the server. The terminal itself uses a local program and a simple data analysis module for processing. Furthermore, the terminal can instantly display feedback provided by the server, and can include prompts such as, "Please suggest ways to adjust the learning materials when the student is tired."

[0760] Users (students) use their devices to progress through personalized learning materials. As students learn, they receive real-time feedback from their devices, which helps them adjust their learning plan. They can also input questions and requests into their devices to receive quick feedback from the server. This allows users to learn at their own pace and according to their emotions, providing an efficient and comfortable learning environment.

[0761] This system aims to maximize students' personalized learning experiences by enabling instant content adjustments based on detailed data, including emotional states. Utilizing generative AI models, it consistently provides optimal support, thereby improving learning motivation and optimizing learning efficiency.

[0762] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0763] Step 1:

[0764] The device collects user (student) information at the start of learning. Specifically, it analyzes the student's facial expressions and voice through the camera and microphone. This allows for real-time understanding of their emotional state, and also records the time taken to answer questions and the results of those questions during learning. This data is sent to the server. The input is data on the student's behavior during learning, and the output is the data sent to the server.

[0765] Step 2:

[0766] The server receives learning information and emotional state data transmitted from the terminal. Cloud computing technology is used to analyze the received data. An AI model analyzes this data and evaluates learning comprehension and emotional state. The data input is student information from the terminal, and the output is a report of the analyzed learning comprehension and emotional state.

[0767] Step 3:

[0768] The server customizes the learning materials based on the analysis results obtained. Specifically, it uses a generative AI model to generate materials suitable for the learning progress. Here, difficulty levels can be adjusted and interactive elements can be introduced. The input is the analysis results, and the output is the customized learning materials.

[0769] Step 4:

[0770] The server generates motivational messages along with the learning materials. This also utilizes a generative AI model to select appropriate words and construct the messages. The input consists of customized learning materials and emotional information, while the output is the learning materials sent along with the motivational messages.

[0771] Step 5:

[0772] The terminal displays customized learning materials and motivational messages sent from the server to the user (student). As the user progresses through the materials, they can input further questions and comments. Input consists of learning materials from the server and user feedback, while output consists of the display to the user and feedback data sent to the server.

[0773] Step 6:

[0774] The device constantly monitors the user's facial expressions and learning progress, and sends any new data obtained to the server. This allows the server to dynamically adjust the learning progress. The input is new reaction data during learning, and the output is the transmission of updated data to the server.

[0775] (Application Example 2)

[0776] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0777] Traditional learning support systems can provide customized learning materials based on students' learning progress and understanding, but they lack the ability to assess students' emotions and motivation in real time. As a result, learners may experience stress or decreased motivation. To maximize learning efficiency, a system is needed that provides appropriate learning materials and feedback while taking into account the learner's emotional state.

[0778] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0779] In this invention, the server includes information gathering means, material generation means, learning support means, review planning means, review promotion means, motivation support means, and adjustment means that evaluate the student's emotional state using facial recognition technology and dynamically adjust the learning content. This enables comprehensive learning optimization that takes into account not only the student's learning progress but also their emotional state.

[0780] "Information gathering methods" refer to systems that collect answer results, answer times, facial expression data, and audio data in order to understand students' learning progress and emotional state.

[0781] The "teaching material generation method" is a function that generates customized teaching materials tailored to each student's level of understanding and emotional state, based on collected data.

[0782] "Learning support methods" refer to the process of presenting generated learning materials to students and supporting their learning through real-time question-and-answer sessions and feedback.

[0783] A "review planning tool" is a function that analyzes students' learning progress and calculates the appropriate timing for review based on the forgetting curve.

[0784] A "review promotion tool" is a system that provides notifications to encourage students to review appropriately based on their review plan.

[0785] "Motivation support methods" refer to the process of generating messages that take students' emotions into consideration and providing support to enhance their motivation to learn.

[0786] The "adjustment mechanism" is a function that uses facial recognition technology to analyze students' emotional states in real time and dynamically adjust the difficulty level of the learning content.

[0787] In the system that realizes this invention, a program is embedded in a consumer robot for home use. The server analyzes students' learning data and emotional information in real time and generates individually customized learning plans. As a means of information gathering, the robot uses its built-in camera and microphone, analyzes facial expressions using the Google Cloud Vision API, and converts speech to text using Google Cloud Speech-to-Text. This allows the robot to understand the emotional state of the students.

[0788] The terminal records the results and response time of the questions answered by the student and sends this data to the server. Based on this data, the server provides appropriate learning materials to the student using a material generation system. The materials are in a multimodal format, including text, still images, and audio. The learning support system enables real-time question answering, allowing students to receive immediate feedback whenever they ask a question.

[0789] Furthermore, by providing emotional support through motivational support mechanisms, students' motivation to learn can be enhanced. When students are tired or stressed, the robot will suggest taking a break in a gentle voice or deliver an encouraging message. For example, if a third-grade elementary school student gets stuck on a difficult math problem, the robot might suggest, "How about taking a short break? Or would you like a hint?"

[0790] An example of a prompt is: "The robot will analyze the child's facial expressions and voice and flexibly adjust the learning content. If the child is having trouble with a difficult problem, please think about what to do." Using this prompt, a generative AI model can suggest an appropriate intervention method.

[0791] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0792] Step 1:

[0793] The device collects student facial expression data captured by its built-in camera and audio data captured by its microphone in real time. This data is sent as input to the Google Cloud Vision API and the Google Cloud Speech-to-Text API. This process converts the image data into facial expression information and the audio data into text information.

[0794] Step 2:

[0795] The server analyzes the facial expression information and audio text information obtained in Step 1 to evaluate the student's emotional state. This analysis uses a generative AI model to predict the student's emotional state using prompt sentences. For example, under the condition "the student is confused," the prediction result is output.

[0796] Step 3:

[0797] Based on the analysis results from Step 2, the server generates learning materials tailored to each student using a material generation method. In this process, a generation AI model is used to create multimodal materials including text, images, and audio with adjusted difficulty levels. The inputs used are the student's emotional assessment data and previous learning history, and the output material content is based on this information.

[0798] Step 4:

[0799] The terminal displays customized learning materials sent from the server to the student. It records the student's response time and results when solving problems, and sends this data back to the server. This data is processed by the server as input data necessary for generating the next set of learning materials.

[0800] Step 5:

[0801] The server aggregates answer data and emotional data, and uses a review planning tool to calculate review timing based on the forgetting curve. Based on this calculation result, information is output to suggest the optimal review date.

[0802] Step 6:

[0803] The device provides students with review reminders based on the review information calculated in step 5. Students, as users, can then receive these notifications and review at the appropriate time.

[0804] Step 7:

[0805] With the aim of providing emotional support to users, the server generates feedback through motivational support methods and delivers it via the terminal. Based on the data (input) obtained through emotion recognition, it optimizes and outputs emotional support messages, such as "How about taking a short break?"

[0806] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0807] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0808] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0809] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0810] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0811] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0812] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0813] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0814] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0815] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0816] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0817] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0818] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0819] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0820] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0821] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0822] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0823] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0824] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0825] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0826] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0827] The following is further disclosed regarding the embodiments described above.

[0828] (Claim 1)

[0829] Information gathering means for collecting and analyzing student learning information,

[0830] A means for generating teaching materials that are customized for each student based on the analysis results,

[0831] A learning support system that provides the generated teaching materials to students and provides real-time feedback to students' questions,

[0832] A review planning method that monitors students' learning progress and calculates the timing of review based on the forgetting curve,

[0833] A means for promoting review that provides a notification to encourage students to review based on the aforementioned review plan,

[0834] To provide emotional support to students and to increase their motivation to learn,

[0835] A system that includes this.

[0836] (Claim 2)

[0837] The system according to claim 1, wherein the information gathering means periodically records learning data including students' response times and correct answer rates.

[0838] (Claim 3)

[0839] The system according to claim 1, wherein the material generation means creates multimodal material including text, images, and audio that are appropriate to the developmental stage of the students.

[0840] "Example 1"

[0841] (Claim 1)

[0842] A data processing means for acquiring and analyzing information related to student education,

[0843] A content provision means for generating individually optimized teaching materials for each student based on the aforementioned analysis results,

[0844] An information support means that displays the generated teaching materials on a communication terminal and immediately responds to questions from students during their studies,

[0845] A scheduling management system that monitors students' learning progress and calculates review timing based on memory retention theory,

[0846] A means of prompting students to review based on the aforementioned review schedule,

[0847] Mental support measures that provide positive feedback to students and improve their motivation to learn,

[0848] A system that includes this.

[0849] (Claim 2)

[0850] The system according to claim 1, wherein the data processing means periodically manages educational data, including students' response times and correct answer rates.

[0851] (Claim 3)

[0852] The system according to claim 1, wherein the content provision means creates diverse media teaching materials, including documents, visual information, and audio, that are appropriate to the age and abilities of the students.

[0853] "Application Example 1"

[0854] (Claim 1)

[0855] A data collection method for collecting and analyzing student learning information,

[0856] Content generation means for generating customized teaching materials for individual students based on the analysis results,

[0857] An educational support system that provides the generated teaching materials to students and provides real-time feedback to students' questions,

[0858] A review planning method that monitors students' learning progress and calculates the timing of review based on the forgetting curve,

[0859] A means for promoting review that provides a notification to encourage students to review based on the aforementioned review plan,

[0860] To provide emotional support to students and to increase their motivation to learn,

[0861] A communication method that allows educational machines to communicate with a server to manage students' learning progress,

[0862] The aforementioned educational machine includes a communication control means that transmits the student's learning status to a server in real time,

[0863] A system that includes this.

[0864] (Claim 2)

[0865] The system according to claim 1, wherein the information gathering means includes a control means for periodically recording learning data, including students' response times and correct answer rates.

[0866] (Claim 3)

[0867] The system according to claim 1, wherein the material generation means generates multimodal material including text, images, and audio according to the developmental stage of the students.

[0868] "Example 2 of combining an emotion engine"

[0869] (Claim 1)

[0870] Information gathering means for collecting and analyzing student learning information,

[0871] A means for generating teaching materials that are customized for each student based on the analysis results,

[0872] A learning support system that provides the generated teaching materials to students and provides real-time feedback to students' questions,

[0873] A review planning method that monitors students' learning progress and calculates the timing of review based on the forgetting curve,

[0874] A means for promoting review that provides a notification to encourage students to review based on the aforementioned review plan,

[0875] A means of analyzing students' emotional states and adjusting learning content accordingly,

[0876] A motivation enhancement method that utilizes a generative AI model to generate and provide motivational messages,

[0877] A system that includes this.

[0878] (Claim 2)

[0879] The system according to claim 1, wherein the information gathering means periodically records learning data including the student's response time, correct answer rate, and emotional state.

[0880] (Claim 3)

[0881] The system according to claim 1, wherein the material generation means creates multimodal material including text, images, and audio that corresponds to the developmental stage and emotional state of the students.

[0882] "Application example 2 when combining with an emotional engine"

[0883] (Claim 1)

[0884] Information gathering means for collecting and analyzing student learning information,

[0885] A means for generating teaching materials that are customized for each student based on the analysis results,

[0886] A learning support system that provides the generated teaching materials to students and provides real-time feedback to students' questions,

[0887] A review planning method that monitors students' learning progress and calculates the timing of review based on the forgetting curve,

[0888] A means for promoting review that provides a notification to encourage students to review based on the aforementioned review plan,

[0889] To provide emotional support to students and to increase their motivation to learn,

[0890] An adjustment mechanism that uses facial recognition technology to evaluate students' emotional states and dynamically adjusts learning content,

[0891] A system that includes this.

[0892] (Claim 2)

[0893] The system according to claim 1, wherein the information gathering means includes means for periodically recording learning data including students' response times and correct answer rates, and means for inferring emotions in real time from facial expressions and voice.

[0894] (Claim 3)

[0895] The system according to claim 1, wherein the material generation means has means for creating multimodal material including text, still images and audio according to the developmental stage of students, and for appropriately adjusting the difficulty level based on the emotional response of individual learners. [Explanation of symbols]

[0896] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. Information gathering means for collecting and analyzing student learning information, A means for generating teaching materials that are customized for each student based on the analysis results, A learning support system that provides the generated teaching materials to students and provides real-time feedback to students' questions, A review planning method that monitors students' learning progress and calculates the timing of review based on the forgetting curve, A means for promoting review that provides a notification to encourage students to review based on the aforementioned review plan, To provide emotional support to students and to increase their motivation to learn, A system that includes this.

2. The system according to claim 1, wherein the information gathering means periodically records learning data including students' response times and correct answer rates.

3. The system according to claim 1, wherein the material generation means creates multimodal material including text, images, and audio that are appropriate to the developmental stage of the students.

Images