system

Abstract

We provide the system. [Solution] A means of collecting and integrating learning data from multiple educational institutions, A means for analyzing the aforementioned learning data to identify the learning style and weaknesses of individual learners, A means for generating an appropriate learning plan based on the identified learning style and weaknesses, A means for presenting the aforementioned learning plan through a user interface, A means for collecting learner responses and progress data in real time and adjusting the learning plan, A means of notifying educators and guardians of the progress of learners, A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a character of the chatbot, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance as a response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In the modern educational environment, learners can access various learning opportunities provided by different educational institutions such as schools, cram schools, and distance education. However, it is not possible to integrally manage this information and provide appropriate learning support according to the learning styles and progress of each learner. As a result, problems such as a lack of learning support optimized for individual learners, difficulty in grasping progress in real time, and a decrease in the efficiency of follow-up by educators and guardians have occurred. In addition, the burden on teachers is also large. There is a need to solve these problems and provide more effective and individualized learning support.

Means for Solving the Problems

[0005] This invention provides a system for collecting and integrating learning data from multiple educational institutions. Within this system, it analyzes learning data to identify individual learners' learning styles and weaknesses, and generates appropriate learning plans. The generated learning plans are presented through a user interface, and learners' answers and progress data can be collected in real time, allowing for continuous plan adjustments. Furthermore, it provides a means to notify educators and parents of learners' progress and implement effective follow-up. This enables the individual optimization of the learning environment and supports efficient learning tailored to each learner.

[0006] "Learning data" refers to information related to educational activities, such as learners' learning history, test results, and attendance records.

[0007] "Educational institutions" refer to organizations that provide education to learners, such as schools, cram schools, and distance learning services.

[0008] "Learning style" refers to the characteristics and tendencies of the methods by which learners acquire knowledge most effectively as they progress through their studies.

[0009] A "weakness" refers to a specific area or topic that learners find relatively more difficult to understand or master compared to other areas.

[0010] A "learning plan" refers to a set of guidelines designed for a specific learner, including what to learn next, recommended learning methods, and learning materials.

[0011] "User interface" refers to the collective term for the screens, designs, and navigation that learners, educators, and parents use to interact with a system.

[0012] "In real time" refers to the process of information being processed almost simultaneously with events in the real world, resulting in immediate data updates and reflections.

[0013] An "educator" is someone who is responsible for teaching knowledge to learners and supporting their learning.

[0014] "Guardian" refers to a person who is responsible for the education and welfare of a learner, generally a parent or their representative. [Brief explanation of the drawing]

[0015] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11] This is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] This is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13]It is a sequence diagram showing the processing flow of the data processing system in Embodiment 2 when the emotion engine is combined. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when the emotion engine is combined.

Mode for Carrying Out the Invention

[0016] Hereinafter, an example of an embodiment of the system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0017] First, the terms used in the following description will be explained.

[0018] In the following embodiments, the numbered processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0019] In the following embodiments, the numbered RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0020] In the following embodiments, the numbered storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, etc.

[0021] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0022] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0023] [First Embodiment]

[0024] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0025] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0026] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0027] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0028] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0029] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0030] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0031] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0032] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0033] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0034] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0035] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0036] The system of the present invention mainly consists of three components: a server, a terminal, and a user. The server collects learning data from multiple educational institutions and stores it in an integrated database. The data includes learners' learning history, test results, and attendance information. Based on this information, the server uses a machine learning algorithm to analyze the characteristics of the learners. This analysis identifies each learner's learning style and weaknesses.

[0037] Next, the server generates an optimal learning plan based on the identified learning style and weaknesses. The generated learning plan includes the next topics to study, recommended materials, video explanations, and links to interactive exercises. This information is sent from the server to the terminal.

[0038] The device visually organizes the received learning plan in an easy-to-understand way and presents it to the user through a user interface. The user can follow the learning plan via the device, answer the given questions, and input their progress. For example, when a learner is working on an application problem involving fractions, the device provides a video explanation along with the problem, supporting the learner's progress at their own pace.

[0039] Progress data entered by the user on their device is sent to the server in real time. The server uses this progress data to re-evaluate the learner's understanding and adjust the learning plan as needed. For example, if a learner completes a fraction problem, the server adds the following related topic to the learning plan.

[0040] Furthermore, the server has a function to notify educators and parents of the learners' progress. This allows educators and parents to understand the learners' situation and provide the necessary support.

[0041] As described above, the system of the present invention can personalize the educational process to suit the individual needs of learners and support learning in an integrated and efficient manner.

[0042] The following describes the processing flow.

[0043] Step 1:

[0044] The server collects learning data through APIs from multiple educational institutions. This includes students' learning history, test results, and attendance information, and the collected data is stored in an integrated database.

[0045] Step 2:

[0046] The server runs machine learning algorithms to analyze the collected data. The algorithms identify each child's learning style and weaknesses, and determine the optimal learning approach.

[0047] Step 3:

[0048] The server generates a personalized learning plan based on the identified learning style and weaknesses. The plan includes topics to study next, recommended materials, and video explanations.

[0049] Step 4:

[0050] The device receives the learning plan sent from the server and displays it in a visually easy-to-understand format on the user interface. This display includes learning tasks, progress, and a recommended learning order.

[0051] Step 5:

[0052] The user (child) proceeds with learning according to the learning plan presented via the device. As the learning progresses, they input answers and progress status into the device.

[0053] Step 6:

[0054] The device transmits the entered progress data to the server in real time. This allows the server to check the learner's latest learning status.

[0055] Step 7:

[0056] The server re-evaluates the learning plan based on newly acquired progress data and adjusts it as needed. This maximizes the effectiveness of the learning process.

[0057] Step 8:

[0058] The device receives update information from the server and presents it to the user. Users can always progress through their learning based on the latest learning plan.

[0059] Step 9:

[0060] The server sends notifications to educators and parents, sharing learner progress and newly set learning objectives. This allows stakeholders to continue providing learning support.

[0061] (Example 1)

[0062] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0063] Traditional education systems faced challenges in accurately understanding each student's individual learning style and weaknesses, and providing appropriate instructional plans. Furthermore, they lacked mechanisms for evaluating students' progress in real time and dynamically adjusting instructional plans. Additionally, methods for using generative AI models to provide students with the optimal learning process were insufficient.

[0064] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0065] In this invention, the server includes means for collecting and integrating learning information from multiple educational institutions, means for analyzing the learning information to identify the learning patterns and weaknesses of individual students, and means for using a generated AI model based on the characteristics of each student and optimizing the instruction plan using prompt statements. This makes it possible to provide an optimal instruction plan for each student and efficiently support the learning process.

[0066] "Learning information" refers to educational data such as students' learning history, test results, and attendance records.

[0067] "Educational institution" refers to an organization or group that provides education, such as a school or online education service.

[0068] "Participant" refers to a learner who is taking classes or courses from an educational institution.

[0069] "Learning patterns" refer to characteristics that indicate the tendencies and styles of how learners proceed with their studies.

[0070] A "weakness" refers to an area or concept that a student will find difficult to learn.

[0071] A "teaching plan" refers to a plan of what students should learn and the materials they should use, formulated to improve their learning efficiency.

[0072] A "user interface" refers to a system that provides users with the necessary screens and navigation to interact with the system.

[0073] A "generative AI model" refers to a program or algorithm that uses artificial intelligence to automatically perform a specific task.

[0074] A "prompt statement" refers to input data or instructions used when giving commands to a generative AI model.

[0075] This system aims to collect and integrate learning information from educational institutions for each student and provide them with a suitable instructional plan. To achieve this, the system consists mainly of three elements: a server, terminals, and users.

[0076] The server plays a central role in data collection and analysis. It retrieves learning information from multiple educational institutions via APIs. This learning information includes student learning history, test results, and attendance records. The server uses a database management system (e.g., MySQL® or PostgreSQL) to store this information in an integrated database. Next, the server uses Python libraries (e.g., scikit-learn) to perform machine learning processing to identify students' learning patterns and weaknesses.

[0077] In particular, the server utilizes a generative AI model to generate prompts and develop an optimal lesson plan for the learner. This lesson plan includes what to learn next, recommended materials, video explanations, and links to interactive exercises. For example, if a learner is studying fractions in mathematics, the server will incorporate more advanced fraction challenges and visual aids into the lesson plan according to the learner's progress. An example of a prompt is, "Based on the learner's strengths and weaknesses in mathematical geometry, please suggest the next learning topic and recommended materials."

[0078] The terminal's role is to visually present the lesson plan sent from the server to the user. The terminal uses HTML, CSS, and JavaScript (registered trademark) to build an interface that is easy for learners to understand. Based on the lesson plan, it supports learners in recording their progress and working on problems.

[0079] Users progress through their learning according to the lesson plan presented via their device. Learners input their progress into their device and send it to the server in real time to receive feedback. Based on this transmitted data, the server adjusts the lesson plan as needed. In this way, the system can provide a customized learning experience for each individual learner.

[0080] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0081] Step 1:

[0082] The server collects learning information from each educational institution. This collection is done via an API and includes data such as learning history, test results, and attendance information. Input data is retrieved from the educational institutions' database systems and stored in the server's integrated database. Specifically, the server schedules periodic data retrieval jobs to automatically collect all data.

[0083] Step 2:

[0084] The server analyzes the collected data to identify each student's learning patterns and weaknesses. This analysis uses a Python machine learning library (e.g., scikit-learn). Learning information stored in an integrated database is used as input, and the output provides insights into each student's learning style and areas that need improvement. Specifically, a decision tree is used to classify learner performance and extract weaknesses.

[0085] Step 3:

[0086] The server generates an optimal lesson plan for each student based on identified learning patterns and weaknesses. This process uses a generative AI model, taking appropriate prompt sentences as input. The output is a customized lesson plan that includes topics to be learned, recommended materials, video links, and interactive exercises. Specifically, the AI ​​model makes dynamic decisions and constructs the optimal learning sequence as the lesson plan is generated.

[0087] Step 4:

[0088] The terminal receives the lesson plan sent from the server and presents it visually to the student. It receives data from the server as input and displays a user-friendly interface on the screen as output. Specifically, the terminal uses HTML / CSS to display each item in a card format, allowing the student to intuitively understand the plan.

[0089] Step 5:

[0090] Users progress through the learning process according to the lesson plan displayed on their device, recording their progress. Input includes the student's responses and progress, which are sent from the device to the server. Output includes an evaluation of the progress details, and the lesson plan is adjusted on the server side as needed. Specifically, users input their answers through interactive forms, and their learning progress is automatically tracked.

[0091] Step 6:

[0092] The server analyzes progress data from users and adjusts the lesson plan. It receives learning status data transmitted in real time as input, and as output, it incorporates suggestions for what to learn next and additional materials into the lesson plan. Specifically, the server uses functions to re-evaluate learner data and automatically generates new teaching strategies.

[0093] (Application Example 1)

[0094] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0095] Traditional education systems have faced challenges in providing optimal learning plans tailored to the individual characteristics and progress of each learner, and in the time-consuming process of making such adjustments. This has resulted in learners not receiving appropriate learning and thus the educational effectiveness being underreported. Furthermore, because parents and educators cannot monitor learners' progress in real time, opportunities to provide appropriate support are often missed.

[0096] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0097] In this invention, the server includes means for collecting and integrating learning data from multiple educational institutions, means for identifying the learning style and weaknesses of individual learners, and means for generating appropriate learning plans. This makes it possible to provide personalized learning plans to individual learners, thereby enhancing learning effectiveness, and enabling parents and educators to easily grasp the progress of learners and provide appropriate support.

[0098] "Learning data" refers to information about the educational process, including learners' learning history, test results, and attendance information.

[0099] "Educational facilities" refer to schools, cram schools, and other educational institutions where learning activities take place.

[0100] "Learning style" refers to the tendencies and patterns of how learners understand and process information.

[0101] A "weakness" refers to a specific learning area or skill that a learner finds difficult to understand or acquire.

[0102] A "learning plan" is a plan that outlines the learning content, methods, and progress, generated based on the learner's characteristics.

[0103] "User interface" refers to the display screen and operating methods that learners use to interact with a system and receive information.

[0104] "Progress information" refers to information about the extent to which learners have achieved their learning goals in their current learning situation.

[0105] "Education professionals" refers to teachers and instructors who are responsible for educating learners.

[0106] "Guardian" refers to a person who has a legal or moral responsibility to protect a learner.

[0107] "Dynamic adjustment" refers to changing the learning plan in real time according to the learning progress.

[0108] The system for implementing this invention mainly consists of a server, a terminal, and a user. The server is built on a cloud computing platform (e.g., AWS® or Google® Cloud) and collects learning data from multiple educational institutions and stores it in an integrated database (MySQL or MongoDB). The learning data includes learners' learning history, test results, and attendance information. The server analyzes this data using a generative AI model to identify each learner's learning style and weaknesses. AI models such as TENSORFLOW® or PyTorch are used.

[0109] Based on identified learning styles and weaknesses, the server generates an optimal learning plan. This plan includes what to learn next and recommended educational materials (including audio, video, and interactive elements). The generated data is transmitted to the terminal via data transmission technology and presented on the user interface in a visually clear and easy-to-understand format.

[0110] Users can follow a learning plan, answer assigned questions, and input their progress information via a smartphone or tablet. On the device, for example, video explanations for fraction problems are automatically played to help deepen the learner's understanding. The progress information entered by the user is immediately sent to the server, which then re-evaluates the learner's understanding based on this information and dynamically adjusts the learning plan as needed.

[0111] Furthermore, the server also notifies educators and parents of the learners' progress. This allows educators and parents to understand the learners' learning status and provide necessary support.

[0112] For example, if a sixth-grade student struggles with applied fraction problems, the system will prioritize presenting video lectures and interactive exercises related to that topic. Parents and educators will receive regular progress reports via email, along with detailed explanations and next learning steps. An example of a prompt might be, "Please suggest interactive materials to help the sixth-grade student better understand applied fraction problems."

[0113] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0114] Step 1:

[0115] The server collects learning data from multiple educational institutions. Inputs include learners' learning history, test results, and attendance information, which are stored in a unified database on the cloud. Output is a database containing all learner data.

[0116] Step 2:

[0117] The server uses collected training data to generate an AI model and identify each learner's learning style and weaknesses. The input is learner data from an integrated database, and the AI ​​model performs data analysis. The output is the characteristic analysis results for each learner.

[0118] Step 3:

[0119] The server generates an optimal learning plan based on the analysis results. The input is the results of the learner's characteristics analysis, and the generated learning plan includes what to learn next and recommended educational materials. The output is a customized learning plan.

[0120] Step 4:

[0121] The server sends the generated training plan to the terminal. The input is a customized training plan, which is sent to the terminal over the network. The output is the training plan received by the terminal.

[0122] Step 5:

[0123] The device visually presents the learning plan via a user interface. The input is the learning plan received by the device, which is then processed and formatted to match the display format on the user interface. The output is a visual plan that the learner can view.

[0124] Step 6:

[0125] The user uses a terminal to follow the learning plan, answer questions, and input progress information. Input consists of the user's answers and progress information, which are recorded on the terminal. Output is the progress information sent to the server.

[0126] Step 7:

[0127] The server collects progress information received from the user in real time and dynamically adjusts the learning plan. The input is progress information, the AI ​​model re-evaluates the level of understanding, and reconstructs the learning plan as needed. The output is the adjusted learning plan.

[0128] Step 8:

[0129] The server notifies educators and parents of the learners' progress. Inputs include current progress information and plan adjustments, which are provided via email and dashboards. Outputs are progress notifications received by stakeholders.

[0130] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0131] The system of the present invention consists of a server, a terminal, and a user, and by incorporating a new emotion engine, it is possible to understand the emotional state of the learner and dynamically adjust learning support accordingly.

[0132] The server collects learning data from multiple educational institutions and stores it in an integrated database. This data includes grades, responses, and attendance information. The server analyzes this data to identify learner characteristics. When generating the optimal learning plan, the server uses the results of the characteristic analysis to determine effective learning materials and learning sequences.

[0133] The device provides a user interface and presents the learner with a learning plan sent from the server. This includes text, audio, video, and interactive learning materials. As the learner progresses through the learning process using the device, the camera and microphone installed on the device collect data for the emotion engine. The emotion engine analyzes this data in real time and infers the learner's emotional state from their facial expressions and tone of voice. This information is immediately transmitted to the server.

[0134] The server adjusts the learning plan based on the sentiment data it receives. For example, if a learner is feeling frustrated, the server will lower the difficulty level or suggest an alternative approach. Conversely, if the learner is finding it interesting, it will add more relevant and in-depth learning content.

[0135] Users (learners) can gain a more optimized learning experience based on the adjustable learning plans provided by this system. Furthermore, educators and parents can enhance support at home and school as they receive immediate notifications regarding the learner's emotional state and learning progress.

[0136] For example, suppose a learner is studying a science experiment using video materials, and the emotion engine recognizes the learner's excitement. In this case, the server incorporates additional video content or simulations related to the experiment into the learning plan, further deepening the learner's understanding while maintaining their interest.

[0137] The configuration of this invention enables dynamic learning support that takes into account the learner's emotional state, thereby improving the quality of the learning experience.

[0138] The following describes the processing flow.

[0139] Step 1:

[0140] The server collects learning data from various educational institutions using APIs. This includes students' test results, assignment performance, and attendance information, which allows for the accumulation of detailed learning histories for each student in an integrated database.

[0141] Step 2:

[0142] The server analyzes the collected data using machine learning algorithms. The algorithms identify the learner's learning style (e.g., visual, auditory, experiential) and specific weaknesses, and then begin generating a learning plan optimized for each learner.

[0143] Step 3:

[0144] The server selects learning materials and determines the next steps based on the identified learning style and weaknesses. The generated learning plan includes text, videos, and interactive exercises to deepen understanding, and is sent to the device.

[0145] Step 4:

[0146] The device displays the learning plan sent from the server on its interface. The user then begins learning with personalized materials based on the provided learning plan.

[0147] Step 5:

[0148] The camera and microphone built into the device continuously record the user's facial expressions and voice tone during learning, sending the data to the emotion engine. Based on this real-time data, the emotion engine analyzes the learner's emotional state.

[0149] Step 6:

[0150] The emotion engine sends the analyzed emotion data to the server. The server receives this data and determines whether the learner is feeling excited, irritated, or otherwise upset by the current content.

[0151] Step 7:

[0152] The server comprehensively considers emotional and progress data and adjusts the learning plan in real time as needed. For example, if a learner finds something difficult, it will lower the difficulty level of the materials or provide different explanations.

[0153] Step 8:

[0154] The device instantly updates the user interface to display the adjusted new learning plan. The user can then continue learning according to the new plan.

[0155] Step 9:

[0156] The server instantly notifies educators and parents of the learner's emotional state and progress. This allows educators to understand the learner's situation and provide timely support.

[0157] (Example 2)

[0158] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0159] Conventional educational support systems have struggled to provide dynamic learning plans that adequately consider learners' emotional changes regarding learning, and have therefore failed to provide the optimal learning environment for learners. Furthermore, the difficulty in adjusting plans based on learners' characteristics and understanding their emotional states has resulted in limited educational effectiveness.

[0160] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0161] In this invention, the server includes means for collecting and integrating learning information from multiple educational organizations, means for analyzing the learning information to identify the characteristics and weaknesses of individual learners, and means for collecting learners' emotional states in real time using sensor devices and estimating them using an emotion analysis engine. This makes it possible to provide a dynamic learning plan that takes into account the learners' characteristics and emotional states.

[0162] "Learning information" refers to educational data about learners, such as grades, response history, and attendance records.

[0163] An "educational organization" refers to institutions that provide education to learners, such as schools and cram schools.

[0164] "Characteristics" refer to attributes unique to a learner, such as their learning style, areas of expertise, and progress in their studies.

[0165] A "weakness" refers to an area or skill that a learner finds difficult to understand or acquire.

[0166] A "learning plan" refers to an instructional program that includes learning materials and a learning sequence optimized based on the learner's characteristics and weaknesses.

[0167] An "information display device" refers to a device that visually provides digital content to learners through the screen of a terminal or computer.

[0168] A "sensor device" refers to a device that includes cameras and microphones for detecting and acquiring data on a learner's biometric information, facial expressions, and voice in real time.

[0169] A "sentiment analysis engine" refers to an algorithm or program that analyzes collected biometric information and voice data to estimate the emotional state of a learner.

[0170] "Dynamic adjustment" refers to optimizing and modifying learning plans and content in real time according to learning progress and emotional state.

[0171] The system for implementing this invention consists of a server, a terminal, and a user as its main components. These elements work together to provide dynamic learning support that takes into account the learner's emotional state.

[0172] The server collects learning information from multiple educational organizations and integrates it into a cloud-based database. The server uses machine learning algorithms to analyze the collected information. Specifically, it uses data science libraries in Python (e.g., Pandas, Scikit-learn) to identify learners' characteristics and weaknesses. Based on the analysis results, the server generates a learning plan tailored to each learner.

[0173] The device presents the learner with a learning plan sent from the server via a user interface. This display includes text, audio, video, and interactive elements. The device is equipped with a camera and microphone, which allows for the real-time collection of the learner's facial expressions and voice. The collected data is analyzed by an emotion analysis engine running on the device. For emotion analysis, open-source libraries such as OpenCV and TensorFlow are used, for example. This allows for the estimation of the learner's emotional state (e.g., excitement, frustration, concentration).

[0174] Through this system, users (learners) can progress through their learning based on their learning plan. The server dynamically adjusts the learning content according to the learner's state. For example, if a learner shows frustration with a math problem, the server can reduce the learner's stress by providing easier materials. Conversely, if the learner shows interest, more in-depth learning content will be added.

[0175] Educators and parents receive notifications from the server regarding the learner's progress and emotional state. This allows for more effective learning support at home and in educational settings.

[0176] For example, when a learner shows particular interest in a topic, the emotion engine can recognize this state and the server can immediately adjust to provide additional learning resources related to that topic. Another example of a prompt to input into a generative AI model is, "Please explain in detail the methods for dynamically adjusting the learning plan based on the learner's emotional state."

[0177] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0178] Step 1:

[0179] The server collects learning information from multiple educational institutions. As input, it receives grade data and attendance information from each institution. Upon receiving this information, the server standardizes the format and cleans up any inconsistencies before storing it in the database. The output is a cleaned, integrated dataset.

[0180] Step 2:

[0181] The server analyzes learners' characteristics and weaknesses using an integrated dataset. Organized learning information is required as input. The server applies machine learning algorithms, using clustering and classification techniques to extract characteristics and weaknesses. The output generates a profile indicating each learner's learning style and areas of difficulty.

[0182] Step 3:

[0183] The server generates a personalized learning plan based on the learner's profile. The input requires the learner's characteristic profile and information on available learning materials. The server selects the optimal learning sequence and appropriate materials to construct the plan. The output is a learning plan optimized for the learner.

[0184] Step 4:

[0185] The terminal provides learners with learning plans received from the server through a user interface. Learning plan information from the server is required as input. The terminal presents learning materials that combine text, audio, and video, including interactive elements. As output, learners receive clear guidelines to begin their learning.

[0186] Step 5:

[0187] The device collects and analyzes the learner's emotional state in real time. Input includes facial and audio data obtained from the device's camera and microphone. The emotion analysis engine uses this data to estimate the learner's emotional state. The estimated results regarding the learner's emotional state are sent to the server as output.

[0188] Step 6:

[0189] The server dynamically adjusts the learning plan based on the received emotion data. Its inputs include estimated emotion states and the current learning plan. The server changes the difficulty level of the learning materials or adds new materials as needed. The adjusted learning plan is provided to the learner as output.

[0190] Step 7:

[0191] The user (learner) progresses through the learning process according to a pre-configured learning plan. The input is an optimized learning plan provided by the device. The user acquires knowledge through learning materials, answers questions, and records their progress. The output is an update of the learner's progress, which is then communicated to the educator and guardian.

[0192] (Application Example 2)

[0193] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0194] In today's world, there is a growing need for learning support that takes into account the individual emotional states of learners. While conventional systems adjust based on learners' learning styles and answer results, they have limitations in providing dynamic learning support that reflects emotional states. In particular, when learners exhibit emotions such as frustration or agitation, it is necessary to respond immediately to these situations to enhance learning effectiveness. Furthermore, this requires educators and supervisors to appropriately monitor learners' progress in real time.

[0195] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0196] In this invention, the server includes means for collecting and integrating learning data from multiple educational organizations, means for acquiring and analyzing facial and voice data to recognize the learner's emotional state, and means for dynamically adjusting the learning plan based on the emotional state. This makes it possible to provide an appropriate learning plan that corresponds to the learner's emotional state.

[0197] "Learning data" refers to information related to an individual learner's learning progress, grades, and the materials they use.

[0198] An "educational organization" refers to an institution or facility that provides educational services to learners.

[0199] "Integration" refers to the process of gathering multiple data and pieces of information and combining them into a single database or system.

[0200] "Learning characteristics" refer to the distinctive traits of each learner, such as their areas of expertise, level of understanding, and learning style.

[0201] A "learning plan" refers to a guideline that organizes learning materials, content, and procedures optimized for each individual learner.

[0202] An "information display device" refers to a device, such as a computer or tablet, that presents learning content to learners visually or audibly.

[0203] "Emotional state" refers to the psychological or emotional responses that learners exhibit during the learning process.

[0204] "Facial and voice data" refers to digital information based on the learner's facial movements and voice tone, which is used to analyze their emotional state.

[0205] "Dynamic adjustment" refers to instantly changing plans and actions in response to real-time changes in information and circumstances.

[0206] "Education providers" refer to teachers and instructors who directly engage in educational activities with learners.

[0207] A "supervisor" refers to a parent or educational administrator who has the role of managing, evaluating, and supporting the learning progress of students.

[0208] This invention is implemented as a system that analyzes learners' emotional states and dynamically adjusts their learning plans. A server collects and integrates learning data from various educational institutions, building a centrally managed database. The server also analyzes the collected data and executes algorithms to identify individual learner learning characteristics. This analysis utilizes specialized software or programs (e.g., Python data analysis libraries).

[0209] Furthermore, the device observes the learner's learning progress in real time and acquires facial and audio data using its camera and microphone. This data is processed by an emotion analysis library (e.g., OpenCV, DeepFace) to infer the learner's emotional state. This information is sent to a server, which adjusts the learning plan to an appropriate level based on the emotional state. General information processing devices such as smartphones and personal computers are used for processing on the device side.

[0210] Through this interactive feedback, users (learners) can more effectively advance their learning progress. Learning status and emotional data are also shared with educators and supervisors, improving the efficiency of support activities. For example, if a learner shows frustration while solving a math problem, the system adjusts the difficulty of the problem to provide a more understandable solution.

[0211] Specific examples of prompt statements for generative AI models:

[0212] "Please explain how a robot can determine a child's emotions from their facial expressions and provide appropriate learning support."

[0213] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0214] Step 1:

[0215] The server collects learning data from multiple educational institutions. Inputs include student grades, learning materials, and attendance information. The server integrates this data and stores it in a database for centralized management. The output is an integrated learning database.

[0216] Step 2:

[0217] The server analyzes integrated training data to identify the learning characteristics and weaknesses of individual learners. The input consists of individual data points from the training database. The server then executes an analysis algorithm to process the data and identify learning characteristics and weaknesses. The output is profile information reflecting these learning characteristics and weaknesses.

[0218] Step 3:

[0219] The server generates an optimal learning plan based on identified learning characteristics and weaknesses. The input is learner profile information. The server uses a generation algorithm to determine the optimal learning materials and their learning order, and then creates a learning plan. The output is an individualized learning plan.

[0220] Step 4:

[0221] The terminal presents the learner with a learning plan and monitors their learning progress in real time. The input is the learning plan sent from the server. The terminal displays the plan through a user interface and acquires facial expressions and voice data using a camera and microphone. The output is the learner's learning progress and emotion data.

[0222] Step 5:

[0223] The device analyzes acquired facial and audio data in real time to infer emotional states. Input consists of data acquired by the camera and microphone. The device uses an emotion analysis library (e.g., OpenCV, DeepFace) to perform data calculations to infer emotional states. Output is the analysis result indicating the emotional state.

[0224] Step 6:

[0225] The server receives emotional state data sent from the terminal and dynamically adjusts the learning plan. The input is emotional state data. The server uses an algorithm to modify the plan as needed. The output is the updated learning plan.

[0226] Step 7:

[0227] The user (learner) continues learning based on the updated learning plan. The input is the updated learning plan sent from the server. The user deepens their learning through the provided interactive materials and notifies the educator and supervisor of their progress. The output is learning progress information.

[0228] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0229] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0230] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0231] [Second Embodiment]

[0232] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0233] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0234] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0235] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0236] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0237] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0238] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0239] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0240] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0241] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0242] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0243] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0244] The system of the present invention mainly consists of three components: a server, a terminal, and a user. The server collects learning data from multiple educational institutions and stores it in an integrated database. The data includes learners' learning history, test results, and attendance information. Based on this information, the server uses a machine learning algorithm to analyze the characteristics of the learners. This analysis identifies each learner's learning style and weaknesses.

[0245] Next, the server generates an optimal learning plan based on the identified learning style and weaknesses. The generated learning plan includes the next topics to study, recommended materials, video explanations, and links to interactive exercises. This information is sent from the server to the terminal.

[0246] The device visually organizes the received learning plan in an easy-to-understand way and presents it to the user through a user interface. The user can follow the learning plan via the device, answer the given questions, and input their progress. For example, when a learner is working on an application problem involving fractions, the device provides a video explanation along with the problem, supporting the learner's progress at their own pace.

[0247] Progress data entered by the user on their device is sent to the server in real time. The server uses this progress data to re-evaluate the learner's understanding and adjust the learning plan as needed. For example, if a learner completes a fraction problem, the server adds the following related topic to the learning plan.

[0248] Furthermore, the server has a function to notify educators and parents of the learners' progress. This allows educators and parents to understand the learners' situation and provide the necessary support.

[0249] As described above, the system of the present invention can personalize the educational process to suit the individual needs of learners and support learning in an integrated and efficient manner.

[0250] The following describes the processing flow.

[0251] Step 1:

[0252] The server collects learning data through APIs from multiple educational institutions. This includes students' learning history, test results, and attendance information, and the collected data is stored in an integrated database.

[0253] Step 2:

[0254] The server runs machine learning algorithms to analyze the collected data. The algorithms identify each child's learning style and weaknesses, and determine the optimal learning approach.

[0255] Step 3:

[0256] The server generates a personalized learning plan based on the identified learning style and weaknesses. The plan includes topics to study next, recommended materials, and video explanations.

[0257] Step 4:

[0258] The device receives the learning plan sent from the server and displays it in a visually easy-to-understand format on the user interface. This display includes learning tasks, progress, and a recommended learning order.

[0259] Step 5:

[0260] The user (child) proceeds with learning according to the learning plan presented via the device. As the learning progresses, they input answers and progress status into the device.

[0261] Step 6:

[0262] The device transmits the entered progress data to the server in real time. This allows the server to check the learner's latest learning status.

[0263] Step 7:

[0264] The server re-evaluates the learning plan based on newly acquired progress data and adjusts it as needed. This maximizes the effectiveness of the learning process.

[0265] Step 8:

[0266] The device receives update information from the server and presents it to the user. Users can always progress through their learning based on the latest learning plan.

[0267] Step 9:

[0268] The server sends notifications to educators and parents, sharing learner progress and newly set learning objectives. This allows stakeholders to continue providing learning support.

[0269] (Example 1)

[0270] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0271] Traditional education systems faced challenges in accurately understanding each student's individual learning style and weaknesses, and providing appropriate instructional plans. Furthermore, they lacked mechanisms for evaluating students' progress in real time and dynamically adjusting instructional plans. Additionally, methods for using generative AI models to provide students with the optimal learning process were insufficient.

[0272] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0273] In this invention, the server includes means for collecting and integrating learning information from multiple educational institutions, means for analyzing the learning information to identify the learning patterns and weaknesses of individual students, and means for using a generated AI model based on the characteristics of each student and optimizing the instruction plan using prompt statements. This makes it possible to provide an optimal instruction plan for each student and efficiently support the learning process.

[0274] "Learning information" refers to educational data such as students' learning history, test results, and attendance records.

[0275] "Educational institution" refers to an organization or group that provides education, such as a school or online education service.

[0276] "Participant" refers to a learner who is taking classes or courses from an educational institution.

[0277] "Learning patterns" refer to characteristics that indicate the tendencies and styles of how learners proceed with their studies.

[0278] A "weakness" refers to an area or concept that a student will find difficult to learn.

[0279] A "teaching plan" refers to a plan of what students should learn and the materials they should use, formulated to improve their learning efficiency.

[0280] A "user interface" refers to a system that provides users with the necessary screens and navigation to interact with the system.

[0281] A "generative AI model" refers to a program or algorithm that uses artificial intelligence to automatically perform a specific task.

[0282] A "prompt statement" refers to input data or instructions used when giving commands to a generative AI model.

[0283] This system aims to collect and integrate learning information from educational institutions for each student and provide them with a suitable instructional plan. To achieve this, the system consists mainly of three elements: a server, terminals, and users.

[0284] The server plays a central role in data collection and analysis. It acquires learning information from multiple educational institutions through APIs. This learning information includes learners' learning histories, test results, attendance information, etc. The server uses a database management system (e.g., MySQL or PostgreSQL) to store this information in an integrated database. Next, the server performs machine learning processing using Python libraries (e.g., scikit-learn) to identify learners' learning patterns and weaknesses.

[0285] In particular, the server utilizes a generative AI model to generate prompt texts for formulating an optimal guidance plan for the learners. This guidance plan includes what to learn next, recommended teaching materials, video explanations, and interactive exercise links. As a specific example, when a learner is studying fractions in mathematics, the server incorporates more advanced fraction problems and visual teaching materials into the guidance plan according to the learner's progress. An example of a prompt text is "Please propose the next learning topics and recommended teaching materials based on the characteristics and weaknesses of the learner in geometry of mathematics."

[0286] The terminal has the role of visually presenting the guidance plan sent from the server to the user. The terminal constructs an interface using HTML, CSS, and JavaScript to make it easy for the learners to understand. Based on the guidance plan, it supports the learners in recording their progress and performing operations when dealing with problems.

[0287] The user progresses with learning according to the guidance plan presented through the terminal. The learner inputs their progress during the course into the terminal and sends it to the server in real time to obtain feedback. Based on this transmitted data, the server adjusts the guidance plan as needed. In the above manner, the system can provide a customized learning experience for each learner.

[0288] The flow of the specific processing in Example 1 will be described using FIG. 11.

[0289] Step 1:

[0290] The server collects learning information from each educational institution. This collection is done via an API and includes data such as learning history, test results, and attendance information. Input data is retrieved from the educational institutions' database systems and stored in the server's integrated database. Specifically, the server schedules periodic data retrieval jobs to automatically collect all data.

[0291] Step 2:

[0292] The server analyzes the collected data to identify each student's learning patterns and weaknesses. This analysis uses a Python machine learning library (e.g., scikit-learn). Learning information stored in an integrated database is used as input, and the output provides insights into each student's learning style and areas that need improvement. Specifically, a decision tree is used to classify learner performance and extract weaknesses.

[0293] Step 3:

[0294] The server generates an optimal lesson plan for each student based on identified learning patterns and weaknesses. This process uses a generative AI model, taking appropriate prompt sentences as input. The output is a customized lesson plan that includes topics to be learned, recommended materials, video links, and interactive exercises. Specifically, the AI ​​model makes dynamic decisions and constructs the optimal learning sequence as the lesson plan is generated.

[0295] Step 4:

[0296] The terminal receives the lesson plan sent from the server and presents it visually to the student. It receives data from the server as input and displays a user-friendly interface on the screen as output. Specifically, the terminal uses HTML / CSS to display each item in a card format, allowing the student to intuitively understand the plan.

[0297] Step 5:

[0298] Users progress through the learning process according to the lesson plan displayed on their device, recording their progress. Input includes the student's responses and progress, which are sent from the device to the server. Output includes an evaluation of the progress details, and the lesson plan is adjusted on the server side as needed. Specifically, users input their answers through interactive forms, and their learning progress is automatically tracked.

[0299] Step 6:

[0300] The server analyzes progress data from users and adjusts the lesson plan. It receives learning status data transmitted in real time as input, and as output, it incorporates suggestions for what to learn next and additional materials into the lesson plan. Specifically, the server uses functions to re-evaluate learner data and automatically generates new teaching strategies.

[0301] (Application Example 1)

[0302] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0303] Traditional education systems have faced challenges in providing optimal learning plans tailored to the individual characteristics and progress of each learner, and in the time-consuming process of making such adjustments. This has resulted in learners not receiving appropriate learning and thus the educational effectiveness being underreported. Furthermore, because parents and educators cannot monitor learners' progress in real time, opportunities to provide appropriate support are often missed.

[0304] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0305] In this invention, the server includes means for collecting and integrating learning data from multiple educational institutions, means for identifying the learning styles and weaknesses of individual learners, and means for generating appropriate learning plans. This enables the provision of personalized learning plans for individual learners, enhancing the learning effect, and allowing parents and educational staff to easily grasp the progress of learners and provide appropriate support.

[0306] "Learning data" refers to information related to the educational process, including the learning history, test results, and attendance information of learners.

[0307] "Educational institution" refers to schools, cram schools, and other educational institutions where learning activities are carried out.

[0308] "Learning style" refers to the tendency or pattern of how a learner understands and processes information.

[0309] "Weakness" refers to specific learning areas or skills in which learners have difficulty understanding or acquiring.

[0310] "Learning plan" refers to a plan regarding learning content, methods, and progress, generated based on the characteristics of learners.

[0311] "User interface" refers to the display screen and operating means for learners to interact with the system and receive information.

[0312] "Progress information" refers to information on how much a learner has achieved the learning goal in the current learning situation.

[0313] "Educational staff" refers to teachers and instructors responsible for the education of learners.

[0314] "Parent" refers to a person who has legal or moral responsibility for protecting a learner.

[0315] "Adjust dynamically" refers to changing the learning plan in real time according to the learning situation.

[0316] The system for implementing this invention mainly consists of a server, a terminal, and a user. The server is built on a cloud computing platform (e.g., AWS or Google Cloud) and collects learning data from multiple educational institutions and stores it in an integrated database (MySQL or MongoDB). The learning data includes learners' learning history, test results, and attendance information. The server analyzes this data using a generative AI model to identify each learner's learning style and weaknesses. TensorFlow or PyTorch are used for the AI ​​model.

[0317] Based on identified learning styles and weaknesses, the server generates an optimal learning plan. This plan includes what to learn next and recommended educational materials (including audio, video, and interactive elements). The generated data is transmitted to the terminal via data transmission technology and presented on the user interface in a visually clear and easy-to-understand format.

[0318] Users can follow a learning plan, answer assigned questions, and input their progress information via a smartphone or tablet. On the device, for example, video explanations for fraction problems are automatically played to help deepen the learner's understanding. The progress information entered by the user is immediately sent to the server, which then re-evaluates the learner's understanding based on this information and dynamically adjusts the learning plan as needed.

[0319] Furthermore, the server also notifies educators and parents of the learners' progress. This allows educators and parents to understand the learners' learning status and provide necessary support.

[0320] For example, if a sixth-grade student struggles with applied fraction problems, the system will prioritize presenting video lectures and interactive exercises related to that topic. Parents and educators will receive regular progress reports via email, along with detailed explanations and next learning steps. An example of a prompt might be, "Please suggest interactive materials to help the sixth-grade student better understand applied fraction problems."

[0321] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0322] Step 1:

[0323] The server collects learning data from multiple educational institutions. Inputs include learners' learning history, test results, and attendance information, which are stored in a unified database on the cloud. Output is a database containing all learner data.

[0324] Step 2:

[0325] The server uses collected training data to generate an AI model and identify each learner's learning style and weaknesses. The input is learner data from an integrated database, and the AI ​​model performs data analysis. The output is the characteristic analysis results for each learner.

[0326] Step 3:

[0327] The server generates an optimal learning plan based on the analysis results. The input is the results of the learner's characteristics analysis, and the generated learning plan includes what to learn next and recommended educational materials. The output is a customized learning plan.

[0328] Step 4:

[0329] The server sends the generated training plan to the terminal. The input is a customized training plan, which is sent to the terminal over the network. The output is the training plan received by the terminal.

[0330] Step 5:

[0331] The device visually presents the learning plan via a user interface. The input is the learning plan received by the device, which is then processed and formatted to match the display format on the user interface. The output is a visual plan that the learner can view.

[0332] Step 6:

[0333] The user uses a terminal to follow the learning plan, answer questions, and input progress information. Input consists of the user's answers and progress information, which are recorded on the terminal. Output is the progress information sent to the server.

[0334] Step 7:

[0335] The server collects progress information received from the user in real time and dynamically adjusts the learning plan. The input is progress information, the AI ​​model re-evaluates the level of understanding, and reconstructs the learning plan as needed. The output is the adjusted learning plan.

[0336] Step 8:

[0337] The server notifies educators and parents of the learners' progress. Inputs include current progress information and plan adjustments, which are provided via email and dashboards. Outputs are progress notifications received by stakeholders.

[0338] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0339] The system of the present invention consists of a server, a terminal, and a user, and by incorporating a new emotion engine, it is possible to understand the emotional state of the learner and dynamically adjust learning support accordingly.

[0340] The server collects learning data from multiple educational institutions and stores it in an integrated database. This data includes grades, responses, and attendance information. The server analyzes this data to identify learner characteristics. When generating the optimal learning plan, the server uses the results of the characteristic analysis to determine effective learning materials and learning sequences.

[0341] The device provides a user interface and presents the learner with a learning plan sent from the server. This includes text, audio, video, and interactive learning materials. As the learner progresses through the learning process using the device, the camera and microphone installed on the device collect data for the emotion engine. The emotion engine analyzes this data in real time and infers the learner's emotional state from their facial expressions and tone of voice. This information is immediately transmitted to the server.

[0342] The server adjusts the learning plan based on the sentiment data it receives. For example, if a learner is feeling frustrated, the server will lower the difficulty level or suggest an alternative approach. Conversely, if the learner is finding it interesting, it will add more relevant and in-depth learning content.

[0343] Users (learners) can gain a more optimized learning experience based on the adjustable learning plans provided by this system. Furthermore, educators and parents can enhance support at home and school as they receive immediate notifications regarding the learner's emotional state and learning progress.

[0344] For example, suppose a learner is studying a science experiment using video materials, and the emotion engine recognizes the learner's excitement. In this case, the server incorporates additional video content or simulations related to the experiment into the learning plan, further deepening the learner's understanding while maintaining their interest.

[0345] The configuration of this invention enables dynamic learning support that takes into account the learner's emotional state, thereby improving the quality of the learning experience.

[0346] The following describes the processing flow.

[0347] Step 1:

[0348] The server collects learning data from various educational institutions using APIs. This includes students' test results, assignment performance, and attendance information, which allows for the accumulation of detailed learning histories for each student in an integrated database.

[0349] Step 2:

[0350] The server analyzes the collected data using machine learning algorithms. The algorithms identify the learner's learning style (e.g., visual, auditory, experiential) and specific weaknesses, and then begin generating a learning plan optimized for each learner.

[0351] Step 3:

[0352] The server selects learning materials and determines the next steps based on the identified learning style and weaknesses. The generated learning plan includes text, videos, and interactive exercises to deepen understanding, and is sent to the device.

[0353] Step 4:

[0354] The device displays the learning plan sent from the server on its interface. The user then begins learning with personalized materials based on the provided learning plan.

[0355] Step 5:

[0356] The camera and microphone built into the device continuously record the user's facial expressions and voice tone during learning, sending the data to the emotion engine. Based on this real-time data, the emotion engine analyzes the learner's emotional state.

[0357] Step 6:

[0358] The emotion engine sends the analyzed emotion data to the server. The server receives this data and determines whether the learner is feeling excited, irritated, or otherwise upset by the current content.

[0359] Step 7:

[0360] The server comprehensively considers emotional and progress data and adjusts the learning plan in real time as needed. For example, if a learner finds something difficult, it will lower the difficulty level of the materials or provide different explanations.

[0361] Step 8:

[0362] The device instantly updates the user interface to display the adjusted new learning plan. The user can then continue learning according to the new plan.

[0363] Step 9:

[0364] The server instantly notifies educators and parents of the learner's emotional state and progress. This allows educators to understand the learner's situation and provide timely support.

[0365] (Example 2)

[0366] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0367] Conventional educational support systems have struggled to provide dynamic learning plans that adequately consider learners' emotional changes regarding learning, and have therefore failed to provide the optimal learning environment for learners. Furthermore, the difficulty in adjusting plans based on learners' characteristics and understanding their emotional states has resulted in limited educational effectiveness.

[0368] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0369] In this invention, the server includes means for collecting and integrating learning information from multiple educational organizations, means for analyzing the learning information to identify the characteristics and weaknesses of individual learners, and means for collecting learners' emotional states in real time using sensor devices and estimating them using an emotion analysis engine. This makes it possible to provide a dynamic learning plan that takes into account the learners' characteristics and emotional states.

[0370] "Learning information" refers to educational data about learners, such as grades, response history, and attendance records.

[0371] An "educational organization" refers to institutions that provide education to learners, such as schools and cram schools.

[0372] "Characteristics" refer to attributes unique to a learner, such as their learning style, areas of expertise, and progress in their studies.

[0373] A "weakness" refers to an area or skill that a learner finds difficult to understand or acquire.

[0374] A "learning plan" refers to an instructional program that includes learning materials and a learning sequence optimized based on the learner's characteristics and weaknesses.

[0375] An "information display device" refers to a device that visually provides digital content to learners through the screen of a terminal or computer.

[0376] A "sensor device" refers to a device that includes cameras and microphones for detecting and acquiring data on a learner's biometric information, facial expressions, and voice in real time.

[0377] A "sentiment analysis engine" refers to an algorithm or program that analyzes collected biometric information and voice data to estimate the emotional state of a learner.

[0378] "Dynamic adjustment" refers to optimizing and modifying learning plans and content in real time according to learning progress and emotional state.

[0379] The system for implementing this invention consists of a server, a terminal, and a user as its main components. These elements work together to provide dynamic learning support that takes into account the learner's emotional state.

[0380] The server collects learning information from multiple educational organizations and integrates it into a cloud-based database. The server uses machine learning algorithms to analyze the collected information. Specifically, it uses data science libraries in Python (e.g., Pandas, Scikit-learn) to identify learners' characteristics and weaknesses. Based on the analysis results, the server generates a learning plan tailored to each learner.

[0381] The device presents the learner with a learning plan sent from the server via a user interface. This display includes text, audio, video, and interactive elements. The device is equipped with a camera and microphone, which allows for the real-time collection of the learner's facial expressions and voice. The collected data is analyzed by an emotion analysis engine running on the device. For emotion analysis, open-source libraries such as OpenCV and TensorFlow are used, for example. This allows for the estimation of the learner's emotional state (e.g., excitement, frustration, concentration).

[0382] Through this system, users (learners) can progress through their learning based on their learning plan. The server dynamically adjusts the learning content according to the learner's state. For example, if a learner shows frustration with a math problem, the server can reduce the learner's stress by providing easier materials. Conversely, if the learner shows interest, more in-depth learning content will be added.

[0383] Educators and parents receive notifications from the server regarding the learner's progress and emotional state. This allows for more effective learning support at home and in educational settings.

[0384] For example, when a learner shows particular interest in a topic, the emotion engine can recognize this state and the server can immediately adjust to provide additional learning resources related to that topic. Another example of a prompt to input into a generative AI model is, "Please explain in detail the methods for dynamically adjusting the learning plan based on the learner's emotional state."

[0385] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0386] Step 1:

[0387] The server collects learning information from multiple educational institutions. As input, it receives grade data and attendance information from each institution. Upon receiving this information, the server standardizes the format and cleans up any inconsistencies before storing it in the database. The output is a cleaned, integrated dataset.

[0388] Step 2:

[0389] The server analyzes learners' characteristics and weaknesses using an integrated dataset. Organized learning information is required as input. The server applies machine learning algorithms, using clustering and classification techniques to extract characteristics and weaknesses. The output generates a profile indicating each learner's learning style and areas of difficulty.

[0390] Step 3:

[0391] The server generates a personalized learning plan based on the learner's profile. The input requires the learner's characteristic profile and information on available learning materials. The server selects the optimal learning sequence and appropriate materials to construct the plan. The output is a learning plan optimized for the learner.

[0392] Step 4:

[0393] The terminal provides learners with learning plans received from the server through a user interface. Learning plan information from the server is required as input. The terminal presents learning materials that combine text, audio, and video, including interactive elements. As output, learners receive clear guidelines to begin their learning.

[0394] Step 5:

[0395] The device collects and analyzes the learner's emotional state in real time. Input includes facial and audio data obtained from the device's camera and microphone. The emotion analysis engine uses this data to estimate the learner's emotional state. The estimated results regarding the learner's emotional state are sent to the server as output.

[0396] Step 6:

[0397] The server dynamically adjusts the learning plan based on the received emotion data. Its inputs include estimated emotion states and the current learning plan. The server changes the difficulty level of the learning materials or adds new materials as needed. The adjusted learning plan is provided to the learner as output.

[0398] Step 7:

[0399] The user (learner) progresses through the learning process according to a pre-configured learning plan. The input is an optimized learning plan provided by the device. The user acquires knowledge through learning materials, answers questions, and records their progress. The output is an update of the learner's progress, which is then communicated to the educator and guardian.

[0400] (Application Example 2)

[0401] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the smart glasses 214 as the "terminal".

[0402] In today's world, there is a growing need for learning support that takes into account the individual emotional states of learners. While conventional systems adjust based on learners' learning styles and answer results, they have limitations in providing dynamic learning support that reflects emotional states. In particular, when learners exhibit emotions such as frustration or agitation, it is necessary to respond immediately to these situations to enhance learning effectiveness. Furthermore, this requires educators and supervisors to appropriately monitor learners' progress in real time.

[0403] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0404] In this invention, the server includes means for collecting and integrating learning data from multiple educational organizations, means for acquiring and analyzing facial and voice data to recognize the learner's emotional state, and means for dynamically adjusting the learning plan based on the emotional state. This makes it possible to provide an appropriate learning plan that corresponds to the learner's emotional state.

[0405] "Learning data" refers to information related to an individual learner's learning progress, grades, and the materials they use.

[0406] An "educational organization" refers to an institution or facility that provides educational services to learners.

[0407] "Integration" refers to the process of gathering multiple data and pieces of information and combining them into a single database or system.

[0408] "Learning characteristics" refer to the distinctive traits of each learner, such as their areas of expertise, level of understanding, and learning style.

[0409] A "learning plan" refers to a guideline that organizes learning materials, content, and procedures optimized for each individual learner.

[0410] An "information display device" refers to a device, such as a computer or tablet, that presents learning content to learners visually or audibly.

[0411] "Emotional state" refers to the psychological or emotional responses that learners exhibit during the learning process.

[0412] "Facial and voice data" refers to digital information based on the learner's facial movements and voice tone, which is used to analyze their emotional state.

[0413] "Dynamic adjustment" refers to instantly changing plans and actions in response to real-time changes in information and circumstances.

[0414] "Education providers" refer to teachers and instructors who directly engage in educational activities with learners.

[0415] A "supervisor" refers to a parent or educational administrator who has the role of managing, evaluating, and supporting the learning progress of students.

[0416] This invention is implemented as a system that analyzes learners' emotional states and dynamically adjusts their learning plans. A server collects and integrates learning data from various educational institutions, building a centrally managed database. The server also analyzes the collected data and executes algorithms to identify individual learner learning characteristics. This analysis utilizes specialized software or programs (e.g., Python data analysis libraries).

[0417] Furthermore, the device observes the learner's learning progress in real time and acquires facial and audio data using its camera and microphone. This data is processed by an emotion analysis library (e.g., OpenCV, DeepFace) to infer the learner's emotional state. This information is sent to a server, which adjusts the learning plan to an appropriate level based on the emotional state. General information processing devices such as smartphones and personal computers are used for processing on the device side.

[0418] Through this interactive feedback, users (learners) can more effectively advance their learning progress. Learning status and emotional data are also shared with educators and supervisors, improving the efficiency of support activities. For example, if a learner shows frustration while solving a math problem, the system adjusts the difficulty of the problem to provide a more understandable solution.

[0419] Specific examples of prompt statements for generative AI models:

[0420] "Please explain how a robot can determine a child's emotions from their facial expressions and provide appropriate learning support."

[0421] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0422] Step 1:

[0423] The server collects learning data from multiple educational institutions. Inputs include student grades, learning materials, and attendance information. The server integrates this data and stores it in a database for centralized management. The output is an integrated learning database.

[0424] Step 2:

[0425] The server analyzes integrated training data to identify the learning characteristics and weaknesses of individual learners. The input consists of individual data points from the training database. The server then executes an analysis algorithm to process the data and identify learning characteristics and weaknesses. The output is profile information reflecting these learning characteristics and weaknesses.

[0426] Step 3:

[0427] The server generates an optimal learning plan based on identified learning characteristics and weaknesses. The input is learner profile information. The server uses a generation algorithm to determine the optimal learning materials and their learning order, and then creates a learning plan. The output is an individualized learning plan.

[0428] Step 4:

[0429] The terminal presents the learner with a learning plan and monitors their learning progress in real time. The input is the learning plan sent from the server. The terminal displays the plan through a user interface and acquires facial expressions and voice data using a camera and microphone. The output is the learner's learning progress and emotion data.

[0430] Step 5:

[0431] The device analyzes acquired facial and audio data in real time to infer emotional states. Input consists of data acquired by the camera and microphone. The device uses an emotion analysis library (e.g., OpenCV, DeepFace) to perform data calculations to infer emotional states. Output is the analysis result indicating the emotional state.

[0432] Step 6:

[0433] The server receives emotional state data sent from the terminal and dynamically adjusts the learning plan. The input is emotional state data. The server uses an algorithm to modify the plan as needed. The output is the updated learning plan.

[0434] Step 7:

[0435] The user (learner) continues learning based on the updated learning plan. The input is the updated learning plan sent from the server. The user deepens their learning through the provided interactive materials and notifies the educator and supervisor of their progress. The output is learning progress information.

[0436] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0437] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0438] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0439] [Third Embodiment]

[0440] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0441] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0442] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0443] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0444] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0445] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0446] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0447] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0448] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0449] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0450] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0451] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0452] The system of the present invention mainly consists of three components: a server, a terminal, and a user. The server collects learning data from multiple educational institutions and stores it in an integrated database. The data includes learners' learning history, test results, and attendance information. Based on this information, the server uses a machine learning algorithm to analyze the characteristics of the learners. This analysis identifies each learner's learning style and weaknesses.

[0453] Next, the server generates an optimal learning plan based on the identified learning style and weaknesses. The generated learning plan includes the next topics to study, recommended materials, video explanations, and links to interactive exercises. This information is sent from the server to the terminal.

[0454] The device visually organizes the received learning plan in an easy-to-understand way and presents it to the user through a user interface. The user can follow the learning plan via the device, answer the given questions, and input their progress. For example, when a learner is working on an application problem involving fractions, the device provides a video explanation along with the problem, supporting the learner's progress at their own pace.

[0455] Progress data entered by the user on their device is sent to the server in real time. The server uses this progress data to re-evaluate the learner's understanding and adjust the learning plan as needed. For example, if a learner completes a fraction problem, the server adds the following related topic to the learning plan.

[0456] Furthermore, the server has a function to notify educators and parents of the learners' progress. This allows educators and parents to understand the learners' situation and provide the necessary support.

[0457] As described above, the system of the present invention can personalize the educational process to suit the individual needs of learners and support learning in an integrated and efficient manner.

[0458] The following describes the processing flow.

[0459] Step 1:

[0460] The server collects learning data through APIs from multiple educational institutions. This includes students' learning history, test results, and attendance information, and the collected data is stored in an integrated database.

[0461] Step 2:

[0462] The server runs machine learning algorithms to analyze the collected data. The algorithms identify each child's learning style and weaknesses, and determine the optimal learning approach.

[0463] Step 3:

[0464] The server generates a personalized learning plan based on the identified learning style and weaknesses. The plan includes topics to study next, recommended materials, and video explanations.

[0465] Step 4:

[0466] The device receives the learning plan sent from the server and displays it in a visually easy-to-understand format on the user interface. This display includes learning tasks, progress, and a recommended learning order.

[0467] Step 5:

[0468] The user (child) proceeds with learning according to the learning plan presented via the device. As the learning progresses, they input answers and progress status into the device.

[0469] Step 6:

[0470] The device transmits the entered progress data to the server in real time. This allows the server to check the learner's latest learning status.

[0471] Step 7:

[0472] The server re-evaluates the learning plan based on newly acquired progress data and adjusts it as needed. This maximizes the effectiveness of the learning process.

[0473] Step 8:

[0474] The device receives update information from the server and presents it to the user. Users can always progress through their learning based on the latest learning plan.

[0475] Step 9:

[0476] The server sends notifications to educators and parents, sharing learner progress and newly set learning objectives. This allows stakeholders to continue providing learning support.

[0477] (Example 1)

[0478] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0479] Traditional education systems faced challenges in accurately understanding each student's individual learning style and weaknesses, and providing appropriate instructional plans. Furthermore, they lacked mechanisms for evaluating students' progress in real time and dynamically adjusting instructional plans. Additionally, methods for using generative AI models to provide students with the optimal learning process were insufficient.

[0480] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0481] In this invention, the server includes means for collecting and integrating learning information from multiple educational institutions, means for analyzing the learning information to identify the learning patterns and weaknesses of individual students, and means for using a generated AI model based on the characteristics of each student and optimizing the instruction plan using prompt statements. This makes it possible to provide an optimal instruction plan for each student and efficiently support the learning process.

[0482] "Learning information" refers to educational data such as students' learning history, test results, and attendance records.

[0483] "Educational institution" refers to an organization or group that provides education, such as a school or online education service.

[0484] "Participant" refers to a learner who is taking classes or courses from an educational institution.

[0485] "Learning patterns" refer to characteristics that indicate the tendencies and styles of how learners proceed with their studies.

[0486] A "weakness" refers to an area or concept that a student will find difficult to learn.

[0487] A "teaching plan" refers to a plan of what students should learn and the materials they should use, formulated to improve their learning efficiency.

[0488] A "user interface" refers to a system that provides users with the necessary screens and navigation to interact with the system.

[0489] A "generative AI model" refers to a program or algorithm that uses artificial intelligence to automatically perform a specific task.

[0490] A "prompt statement" refers to input data or instructions used when giving commands to a generative AI model.

[0491] This system aims to collect and integrate learning information from educational institutions for each student and provide them with a suitable instructional plan. To achieve this, the system consists mainly of three elements: a server, terminals, and users.

[0492] The server plays a central role in data collection and analysis. It retrieves learning information from multiple educational institutions via APIs. This learning information includes student learning history, test results, and attendance records. The server uses a database management system (e.g., MySQL or PostgreSQL) to store this information in an integrated database. Next, the server uses Python libraries (e.g., scikit-learn) to perform machine learning processing to identify student learning patterns and weaknesses.

[0493] In particular, the server utilizes a generative AI model to generate prompts and develop an optimal lesson plan for the learner. This lesson plan includes what to learn next, recommended materials, video explanations, and links to interactive exercises. For example, if a learner is studying fractions in mathematics, the server will incorporate more advanced fraction challenges and visual aids into the lesson plan according to the learner's progress. An example of a prompt is, "Based on the learner's strengths and weaknesses in mathematical geometry, please suggest the next learning topic and recommended materials."

[0494] The terminal's role is to visually present the lesson plan sent from the server to the user. The terminal uses HTML, CSS, and JavaScript to build an interface that is easy for learners to understand. Based on the lesson plan, it supports learners in recording their progress and working on problems.

[0495] Users progress through their learning according to the lesson plan presented via their device. Learners input their progress into their device and send it to the server in real time to receive feedback. Based on this transmitted data, the server adjusts the lesson plan as needed. In this way, the system can provide a customized learning experience for each individual learner.

[0496] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0497] Step 1:

[0498] The server collects learning information from each educational institution. This collection is done via an API and includes data such as learning history, test results, and attendance information. Input data is retrieved from the educational institutions' database systems and stored in the server's integrated database. Specifically, the server schedules periodic data retrieval jobs to automatically collect all data.

[0499] Step 2:

[0500] The server analyzes the collected data to identify each student's learning patterns and weaknesses. This analysis uses a Python machine learning library (e.g., scikit-learn). Learning information stored in an integrated database is used as input, and the output provides insights into each student's learning style and areas that need improvement. Specifically, a decision tree is used to classify learner performance and extract weaknesses.

[0501] Step 3:

[0502] The server generates an optimal lesson plan for each student based on identified learning patterns and weaknesses. This process uses a generative AI model, taking appropriate prompt sentences as input. The output is a customized lesson plan that includes topics to be learned, recommended materials, video links, and interactive exercises. Specifically, the AI ​​model makes dynamic decisions and constructs the optimal learning sequence as the lesson plan is generated.

[0503] Step 4:

[0504] The terminal receives the lesson plan sent from the server and presents it visually to the student. It receives data from the server as input and displays a user-friendly interface on the screen as output. Specifically, the terminal uses HTML / CSS to display each item in a card format, allowing the student to intuitively understand the plan.

[0505] Step 5:

[0506] Users progress through the learning process according to the lesson plan displayed on their device, recording their progress. Input includes the student's responses and progress, which are sent from the device to the server. Output includes an evaluation of the progress details, and the lesson plan is adjusted on the server side as needed. Specifically, users input their answers through interactive forms, and their learning progress is automatically tracked.

[0507] Step 6:

[0508] The server analyzes progress data from users and adjusts the lesson plan. It receives learning status data transmitted in real time as input, and as output, it incorporates suggestions for what to learn next and additional materials into the lesson plan. Specifically, the server uses functions to re-evaluate learner data and automatically generates new teaching strategies.

[0509] (Application Example 1)

[0510] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0511] Traditional education systems have faced challenges in providing optimal learning plans tailored to the individual characteristics and progress of each learner, and in the time-consuming process of making such adjustments. This has resulted in learners not receiving appropriate learning and thus the educational effectiveness being underreported. Furthermore, because parents and educators cannot monitor learners' progress in real time, opportunities to provide appropriate support are often missed.

[0512] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0513] In this invention, the server includes means for collecting and integrating learning data from multiple educational institutions, means for identifying the learning style and weaknesses of individual learners, and means for generating appropriate learning plans. This makes it possible to provide personalized learning plans to individual learners, thereby enhancing learning effectiveness, and enabling parents and educators to easily grasp the progress of learners and provide appropriate support.

[0514] "Learning data" refers to information about the educational process, including learners' learning history, test results, and attendance information.

[0515] "Educational facilities" refer to schools, cram schools, and other educational institutions where learning activities take place.

[0516] "Learning style" refers to the tendencies and patterns of how learners understand and process information.

[0517] A "weakness" refers to a specific learning area or skill that a learner finds difficult to understand or acquire.

[0518] A "learning plan" is a plan that outlines the learning content, methods, and progress, generated based on the learner's characteristics.

[0519] "User interface" refers to the display screen and operating methods that learners use to interact with a system and receive information.

[0520] "Progress information" refers to information about the extent to which learners have achieved their learning goals in their current learning situation.

[0521] "Education professionals" refers to teachers and instructors who are responsible for educating learners.

[0522] "Guardian" refers to a person who has a legal or moral responsibility to protect a learner.

[0523] "Dynamic adjustment" refers to changing the learning plan in real time according to the learning progress.

[0524] The system for implementing this invention mainly consists of a server, a terminal, and a user. The server is built on a cloud computing platform (e.g., AWS or Google Cloud) and collects learning data from multiple educational institutions and stores it in an integrated database (MySQL or MongoDB). The learning data includes learners' learning history, test results, and attendance information. The server analyzes this data using a generative AI model to identify each learner's learning style and weaknesses. TensorFlow or PyTorch are used for the AI ​​model.

[0525] Based on identified learning styles and weaknesses, the server generates an optimal learning plan. This plan includes what to learn next and recommended educational materials (including audio, video, and interactive elements). The generated data is transmitted to the terminal via data transmission technology and presented on the user interface in a visually clear and easy-to-understand format.

[0526] Users can follow a learning plan, answer assigned questions, and input their progress information via a smartphone or tablet. On the device, for example, video explanations for fraction problems are automatically played to help deepen the learner's understanding. The progress information entered by the user is immediately sent to the server, which then re-evaluates the learner's understanding based on this information and dynamically adjusts the learning plan as needed.

[0527] Furthermore, the server also notifies educators and parents of the learners' progress. This allows educators and parents to understand the learners' learning status and provide necessary support.

[0528] For example, if a sixth-grade student struggles with applied fraction problems, the system will prioritize presenting video lectures and interactive exercises related to that topic. Parents and educators will receive regular progress reports via email, along with detailed explanations and next learning steps. An example of a prompt might be, "Please suggest interactive materials to help the sixth-grade student better understand applied fraction problems."

[0529] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0530] Step 1:

[0531] The server collects learning data from multiple educational institutions. Inputs include learners' learning history, test results, and attendance information, which are stored in a unified database on the cloud. Output is a database containing all learner data.

[0532] Step 2:

[0533] The server uses collected training data to generate an AI model and identify each learner's learning style and weaknesses. The input is learner data from an integrated database, and the AI ​​model performs data analysis. The output is the characteristic analysis results for each learner.

[0534] Step 3:

[0535] The server generates an optimal learning plan based on the analysis results. The input is the results of the learner's characteristics analysis, and the generated learning plan includes what to learn next and recommended educational materials. The output is a customized learning plan.

[0536] Step 4:

[0537] The server sends the generated training plan to the terminal. The input is a customized training plan, which is sent to the terminal over the network. The output is the training plan received by the terminal.

[0538] Step 5:

[0539] The device visually presents the learning plan via a user interface. The input is the learning plan received by the device, which is then processed and formatted to match the display format on the user interface. The output is a visual plan that the learner can view.

[0540] Step 6:

[0541] The user uses a terminal to follow the learning plan, answer questions, and input progress information. Input consists of the user's answers and progress information, which are recorded on the terminal. Output is the progress information sent to the server.

[0542] Step 7:

[0543] The server collects progress information received from the user in real time and dynamically adjusts the learning plan. The input is progress information, the AI ​​model re-evaluates the level of understanding, and reconstructs the learning plan as needed. The output is the adjusted learning plan.

[0544] Step 8:

[0545] The server notifies educators and parents of the learners' progress. Inputs include current progress information and plan adjustments, which are provided via email and dashboards. Outputs are progress notifications received by stakeholders.

[0546] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0547] The system of the present invention consists of a server, a terminal, and a user, and by incorporating a new emotion engine, it is possible to understand the emotional state of the learner and dynamically adjust learning support accordingly.

[0548] The server collects learning data from multiple educational institutions and stores it in an integrated database. This data includes grades, responses, and attendance information. The server analyzes this data to identify learner characteristics. When generating the optimal learning plan, the server uses the results of the characteristic analysis to determine effective learning materials and learning sequences.

[0549] The device provides a user interface and presents the learner with a learning plan sent from the server. This includes text, audio, video, and interactive learning materials. As the learner progresses through the learning process using the device, the camera and microphone installed on the device collect data for the emotion engine. The emotion engine analyzes this data in real time and infers the learner's emotional state from their facial expressions and tone of voice. This information is immediately transmitted to the server.

[0550] The server adjusts the learning plan based on the sentiment data it receives. For example, if a learner is feeling frustrated, the server will lower the difficulty level or suggest an alternative approach. Conversely, if the learner is finding it interesting, it will add more relevant and in-depth learning content.

[0551] Users (learners) can gain a more optimized learning experience based on the adjustable learning plans provided by this system. Furthermore, educators and parents can enhance support at home and school as they receive immediate notifications regarding the learner's emotional state and learning progress.

[0552] For example, suppose a learner is studying a science experiment using video materials, and the emotion engine recognizes the learner's excitement. In this case, the server incorporates additional video content or simulations related to the experiment into the learning plan, further deepening the learner's understanding while maintaining their interest.

[0553] The configuration of this invention enables dynamic learning support that takes into account the learner's emotional state, thereby improving the quality of the learning experience.

[0554] The following describes the processing flow.

[0555] Step 1:

[0556] The server collects learning data from various educational institutions using APIs. This includes students' test results, assignment performance, and attendance information, which allows for the accumulation of detailed learning histories for each student in an integrated database.

[0557] Step 2:

[0558] The server analyzes the collected data using machine learning algorithms. The algorithms identify the learner's learning style (e.g., visual, auditory, experiential) and specific weaknesses, and then begin generating a learning plan optimized for each learner.

[0559] Step 3:

[0560] The server selects learning materials and determines the next steps based on the identified learning style and weaknesses. The generated learning plan includes text, videos, and interactive exercises to deepen understanding, and is sent to the device.

[0561] Step 4:

[0562] The device displays the learning plan sent from the server on its interface. The user then begins learning with personalized materials based on the provided learning plan.

[0563] Step 5:

[0564] The camera and microphone built into the device continuously record the user's facial expressions and voice tone during learning, sending the data to the emotion engine. Based on this real-time data, the emotion engine analyzes the learner's emotional state.

[0565] Step 6:

[0566] The emotion engine sends the analyzed emotion data to the server. The server receives this data and determines whether the learner is feeling excited, irritated, or otherwise upset by the current content.

[0567] Step 7:

[0568] The server comprehensively considers emotional and progress data and adjusts the learning plan in real time as needed. For example, if a learner finds something difficult, it will lower the difficulty level of the materials or provide different explanations.

[0569] Step 8:

[0570] The device instantly updates the user interface to display the adjusted new learning plan. The user can then continue learning according to the new plan.

[0571] Step 9:

[0572] The server instantly notifies educators and parents of the learner's emotional state and progress. This allows educators to understand the learner's situation and provide timely support.

[0573] (Example 2)

[0574] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0575] Conventional educational support systems have struggled to provide dynamic learning plans that adequately consider learners' emotional changes regarding learning, and have therefore failed to provide the optimal learning environment for learners. Furthermore, the difficulty in adjusting plans based on learners' characteristics and understanding their emotional states has resulted in limited educational effectiveness.

[0576] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0577] In this invention, the server includes means for collecting and integrating learning information from multiple educational organizations, means for analyzing the learning information to identify the characteristics and weaknesses of individual learners, and means for collecting learners' emotional states in real time using sensor devices and estimating them using an emotion analysis engine. This makes it possible to provide a dynamic learning plan that takes into account the learners' characteristics and emotional states.

[0578] "Learning information" refers to educational data about learners, such as grades, response history, and attendance records.

[0579] An "educational organization" refers to institutions that provide education to learners, such as schools and cram schools.

[0580] "Characteristics" refer to attributes unique to a learner, such as their learning style, areas of expertise, and progress in their studies.

[0581] A "weakness" refers to an area or skill that a learner finds difficult to understand or acquire.

[0582] A "learning plan" refers to an instructional program that includes learning materials and a learning sequence optimized based on the learner's characteristics and weaknesses.

[0583] An "information display device" refers to a device that visually provides digital content to learners through the screen of a terminal or computer.

[0584] A "sensor device" refers to a device that includes cameras and microphones for detecting and acquiring data on a learner's biometric information, facial expressions, and voice in real time.

[0585] A "sentiment analysis engine" refers to an algorithm or program that analyzes collected biometric information and voice data to estimate the emotional state of a learner.

[0586] "Dynamic adjustment" refers to optimizing and modifying learning plans and content in real time according to learning progress and emotional state.

[0587] The system for implementing this invention consists of a server, a terminal, and a user as its main components. These elements work together to provide dynamic learning support that takes into account the learner's emotional state.

[0588] The server collects learning information from multiple educational organizations and integrates it into a cloud-based database. The server uses machine learning algorithms to analyze the collected information. Specifically, it uses data science libraries in Python (e.g., Pandas, Scikit-learn) to identify learners' characteristics and weaknesses. Based on the analysis results, the server generates a learning plan tailored to each learner.

[0589] The device presents the learner with a learning plan sent from the server via a user interface. This display includes text, audio, video, and interactive elements. The device is equipped with a camera and microphone, which allows for the real-time collection of the learner's facial expressions and voice. The collected data is analyzed by an emotion analysis engine running on the device. For emotion analysis, open-source libraries such as OpenCV and TensorFlow are used, for example. This allows for the estimation of the learner's emotional state (e.g., excitement, frustration, concentration).

[0590] Through this system, users (learners) can progress through their learning based on their learning plan. The server dynamically adjusts the learning content according to the learner's state. For example, if a learner shows frustration with a math problem, the server can reduce the learner's stress by providing easier materials. Conversely, if the learner shows interest, more in-depth learning content will be added.

[0591] Educators and parents receive notifications from the server regarding the learner's progress and emotional state. This allows for more effective learning support at home and in educational settings.

[0592] For example, when a learner shows particular interest in a topic, the emotion engine can recognize this state and the server can immediately adjust to provide additional learning resources related to that topic. Another example of a prompt to input into a generative AI model is, "Please explain in detail the methods for dynamically adjusting the learning plan based on the learner's emotional state."

[0593] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0594] Step 1:

[0595] The server collects learning information from multiple educational institutions. As input, it receives grade data and attendance information from each institution. Upon receiving this information, the server standardizes the format and cleans up any inconsistencies before storing it in the database. The output is a cleaned, integrated dataset.

[0596] Step 2:

[0597] The server analyzes learners' characteristics and weaknesses using an integrated dataset. Organized learning information is required as input. The server applies machine learning algorithms, using clustering and classification techniques to extract characteristics and weaknesses. The output generates a profile indicating each learner's learning style and areas of difficulty.

[0598] Step 3:

[0599] The server generates a personalized learning plan based on the learner's profile. The input requires the learner's characteristic profile and information on available learning materials. The server selects the optimal learning sequence and appropriate materials to construct the plan. The output is a learning plan optimized for the learner.

[0600] Step 4:

[0601] The terminal provides learners with learning plans received from the server through a user interface. Learning plan information from the server is required as input. The terminal presents learning materials that combine text, audio, and video, including interactive elements. As output, learners receive clear guidelines to begin their learning.

[0602] Step 5:

[0603] The device collects and analyzes the learner's emotional state in real time. Input includes facial and audio data obtained from the device's camera and microphone. The emotion analysis engine uses this data to estimate the learner's emotional state. The estimated results regarding the learner's emotional state are sent to the server as output.

[0604] Step 6:

[0605] The server dynamically adjusts the learning plan based on the received emotion data. Its inputs include estimated emotion states and the current learning plan. The server changes the difficulty level of the learning materials or adds new materials as needed. The adjusted learning plan is provided to the learner as output.

[0606] Step 7:

[0607] The user (learner) progresses through the learning process according to a pre-configured learning plan. The input is an optimized learning plan provided by the device. The user acquires knowledge through learning materials, answers questions, and records their progress. The output is an update of the learner's progress, which is then communicated to the educator and guardian.

[0608] (Application Example 2)

[0609] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0610] In today's world, there is a growing need for learning support that takes into account the individual emotional states of learners. While conventional systems adjust based on learners' learning styles and answer results, they have limitations in providing dynamic learning support that reflects emotional states. In particular, when learners exhibit emotions such as frustration or agitation, it is necessary to respond immediately to these situations to enhance learning effectiveness. Furthermore, this requires educators and supervisors to appropriately monitor learners' progress in real time.

[0611] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0612] In this invention, the server includes means for collecting and integrating learning data from multiple educational organizations, means for acquiring and analyzing facial and voice data to recognize the learner's emotional state, and means for dynamically adjusting the learning plan based on the emotional state. This makes it possible to provide an appropriate learning plan that corresponds to the learner's emotional state.

[0613] "Learning data" refers to information related to an individual learner's learning progress, grades, and the materials they use.

[0614] An "educational organization" refers to an institution or facility that provides educational services to learners.

[0615] "Integration" refers to the process of gathering multiple data and pieces of information and combining them into a single database or system.

[0616] "Learning characteristics" refer to the distinctive traits of each learner, such as their areas of expertise, level of understanding, and learning style.

[0617] A "learning plan" refers to a guideline that organizes learning materials, content, and procedures optimized for each individual learner.

[0618] An "information display device" refers to a device, such as a computer or tablet, that presents learning content to learners visually or audibly.

[0619] "Emotional state" refers to the psychological or emotional responses that learners exhibit during the learning process.

[0620] "Facial and voice data" refers to digital information based on the learner's facial movements and voice tone, which is used to analyze their emotional state.

[0621] "Dynamic adjustment" refers to instantly changing plans and actions in response to real-time changes in information and circumstances.

[0622] "Education providers" refer to teachers and instructors who directly engage in educational activities with learners.

[0623] A "supervisor" refers to a parent or educational administrator who has the role of managing, evaluating, and supporting the learning progress of students.

[0624] This invention is implemented as a system that analyzes learners' emotional states and dynamically adjusts their learning plans. A server collects and integrates learning data from various educational institutions, building a centrally managed database. The server also analyzes the collected data and executes algorithms to identify individual learner learning characteristics. This analysis utilizes specialized software or programs (e.g., Python data analysis libraries).

[0625] Furthermore, the device observes the learner's learning progress in real time and acquires facial and audio data using its camera and microphone. This data is processed by an emotion analysis library (e.g., OpenCV, DeepFace) to infer the learner's emotional state. This information is sent to a server, which adjusts the learning plan to an appropriate level based on the emotional state. General information processing devices such as smartphones and personal computers are used for processing on the device side.

[0626] Through this interactive feedback, users (learners) can more effectively advance their learning progress. Learning status and emotional data are also shared with educators and supervisors, improving the efficiency of support activities. For example, if a learner shows frustration while solving a math problem, the system adjusts the difficulty of the problem to provide a more understandable solution.

[0627] Specific examples of prompt statements for generative AI models:

[0628] "Please explain how a robot can determine a child's emotions from their facial expressions and provide appropriate learning support."

[0629] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0630] Step 1:

[0631] The server collects learning data from multiple educational institutions. Inputs include student grades, learning materials, and attendance information. The server integrates this data and stores it in a database for centralized management. The output is an integrated learning database.

[0632] Step 2:

[0633] The server analyzes integrated training data to identify the learning characteristics and weaknesses of individual learners. The input consists of individual data points from the training database. The server then executes an analysis algorithm to process the data and identify learning characteristics and weaknesses. The output is profile information reflecting these learning characteristics and weaknesses.

[0634] Step 3:

[0635] The server generates an optimal learning plan based on identified learning characteristics and weaknesses. The input is learner profile information. The server uses a generation algorithm to determine the optimal learning materials and their learning order, and then creates a learning plan. The output is an individualized learning plan.

[0636] Step 4:

[0637] The terminal presents the learner with a learning plan and monitors their learning progress in real time. The input is the learning plan sent from the server. The terminal displays the plan through a user interface and acquires facial expressions and voice data using a camera and microphone. The output is the learner's learning progress and emotion data.

[0638] Step 5:

[0639] The device analyzes acquired facial and audio data in real time to infer emotional states. Input consists of data acquired by the camera and microphone. The device uses an emotion analysis library (e.g., OpenCV, DeepFace) to perform data calculations to infer emotional states. Output is the analysis result indicating the emotional state.

[0640] Step 6:

[0641] The server receives emotional state data sent from the terminal and dynamically adjusts the learning plan. The input is emotional state data. The server uses an algorithm to modify the plan as needed. The output is the updated learning plan.

[0642] Step 7:

[0643] The user (learner) continues learning based on the updated learning plan. The input is the updated learning plan sent from the server. The user deepens their learning through the provided interactive materials and notifies the educator and supervisor of their progress. The output is learning progress information.

[0644] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0645] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0646] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0647] [Fourth Embodiment]

[0648] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0649] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0650] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0651] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0652] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0653] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0654] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0655] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0656] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0657] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0658] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0659] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0660] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0661] The system of the present invention mainly consists of three components: a server, a terminal, and a user. The server collects learning data from multiple educational institutions and stores it in an integrated database. The data includes learners' learning history, test results, and attendance information. Based on this information, the server uses a machine learning algorithm to analyze the characteristics of the learners. This analysis identifies each learner's learning style and weaknesses.

[0662] Next, the server generates an optimal learning plan based on the identified learning style and weaknesses. The generated learning plan includes the next topics to study, recommended materials, video explanations, and links to interactive exercises. This information is sent from the server to the terminal.

[0663] The device visually organizes the received learning plan in an easy-to-understand way and presents it to the user through a user interface. The user can follow the learning plan via the device, answer the given questions, and input their progress. For example, when a learner is working on an application problem involving fractions, the device provides a video explanation along with the problem, supporting the learner's progress at their own pace.

[0664] Progress data entered by the user on their device is sent to the server in real time. The server uses this progress data to re-evaluate the learner's understanding and adjust the learning plan as needed. For example, if a learner completes a fraction problem, the server adds the following related topic to the learning plan.

[0665] Furthermore, the server has a function to notify educators and parents of the learners' progress. This allows educators and parents to understand the learners' situation and provide the necessary support.

[0666] As described above, the system of the present invention can personalize the educational process to suit the individual needs of learners and support learning in an integrated and efficient manner.

[0667] The following describes the processing flow.

[0668] Step 1:

[0669] The server collects learning data through APIs from multiple educational institutions. This includes students' learning history, test results, and attendance information, and the collected data is stored in an integrated database.

[0670] Step 2:

[0671] The server runs machine learning algorithms to analyze the collected data. The algorithms identify each child's learning style and weaknesses, and determine the optimal learning approach.

[0672] Step 3:

[0673] The server generates a personalized learning plan based on the identified learning style and weaknesses. The plan includes topics to study next, recommended materials, and video explanations.

[0674] Step 4:

[0675] The device receives the learning plan sent from the server and displays it in a visually easy-to-understand format on the user interface. This display includes learning tasks, progress, and a recommended learning order.

[0676] Step 5:

[0677] The user (child) proceeds with learning according to the learning plan presented via the device. As the learning progresses, they input answers and progress status into the device.

[0678] Step 6:

[0679] The device transmits the entered progress data to the server in real time. This allows the server to check the learner's latest learning status.

[0680] Step 7:

[0681] The server re-evaluates the learning plan based on newly acquired progress data and adjusts it as needed. This maximizes the effectiveness of the learning process.

[0682] Step 8:

[0683] The device receives update information from the server and presents it to the user. Users can always progress through their learning based on the latest learning plan.

[0684] Step 9:

[0685] The server sends notifications to educators and parents, sharing learner progress and newly set learning objectives. This allows stakeholders to continue providing learning support.

[0686] (Example 1)

[0687] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0688] Traditional education systems faced challenges in accurately understanding each student's individual learning style and weaknesses, and providing appropriate instructional plans. Furthermore, they lacked mechanisms for evaluating students' progress in real time and dynamically adjusting instructional plans. Additionally, methods for using generative AI models to provide students with the optimal learning process were insufficient.

[0689] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0690] In this invention, the server includes means for collecting and integrating learning information from multiple educational institutions, means for analyzing the learning information to identify the learning patterns and weaknesses of individual students, and means for using a generated AI model based on the characteristics of each student and optimizing the instruction plan using prompt statements. This makes it possible to provide an optimal instruction plan for each student and efficiently support the learning process.

[0691] "Learning information" refers to educational data such as students' learning history, test results, and attendance records.

[0692] "Educational institution" refers to an organization or group that provides education, such as a school or online education service.

[0693] "Participant" refers to a learner who is taking classes or courses from an educational institution.

[0694] "Learning patterns" refer to characteristics that indicate the tendencies and styles of how learners proceed with their studies.

[0695] A "weakness" refers to an area or concept that a student will find difficult to learn.

[0696] A "teaching plan" refers to a plan of what students should learn and the materials they should use, formulated to improve their learning efficiency.

[0697] A "user interface" refers to a system that provides users with the necessary screens and navigation to interact with the system.

[0698] A "generative AI model" refers to a program or algorithm that uses artificial intelligence to automatically perform a specific task.

[0699] A "prompt statement" refers to input data or instructions used when giving commands to a generative AI model.

[0700] This system aims to collect and integrate learning information from educational institutions for each student and provide them with a suitable instructional plan. To achieve this, the system consists mainly of three elements: a server, terminals, and users.

[0701] The server plays a central role in data collection and analysis. It retrieves learning information from multiple educational institutions via APIs. This learning information includes student learning history, test results, and attendance records. The server uses a database management system (e.g., MySQL or PostgreSQL) to store this information in an integrated database. Next, the server uses Python libraries (e.g., scikit-learn) to perform machine learning processing to identify student learning patterns and weaknesses.

[0702] In particular, the server utilizes a generative AI model to generate prompts and develop an optimal lesson plan for the learner. This lesson plan includes what to learn next, recommended materials, video explanations, and links to interactive exercises. For example, if a learner is studying fractions in mathematics, the server will incorporate more advanced fraction challenges and visual aids into the lesson plan according to the learner's progress. An example of a prompt is, "Based on the learner's strengths and weaknesses in mathematical geometry, please suggest the next learning topic and recommended materials."

[0703] The terminal's role is to visually present the lesson plan sent from the server to the user. The terminal uses HTML, CSS, and JavaScript to build an interface that is easy for learners to understand. Based on the lesson plan, it supports learners in recording their progress and working on problems.

[0704] Users progress through their learning according to the lesson plan presented via their device. Learners input their progress into their device and send it to the server in real time to receive feedback. Based on this transmitted data, the server adjusts the lesson plan as needed. In this way, the system can provide a customized learning experience for each individual learner.

[0705] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0706] Step 1:

[0707] The server collects learning information from each educational institution. This collection is done via an API and includes data such as learning history, test results, and attendance information. Input data is retrieved from the educational institutions' database systems and stored in the server's integrated database. Specifically, the server schedules periodic data retrieval jobs to automatically collect all data.

[0708] Step 2:

[0709] The server analyzes the collected data to identify each student's learning patterns and weaknesses. This analysis uses a Python machine learning library (e.g., scikit-learn). Learning information stored in an integrated database is used as input, and the output provides insights into each student's learning style and areas that need improvement. Specifically, a decision tree is used to classify learner performance and extract weaknesses.

[0710] Step 3:

[0711] The server generates an optimal lesson plan for each student based on identified learning patterns and weaknesses. This process uses a generative AI model, taking appropriate prompt sentences as input. The output is a customized lesson plan that includes topics to be learned, recommended materials, video links, and interactive exercises. Specifically, the AI ​​model makes dynamic decisions and constructs the optimal learning sequence as the lesson plan is generated.

[0712] Step 4:

[0713] The terminal receives the lesson plan sent from the server and presents it visually to the student. It receives data from the server as input and displays a user-friendly interface on the screen as output. Specifically, the terminal uses HTML / CSS to display each item in a card format, allowing the student to intuitively understand the plan.

[0714] Step 5:

[0715] Users progress through the learning process according to the lesson plan displayed on their device, recording their progress. Input includes the student's responses and progress, which are sent from the device to the server. Output includes an evaluation of the progress details, and the lesson plan is adjusted on the server side as needed. Specifically, users input their answers through interactive forms, and their learning progress is automatically tracked.

[0716] Step 6:

[0717] The server analyzes progress data from users and adjusts the lesson plan. It receives learning status data transmitted in real time as input, and as output, it incorporates suggestions for what to learn next and additional materials into the lesson plan. Specifically, the server uses functions to re-evaluate learner data and automatically generates new teaching strategies.

[0718] (Application Example 1)

[0719] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0720] Traditional education systems have faced challenges in providing optimal learning plans tailored to the individual characteristics and progress of each learner, and in the time-consuming process of making such adjustments. This has resulted in learners not receiving appropriate learning and thus the educational effectiveness being underreported. Furthermore, because parents and educators cannot monitor learners' progress in real time, opportunities to provide appropriate support are often missed.

[0721] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0722] In this invention, the server includes means for collecting and integrating learning data from multiple educational institutions, means for identifying the learning style and weaknesses of individual learners, and means for generating appropriate learning plans. This makes it possible to provide personalized learning plans to individual learners, thereby enhancing learning effectiveness, and enabling parents and educators to easily grasp the progress of learners and provide appropriate support.

[0723] "Learning data" refers to information about the educational process, including learners' learning history, test results, and attendance information.

[0724] "Educational facilities" refer to schools, cram schools, and other educational institutions where learning activities take place.

[0725] "Learning style" refers to the tendencies and patterns of how learners understand and process information.

[0726] A "weakness" refers to a specific learning area or skill that a learner finds difficult to understand or acquire.

[0727] A "learning plan" is a plan that outlines the learning content, methods, and progress, generated based on the learner's characteristics.

[0728] "User interface" refers to the display screen and operating methods that learners use to interact with a system and receive information.

[0729] "Progress information" refers to information about the extent to which learners have achieved their learning goals in their current learning situation.

[0730] "Education professionals" refers to teachers and instructors who are responsible for educating learners.

[0731] "Guardian" refers to a person who has a legal or moral responsibility to protect a learner.

[0732] "Dynamic adjustment" refers to changing the learning plan in real time according to the learning progress.

[0733] The system for implementing this invention mainly consists of a server, a terminal, and a user. The server is built on a cloud computing platform (e.g., AWS or Google Cloud) and collects learning data from multiple educational institutions and stores it in an integrated database (MySQL or MongoDB). The learning data includes learners' learning history, test results, and attendance information. The server analyzes this data using a generative AI model to identify each learner's learning style and weaknesses. TensorFlow or PyTorch are used for the AI ​​model.

[0734] Based on identified learning styles and weaknesses, the server generates an optimal learning plan. This plan includes what to learn next and recommended educational materials (including audio, video, and interactive elements). The generated data is transmitted to the terminal via data transmission technology and presented on the user interface in a visually clear and easy-to-understand format.

[0735] Users can follow a learning plan, answer assigned questions, and input their progress information via a smartphone or tablet. On the device, for example, video explanations for fraction problems are automatically played to help deepen the learner's understanding. The progress information entered by the user is immediately sent to the server, which then re-evaluates the learner's understanding based on this information and dynamically adjusts the learning plan as needed.

[0736] Furthermore, the server also notifies educators and parents of the learners' progress. This allows educators and parents to understand the learners' learning status and provide necessary support.

[0737] For example, if a sixth-grade student struggles with applied fraction problems, the system will prioritize presenting video lectures and interactive exercises related to that topic. Parents and educators will receive regular progress reports via email, along with detailed explanations and next learning steps. An example of a prompt might be, "Please suggest interactive materials to help the sixth-grade student better understand applied fraction problems."

[0738] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0739] Step 1:

[0740] The server collects learning data from multiple educational institutions. Inputs include learners' learning history, test results, and attendance information, which are stored in a unified database on the cloud. Output is a database containing all learner data.

[0741] Step 2:

[0742] The server uses collected training data to generate an AI model and identify each learner's learning style and weaknesses. The input is learner data from an integrated database, and the AI ​​model performs data analysis. The output is the characteristic analysis results for each learner.

[0743] Step 3:

[0744] The server generates an optimal learning plan based on the analysis results. The input is the results of the learner's characteristics analysis, and the generated learning plan includes what to learn next and recommended educational materials. The output is a customized learning plan.

[0745] Step 4:

[0746] The server sends the generated training plan to the terminal. The input is a customized training plan, which is sent to the terminal over the network. The output is the training plan received by the terminal.

[0747] Step 5:

[0748] The device visually presents the learning plan via a user interface. The input is the learning plan received by the device, which is then processed and formatted to match the display format on the user interface. The output is a visual plan that the learner can view.

[0749] Step 6:

[0750] The user uses a terminal to follow the learning plan, answer questions, and input progress information. Input consists of the user's answers and progress information, which are recorded on the terminal. Output is the progress information sent to the server.

[0751] Step 7:

[0752] The server collects progress information received from the user in real time and dynamically adjusts the learning plan. The input is progress information, the AI ​​model re-evaluates the level of understanding, and reconstructs the learning plan as needed. The output is the adjusted learning plan.

[0753] Step 8:

[0754] The server notifies educators and parents of the learners' progress. Inputs include current progress information and plan adjustments, which are provided via email and dashboards. Outputs are progress notifications received by stakeholders.

[0755] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0756] The system of the present invention consists of a server, a terminal, and a user, and by incorporating a new emotion engine, it is possible to understand the emotional state of the learner and dynamically adjust learning support accordingly.

[0757] The server collects learning data from multiple educational institutions and stores it in an integrated database. This data includes grades, responses, and attendance information. The server analyzes this data to identify learner characteristics. When generating the optimal learning plan, the server uses the results of the characteristic analysis to determine effective learning materials and learning sequences.

[0758] The device provides a user interface and presents the learner with a learning plan sent from the server. This includes text, audio, video, and interactive learning materials. As the learner progresses through the learning process using the device, the camera and microphone installed on the device collect data for the emotion engine. The emotion engine analyzes this data in real time and infers the learner's emotional state from their facial expressions and tone of voice. This information is immediately transmitted to the server.

[0759] The server adjusts the learning plan based on the sentiment data it receives. For example, if a learner is feeling frustrated, the server will lower the difficulty level or suggest an alternative approach. Conversely, if the learner is finding it interesting, it will add more relevant and in-depth learning content.

[0760] Users (learners) can gain a more optimized learning experience based on the adjustable learning plans provided by this system. Furthermore, educators and parents can enhance support at home and school as they receive immediate notifications regarding the learner's emotional state and learning progress.

[0761] For example, suppose a learner is studying a science experiment using video materials, and the emotion engine recognizes the learner's excitement. In this case, the server incorporates additional video content or simulations related to the experiment into the learning plan, further deepening the learner's understanding while maintaining their interest.

[0762] The configuration of this invention enables dynamic learning support that takes into account the learner's emotional state, thereby improving the quality of the learning experience.

[0763] The following describes the processing flow.

[0764] Step 1:

[0765] The server collects learning data from various educational institutions using APIs. This includes students' test results, assignment performance, and attendance information, which allows for the accumulation of detailed learning histories for each student in an integrated database.

[0766] Step 2:

[0767] The server analyzes the collected data using machine learning algorithms. The algorithms identify the learner's learning style (e.g., visual, auditory, experiential) and specific weaknesses, and then begin generating a learning plan optimized for each learner.

[0768] Step 3:

[0769] The server selects learning materials and determines the next steps based on the identified learning style and weaknesses. The generated learning plan includes text, videos, and interactive exercises to deepen understanding, and is sent to the device.

[0770] Step 4:

[0771] The device displays the learning plan sent from the server on its interface. The user then begins learning with personalized materials based on the provided learning plan.

[0772] Step 5:

[0773] The camera and microphone built into the device continuously record the user's facial expressions and voice tone during learning, sending the data to the emotion engine. Based on this real-time data, the emotion engine analyzes the learner's emotional state.

[0774] Step 6:

[0775] The emotion engine sends the analyzed emotion data to the server. The server receives this data and determines whether the learner is feeling excited, irritated, or otherwise upset by the current content.

[0776] Step 7:

[0777] The server comprehensively considers emotional and progress data and adjusts the learning plan in real time as needed. For example, if a learner finds something difficult, it will lower the difficulty level of the materials or provide different explanations.

[0778] Step 8:

[0779] The device instantly updates the user interface to display the adjusted new learning plan. The user can then continue learning according to the new plan.

[0780] Step 9:

[0781] The server instantly notifies educators and parents of the learner's emotional state and progress. This allows educators to understand the learner's situation and provide timely support.

[0782] (Example 2)

[0783] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0784] Conventional educational support systems have struggled to provide dynamic learning plans that adequately consider learners' emotional changes regarding learning, and have therefore failed to provide the optimal learning environment for learners. Furthermore, the difficulty in adjusting plans based on learners' characteristics and understanding their emotional states has resulted in limited educational effectiveness.

[0785] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0786] In this invention, the server includes means for collecting and integrating learning information from multiple educational organizations, means for analyzing the learning information to identify the characteristics and weaknesses of individual learners, and means for collecting learners' emotional states in real time using sensor devices and estimating them using an emotion analysis engine. This makes it possible to provide a dynamic learning plan that takes into account the learners' characteristics and emotional states.

[0787] "Learning information" refers to educational data about learners, such as grades, response history, and attendance records.

[0788] An "educational organization" refers to institutions that provide education to learners, such as schools and cram schools.

[0789] "Characteristics" refer to attributes unique to a learner, such as their learning style, areas of expertise, and progress in their studies.

[0790] A "weakness" refers to an area or skill that a learner finds difficult to understand or acquire.

[0791] A "learning plan" refers to an instructional program that includes learning materials and a learning sequence optimized based on the learner's characteristics and weaknesses.

[0792] An "information display device" refers to a device that visually provides digital content to learners through the screen of a terminal or computer.

[0793] A "sensor device" refers to a device that includes cameras and microphones for detecting and acquiring data on a learner's biometric information, facial expressions, and voice in real time.

[0794] A "sentiment analysis engine" refers to an algorithm or program that analyzes collected biometric information and voice data to estimate the emotional state of a learner.

[0795] "Dynamic adjustment" refers to optimizing and modifying learning plans and content in real time according to learning progress and emotional state.

[0796] The system for implementing this invention consists of a server, a terminal, and a user as its main components. These elements work together to provide dynamic learning support that takes into account the learner's emotional state.

[0797] The server collects learning information from multiple educational organizations and integrates it into a cloud-based database. The server uses machine learning algorithms to analyze the collected information. Specifically, it uses data science libraries in Python (e.g., Pandas, Scikit-learn) to identify learners' characteristics and weaknesses. Based on the analysis results, the server generates a learning plan tailored to each learner.

[0798] The device presents the learner with a learning plan sent from the server via a user interface. This display includes text, audio, video, and interactive elements. The device is equipped with a camera and microphone, which allows for the real-time collection of the learner's facial expressions and voice. The collected data is analyzed by an emotion analysis engine running on the device. For emotion analysis, open-source libraries such as OpenCV and TensorFlow are used, for example. This allows for the estimation of the learner's emotional state (e.g., excitement, frustration, concentration).

[0799] Through this system, users (learners) can progress through their learning based on their learning plan. The server dynamically adjusts the learning content according to the learner's state. For example, if a learner shows frustration with a math problem, the server can reduce the learner's stress by providing easier materials. Conversely, if the learner shows interest, more in-depth learning content will be added.

[0800] Educators and parents receive notifications from the server regarding the learner's progress and emotional state. This allows for more effective learning support at home and in educational settings.

[0801] For example, when a learner shows particular interest in a topic, the emotion engine can recognize this state and the server can immediately adjust to provide additional learning resources related to that topic. Another example of a prompt to input into a generative AI model is, "Please explain in detail the methods for dynamically adjusting the learning plan based on the learner's emotional state."

[0802] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0803] Step 1:

[0804] The server collects learning information from multiple educational institutions. As input, it receives grade data and attendance information from each institution. Upon receiving this information, the server standardizes the format and cleans up any inconsistencies before storing it in the database. The output is a cleaned, integrated dataset.

[0805] Step 2:

[0806] The server analyzes learners' characteristics and weaknesses using an integrated dataset. Organized learning information is required as input. The server applies machine learning algorithms, using clustering and classification techniques to extract characteristics and weaknesses. The output generates a profile indicating each learner's learning style and areas of difficulty.

[0807] Step 3:

[0808] The server generates a personalized learning plan based on the learner's profile. The input requires the learner's characteristic profile and information on available learning materials. The server selects the optimal learning sequence and appropriate materials to construct the plan. The output is a learning plan optimized for the learner.

[0809] Step 4:

[0810] The terminal provides learners with learning plans received from the server through a user interface. Learning plan information from the server is required as input. The terminal presents learning materials that combine text, audio, and video, including interactive elements. As output, learners receive clear guidelines to begin their learning.

[0811] Step 5:

[0812] The device collects and analyzes the learner's emotional state in real time. Input includes facial and audio data obtained from the device's camera and microphone. The emotion analysis engine uses this data to estimate the learner's emotional state. The estimated results regarding the learner's emotional state are sent to the server as output.

[0813] Step 6:

[0814] The server dynamically adjusts the learning plan based on the received emotion data. Its inputs include estimated emotion states and the current learning plan. The server changes the difficulty level of the learning materials or adds new materials as needed. The adjusted learning plan is provided to the learner as output.

[0815] Step 7:

[0816] The user (learner) progresses through the learning process according to a pre-configured learning plan. The input is an optimized learning plan provided by the device. The user acquires knowledge through learning materials, answers questions, and records their progress. The output is an update of the learner's progress, which is then communicated to the educator and guardian.

[0817] (Application Example 2)

[0818] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0819] In today's world, there is a growing need for learning support that takes into account the individual emotional states of learners. While conventional systems adjust based on learners' learning styles and answer results, they have limitations in providing dynamic learning support that reflects emotional states. In particular, when learners exhibit emotions such as frustration or agitation, it is necessary to respond immediately to these situations to enhance learning effectiveness. Furthermore, this requires educators and supervisors to appropriately monitor learners' progress in real time.

[0820] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0821] In this invention, the server includes means for collecting and integrating learning data from multiple educational organizations, means for acquiring and analyzing facial and voice data to recognize the learner's emotional state, and means for dynamically adjusting the learning plan based on the emotional state. This makes it possible to provide an appropriate learning plan that corresponds to the learner's emotional state.

[0822] "Learning data" refers to information related to an individual learner's learning progress, grades, and the materials they use.

[0823] An "educational organization" refers to an institution or facility that provides educational services to learners.

[0824] "Integration" refers to the process of gathering multiple data and pieces of information and combining them into a single database or system.

[0825] "Learning characteristics" refer to the distinctive traits of each learner, such as their areas of expertise, level of understanding, and learning style.

[0826] A "learning plan" refers to a guideline that organizes learning materials, content, and procedures optimized for each individual learner.

[0827] An "information display device" refers to a device, such as a computer or tablet, that presents learning content to learners visually or audibly.

[0828] "Emotional state" refers to the psychological or emotional responses that learners exhibit during the learning process.

[0829] "Facial and voice data" refers to digital information based on the learner's facial movements and voice tone, which is used to analyze their emotional state.

[0830] "Dynamic adjustment" refers to instantly changing plans and actions in response to real-time changes in information and circumstances.

[0831] "Education providers" refer to teachers and instructors who directly engage in educational activities with learners.

[0832] A "supervisor" refers to a parent or educational administrator who has the role of managing, evaluating, and supporting the learning progress of students.

[0833] This invention is implemented as a system that analyzes learners' emotional states and dynamically adjusts their learning plans. A server collects and integrates learning data from various educational institutions, building a centrally managed database. The server also analyzes the collected data and executes algorithms to identify individual learner learning characteristics. This analysis utilizes specialized software or programs (e.g., Python data analysis libraries).

[0834] Furthermore, the device observes the learner's learning progress in real time and acquires facial and audio data using its camera and microphone. This data is processed by an emotion analysis library (e.g., OpenCV, DeepFace) to infer the learner's emotional state. This information is sent to a server, which adjusts the learning plan to an appropriate level based on the emotional state. General information processing devices such as smartphones and personal computers are used for processing on the device side.

[0835] Through this interactive feedback, users (learners) can more effectively advance their learning progress. Learning status and emotional data are also shared with educators and supervisors, improving the efficiency of support activities. For example, if a learner shows frustration while solving a math problem, the system adjusts the difficulty of the problem to provide a more understandable solution.

[0836] Specific examples of prompt statements for generative AI models:

[0837] "Please explain how a robot can determine a child's emotions from their facial expressions and provide appropriate learning support."

[0838] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0839] Step 1:

[0840] The server collects learning data from multiple educational institutions. Inputs include student grades, learning materials, and attendance information. The server integrates this data and stores it in a database for centralized management. The output is an integrated learning database.

[0841] Step 2:

[0842] The server analyzes integrated training data to identify the learning characteristics and weaknesses of individual learners. The input consists of individual data points from the training database. The server then executes an analysis algorithm to process the data and identify learning characteristics and weaknesses. The output is profile information reflecting these learning characteristics and weaknesses.

[0843] Step 3:

[0844] The server generates an optimal learning plan based on identified learning characteristics and weaknesses. The input is learner profile information. The server uses a generation algorithm to determine the optimal learning materials and their learning order, and then creates a learning plan. The output is an individualized learning plan.

[0845] Step 4:

[0846] The terminal presents the learner with a learning plan and monitors their learning progress in real time. The input is the learning plan sent from the server. The terminal displays the plan through a user interface and acquires facial expressions and voice data using a camera and microphone. The output is the learner's learning progress and emotion data.

[0847] Step 5:

[0848] The device analyzes acquired facial and audio data in real time to infer emotional states. Input consists of data acquired by the camera and microphone. The device uses an emotion analysis library (e.g., OpenCV, DeepFace) to perform data calculations to infer emotional states. Output is the analysis result indicating the emotional state.

[0849] Step 6:

[0850] The server receives emotional state data sent from the terminal and dynamically adjusts the learning plan. The input is emotional state data. The server uses an algorithm to modify the plan as needed. The output is the updated learning plan.

[0851] Step 7:

[0852] The user (learner) continues learning based on the updated learning plan. The input is the updated learning plan sent from the server. The user deepens their learning through the provided interactive materials and notifies the educator and supervisor of their progress. The output is learning progress information.

[0853] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0854] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0855] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0856] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0857] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0858] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0859] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0860] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0861] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0862] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0863] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0864] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0865] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0866] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0867] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0868] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0869] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0870] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0871] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0872] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0873] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0874] The following is further disclosed regarding the embodiments described above.

[0875] (Claim 1)

[0876] A means of collecting and integrating learning data from multiple educational institutions,

[0877] A means for analyzing the aforementioned learning data to identify the learning style and weaknesses of individual learners,

[0878] A means for generating an appropriate learning plan based on the identified learning style and weaknesses,

[0879] A means for presenting the aforementioned learning plan through a user interface,

[0880] A means for collecting learner responses and progress data in real time and adjusting the learning plan,

[0881] A means of notifying educators and guardians of the progress of learners,

[0882] A system that includes this.

[0883] (Claim 2)

[0884] The system according to claim 1, wherein the learning plan generation means provides the learning materials included in the generated plan in a form that includes audio, video, and interactive elements.

[0885] (Claim 3)

[0886] The system according to claim 1, wherein the learning plan adjustment means proposes the next learning content according to the learner's learning progress.

[0887] "Example 1"

[0888] (Claim 1)

[0889] A means of collecting and integrating learning information from multiple educational institutions,

[0890] A means for analyzing the aforementioned learning information to identify the learning patterns and weaknesses of individual students,

[0891] A means for generating an appropriate instructional plan based on the identified learning patterns and weaknesses,

[0892] A means for presenting the aforementioned instruction plan through a user interface,

[0893] A means for immediately collecting responses and progress data from participants and adjusting the aforementioned instruction plan,

[0894] A means of reporting the progress of students to educational administrators and parents,

[0895] A means of optimizing the instruction plan using a generative AI model based on the characteristics of individual students and prompt sentences,

[0896] A system that includes this.

[0897] (Claim 2)

[0898] The system according to claim 1, wherein the instruction plan generation means provides the educational materials included in the generated plan in a form that includes sound, video, and interactive elements.

[0899] (Claim 3)

[0900] The system according to claim 1, wherein the instruction plan adjustment means proposes the next topic to be studied according to the student's learning progress.

[0901] "Application Example 1"

[0902] (Claim 1)

[0903] A means of collecting and integrating learning data from multiple educational institutions,

[0904] A means for analyzing the aforementioned learning data to identify the learning style and weaknesses of individual learners,

[0905] Means for generating an appropriate learning plan based on the identified learning style and weaknesses,

[0906] A means for presenting the aforementioned learning plan through a user interface,

[0907] A means for collecting answers and progress information from learners in real time and adjusting the learning plan accordingly,

[0908] A means of notifying educators and parents of the progress of learners,

[0909] A means to evaluate learners' understanding and dynamically adjust the plan,

[0910] A system that includes this.

[0911] (Claim 2)

[0912] The system according to claim 1, wherein the learning plan generation means provides the educational materials included in the generated plan in a form that includes audio, video, and interactive elements.

[0913] (Claim 3)

[0914] The system according to claim 1, wherein the learning plan adjustment means recommends the next learning content according to the learner's learning progress and displays it on the user interface.

[0915] "Example 2 of combining an emotion engine"

[0916] (Claim 1)

[0917] A means of collecting and integrating learning information from multiple educational organizations,

[0918] A means for analyzing the aforementioned learning information to identify the characteristics and weaknesses of individual learners,

[0919] Means for generating an appropriate learning plan based on the identified characteristics and weaknesses,

[0920] Means for presenting the aforementioned learning plan through an information display device,

[0921] A means of collecting learners' emotional states in real time using sensor devices and estimating them using an emotion analysis engine,

[0922] Means for dynamically adjusting the learning plan based on the estimated emotional state,

[0923] A means of notifying educators and guardians of the learner's progress and emotional state,

[0924] A system that includes this.

[0925] (Claim 2)

[0926] The system according to claim 1, wherein the learning plan generation means provides the learning materials included in the generated plan in a form that includes visual, auditory, and interactive elements.

[0927] (Claim 3)

[0928] The system according to claim 1, wherein the learning plan adjustment means proposes the content to be learned next according to the learner's learning progress and emotional state.

[0929] "Application example 2 when combining with an emotional engine"

[0930] (Claim 1)

[0931] A means of collecting and integrating learning data from multiple educational organizations,

[0932] A means for analyzing the aforementioned learning data to identify the learning characteristics and weaknesses of individual learners,

[0933] Means for generating an appropriate learning plan based on the identified learning characteristics and weaknesses,

[0934] Means for presenting the aforementioned learning plan through an information display device,

[0935] In order to recognize the emotional state of learners, a means is provided to acquire and analyze facial and voice data,

[0936] A means for dynamically adjusting the learning plan based on the aforementioned emotional state,

[0937] A means of notifying educators and supervisors of learners' emotional state and learning progress,

[0938] A system that includes this.

[0939] (Claim 2)

[0940] The system according to claim 1, wherein the learning plan generation means provides the learning resources included in the generated plan in a form that includes audio, video, and interactive elements.

[0941] (Claim 3)

[0942] The system according to claim 1, wherein the learning plan adjustment means proposes the next learning content according to the learner's emotional state and learning progress. [Explanation of symbols]

[0943] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. A means of collecting and integrating learning data from multiple educational institutions, A means for analyzing the aforementioned learning data to identify the learning style and weaknesses of individual learners, A means for generating an appropriate learning plan based on the identified learning style and weaknesses, A means for presenting the aforementioned learning plan through a user interface, A means for collecting learner responses and progress data in real time and adjusting the learning plan, A means of notifying educators and guardians of the progress of learners, A system that includes this.

2. The system according to claim 1, wherein the learning plan generation means provides the learning materials included in the generated plan in a form that includes audio, video, and interactive elements.

3. The system according to claim 1, wherein the learning plan adjustment means proposes the next learning content according to the learner's learning progress.

Images