system

Abstract

We provide the system. [Solution] A means of automatically collecting learning progress data from a learning management database, A means of analyzing collected learning progress data to detect students' level of understanding and learning delays, A means for automatically generating individualized instruction plans for each student based on the analysis results, A means for displaying the generated lesson plan on a display device, A method for automatically generating reports on learning progress for parents, A means of analyzing the learning trends of the entire class, A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor and includes steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a character of the chatbot, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In the educational field, it is a great burden for teachers to appropriately grasp in real time the learning progress and understanding level of each student. In particular, it is difficult to quickly discover and effectively address the learning delays of students who require individual guidance, and the conventional methods are time - consuming and laborious. In addition, due to the lack of appropriate reporting means to parents, learning support at home is often not sufficient. The present invention aims to solve these problems and improve the quality of education.

Means for Solving the Problems

[0005] This invention provides a system that automatically collects and analyzes learning progress data from a learning management database to detect each student's level of understanding and learning delays. Based on the analysis results, this system automatically generates individualized instruction plans for each student and displays them on the teacher's display device, enabling efficient individualized instruction. It also has a function to automatically generate learning progress reports for parents and distributes them to parents via electronic communication, thereby strengthening cooperation between home and school. Furthermore, it analyzes the learning trends of the entire class and provides teachers with suggestions for improving their lesson management, thereby enhancing the quality and effectiveness of education.

[0006] A "learning management database" is a database system used to manage and store students' learning progress and academic performance data.

[0007] "Learning progress data" refers to data that shows the progress of students in their learning process, and includes test results, homework submission status, attendance status, etc.

[0008] "Analysis" is the process of using collected data to analyze students' level of understanding and learning trends, and deriving results from that analysis.

[0009] "Comprehension level" is an indicator that shows how well a student understands a particular learning topic.

[0010] "Learning delay" refers to a state in which a student is not reaching the expected level of learning progress, and usually requires individual support and guidance.

[0011] A "teaching plan" is a plan for creating educational programs and learning activities that meet the learning needs of students.

[0012] A "display device" is a device that allows teachers to visually confirm information, and this includes computer monitors and tablet devices.

[0013] A "report for parents" is a document that outlines the student's learning progress and the teaching plan, and is distributed to parents.

[0014] "Electronic communication methods" refer to means of sending and receiving information in digital format via the internet, email, etc.

[0015] "Class-wide learning trends" refers to the analysis results that show patterns in the learning progress and level of understanding across the entire class. [Brief explanation of the drawing]

[0016] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11] This is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12]It is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] It is a sequence diagram showing the processing flow of the data processing system in Embodiment 2 when combined with an emotion engine. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when combined with an emotion engine.

Mode for Carrying Out the Invention

[0017] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0018] First, the terms used in the following description will be explained.

[0019] In the following embodiments, the numbered processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0020] In the following embodiments, the numbered RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0021] In the following embodiments, the signed storage is one or more non-volatile storage devices that store various programs and various parameters. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes.

[0022] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0023] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0024] [First Embodiment]

[0025] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0026] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0027] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0028] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0029] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0030] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0031] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0032] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0033] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0034] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0035] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0036] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0037] This invention provides a system for efficiently managing the learning progress of each student in an educational institution. The system consists of a server, user terminals, and necessary software components.

[0038] First, the server accesses the school's learning management database and periodically collects learning progress data such as students' test results, homework submission status, and attendance information. This ensures that the data is always up-to-date.

[0039] Teachers, as users, can access learning data through their devices and view individual students' levels of understanding and areas of difficulty in graph and chart format. This allows them to intuitively grasp the current learning situation of their students.

[0040] The server analyzes the collected data and, if a particular student is showing signs of learning delays, uses AI to automatically generate a personalized tutoring plan tailored to that student. This plan includes recommended learning materials and methods for managing learning progress, allowing users to provide specific guidance to the student based on it.

[0041] In addition, the server generates reports for parents that show the student's learning progress and distributes them via electronic means such as email. Through these reports, parents can understand their child's learning activities at school and provide necessary support at home.

[0042] Furthermore, the server analyzes the learning trends of the entire class and provides teachers with information to help improve and adjust their lesson content. This overall analysis allows teachers to develop strategies to improve the learning effectiveness of the entire class.

[0043] For example, if a student's understanding of a specific area of ​​physics is analyzed as insufficient, the server automatically proposes a lesson plan for that student, including specific video materials and practice problems, and notifies the teacher via their device. The teacher can then review this plan and apply it to the student, enabling effective individualized instruction.

[0044] The following describes the processing flow.

[0045] Step 1:

[0046] The server accesses the learning management database to automatically collect student test results, homework submission status, and attendance information.

[0047] Step 2:

[0048] Based on the learning progress data collected by the server, AI is used to analyze each student's learning situation. This analysis utilizes statistical methods and machine learning techniques to detect students' comprehension levels and learning delays.

[0049] Step 3:

[0050] The server uses the analysis results to identify each student's level of understanding and areas of weakness, and then processes this information into graphs and charts.

[0051] Step 4:

[0052] Through the terminal, teachers can access visualized learning progress information and check the learning status of individual students. This information serves as a reference for teachers when making educational decisions.

[0053] Step 5:

[0054] The server automatically generates individualized instruction plans based on the analysis results and proposes specific teaching materials and methods for students who are experiencing learning delays.

[0055] Step 6:

[0056] On their devices, users can review the generated individualized instruction plans and use them as a reference when creating instructional plans for students.

[0057] Step 7:

[0058] The server automatically generates learning progress reports for parents and sends them via email. This allows parents to understand their child's learning progress and provide necessary support at home.

[0059] Step 8:

[0060] The server collects and analyzes learning data for the entire class, providing the teacher (user) with information on the class's overall understanding and learning trends. Based on this information, the teacher can adjust the lesson content.

[0061] (Example 1)

[0062] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0063] Traditional educational management systems struggled to provide appropriate educational plans tailored to the individual progress of learners, and also failed to adequately inform parents and teachers. This resulted in the failure to detect learners' lack of understanding or falling behind early, leading to missed educational opportunities. Furthermore, there was a lack of systems to analyze the learning trends of the entire group and provide specific information for improving educational content.

[0064] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0065] In this invention, the server includes means for automatically collecting information from a data management structure, means for analyzing the collected information to detect learners' level of understanding and progress delays, and means for automatically generating learning plans for learners based on the analysis results. This enables the provision of appropriate learning plans tailored to the individual circumstances of learners and effective information sharing.

[0066] A "data management structure" is a system for systematically storing and managing information, and includes databases and file systems.

[0067] "Means of automatically collecting information" refers to the process of automatically acquiring necessary data from specific sources according to time and conditions.

[0068] "Means of analyzing collected information" refers to methods and techniques for analyzing obtained data and deriving useful insights and patterns.

[0069] "Means for detecting learners' level of understanding and progress delays" refers to analytical methods for determining how well learners understand the learning material and whether their learning is behind schedule.

[0070] "Methods for automatically generating educational plans" refer to the process of creating learning policies and materials optimized for each individual learner based on analyzed data.

[0071] "Means of presenting on a display device" refers to a method of showing generated information or plans to a user through a screen or display.

[0072] "A method for automatically generating learning progress reports for parents" refers to a technique that compiles information on learners' progress and achievements and creates it in a format that is easy for parents to understand.

[0073] "Methods for analyzing the learning trends of an entire group" refer to analytical techniques that clarify common learning trends and challenges among multiple learners and use this information to improve overall education.

[0074] "Methods using generative AI models" refer to technologies that use artificial intelligence to analyze data and generate specific outputs.

[0075] "Methods of displaying data in charts and graphs" refer to methods of displaying data in graph or chart format to make it easier to understand visually.

[0076] "The function of transmitting using electronic communication means" refers to the function of transmitting information using digital communication means such as the internet or email.

[0077] This invention is a system for accurately and efficiently tracking learners' progress in educational management and providing each individual with an optimal educational plan. The invention is implemented as follows.

[0078] The server connects to a data management structure to automatically collect information, periodically retrieving learners' test results, homework submission status, and attendance information. A common database management system (DBMS) is used to connect to the database, and programming languages ​​such as Python and Java (registered trademark) and their corresponding connection libraries are used.

[0079] The acquired information is analyzed on the server to evaluate the learner's level of understanding and any delays in progress. This analysis utilizes libraries such as Pandas and NumPy as data analysis tools. Based on the analysis, the server automates the process of generating customized learning plans for each learner using AI models. TENSORFLOW® and PyTorch are used for these AI models.

[0080] The generated lesson plans are displayed on the display device of the teacher's terminal. The terminal is assumed to be a computer running Windows or macOS®, or a smart device. The program receives data from the server via a REST API and visualizes it. Data visualization technologies such as Matplotlib and Seaborn are used as visualization tools.

[0081] Furthermore, the system automatically generates and sends reports on learning progress to parents via electronic communication. The SMTP protocol is used for email transmission, allowing users to check the information as needed.

[0082] To analyze the learning trends of the entire class, statistical analysis is performed using R or SPSS based on data from the group, and the information is provided to teachers in the form of suggestions for improving the teaching content.

[0083] For example, if a student has a lack of understanding in a specific area of ​​mathematics, the server proposes an educational plan for that student, including video materials and practice problems, and notifies the teacher via the terminal. In this process, a generative AI model participates in selecting the materials, promoting effective learning.

[0084] Example of a prompt:

[0085] "Describe the process of using AI to analyze individual learners' underachievement in mathematics and suggest effective learning materials and problems."

[0086] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0087] Step 1:

[0088] The server retrieves learner test results, homework submission status, and attendance information from the data management structure. SQL queries are executed to collect the necessary information from the database as input. The output is a dataset containing the most recent learning data for each learner. This data is formatted using Python libraries to prepare it for subsequent analysis.

[0089] Step 2:

[0090] The server analyzes the acquired dataset. The input is the organized data obtained in Step 1. Using the data analysis tool Pandas, calculations are performed to evaluate each learner's level of understanding and progress. The output is the performance indicators for each learner and their evaluation results. Based on these results, learning obstacles and areas of strength and weakness are clearly identified.

[0091] Step 3:

[0092] The server generates personalized learning plans based on the analysis results. The input is the evaluation results obtained in step 2. Using a generative AI model, a plan is developed that recommends appropriate teaching materials and learning methods. The output is the content of the learning plan optimized for each learner. This plan is automatically generated by a model using TensorFlow.

[0093] Step 4:

[0094] The terminal presents the generated lesson plan to the teacher. The input is the individualized lesson plan received from the server. Information is transferred to the terminal using a REST API and visualized on the display device. Using Matplotlib, graphics are generated that clearly represent the level of understanding and the lesson plan. The output is a well-organized display of the lesson plan.

[0095] Step 5:

[0096] The server automatically generates and sends emails to parents reporting on the student's learning progress. The input consists of the student's latest progress data and the generated report content. The learning status is sent to a pre-configured email address using the SMTP protocol. The output is an email sent in a format that parents can review.

[0097] Step 6:

[0098] The server analyzes the learning trends of the entire group and reports to the teacher. The input is a collection of progress data for all learners. Statistical analysis is performed using R or SPSS to create a report detailing the learning trends and challenges of the entire class. The output is a report that clarifies the approach the teacher should take for the entire class.

[0099] (Application Example 1)

[0100] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0101] In an environment where individualized educational support is needed, traditional systems struggle to track each student's learning progress in real time and provide timely, appropriate instruction. Furthermore, the inability to assess students' understanding in real time and flexibly adjust instruction plans based on that assessment leads to decreased learning effectiveness. Additionally, parents face challenges in accurately understanding their children's learning progress when providing support at home.

[0102] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0103] In this invention, the server includes means for automatically collecting learning progress data from a learning management database, means for analyzing the collected learning progress data and detecting students' comprehension levels and learning delays, means for automatically generating individualized instruction plans based on the analysis results, means for presenting the generated instruction plans to a support device, means for presenting educational content audibly and visually and collecting student responses, and means for grasping the learning situation in real time based on the collected student responses and adjusting the instruction plan. This enables real-time instruction optimized for each individual student and provides effective learning support.

[0104] A "learning management database" is a database used to systematically store and provide educational information such as students' learning progress, test results, and attendance records.

[0105] "Learning progress data" refers to data that shows the progress of each student's learning activities, including their level of understanding and homework submission status.

[0106] "Analysis" is the process of organizing information based on collected data and evaluating the learning situation and trends of a particular student.

[0107] A "teaching plan" is an educational plan optimized for each individual student based on the analysis results, and includes recommended teaching materials and study schedules.

[0108] A "support device" is a hardware or software system that presents educational content to students and assists them in their learning.

[0109] "Presenting educational content aurally and visually" refers to a method of communicating subject matter and instruction to students using audio and video.

[0110] "Student responses" refer to students' reactions and behaviors towards instruction, and serve as a criterion for measuring their level of understanding and interest.

[0111] "Gathering information in real time and adjusting the teaching plan" means evaluating students' learning progress on the spot and immediately changing educational policies and materials as needed.

[0112] To implement this invention, a server must first connect to a learning management database and automatically collect student learning progress data. This process involves using software that periodically retrieves test results and attendance information from the database. The collected data is then processed by an analysis module, and AI technology is used to identify students' comprehension levels and learning delays. Generative AI models such as TensorFlow and PyTorch are utilized in this analysis.

[0113] Next, based on the analysis results, a personalized instruction plan is automatically generated for each student. This plan includes recommended teaching materials and guidelines for learning progress. The generated plan is presented to the user through an educational support device, which then presents the educational content to the student via voice and display. This support device utilizes voice recognition technology and a display interface, allowing the robot to assist the student's learning.

[0114] Users operate these devices to provide students with personalized learning content. During learning, student responses are recorded by sensors and cameras, and the server analyzes this data in real time, adjusting the lesson plan as needed. In this way, the most suitable education can be provided to each individual student.

[0115] For example, if a student has difficulty with a particular algebra unit, the server can detect this and prepare video materials and review exercises related to that unit. The user then presents these to the student via an assistive device. If the student still doesn't understand, additional practice exercises or explanations from different perspectives are automatically suggested as the next step.

[0116] Examples of prompts for a generative AI model:

[0117] "Based on the student's learning progress data, please suggest the next learning materials and assignments they should tackle. The student is in the second year of junior high school and has particular difficulty with algebra in mathematics."

[0118] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0119] Step 1:

[0120] The server accesses the learning management database and periodically collects learning progress data such as students' test results, homework submission status, and attendance information. Current information from the database is used as input, and the collected data is sent to an analysis module on the server. This ensures that the most up-to-date learning progress is always maintained.

[0121] Step 2:

[0122] The server analyzes the collected learning progress data and uses a generative AI model to detect each student's level of understanding and any learning delays. The input includes the learning progress data obtained in step 1, and the output provides an evaluation of understanding and an indicator of learning delays. Specifically, the AI ​​model uses TensorFlow or PyTorch to process the data and generate the evaluation results.

[0123] Step 3:

[0124] The server automatically generates individualized instruction plans for each student based on the analysis results. The input includes the analysis results from step 2, and the output is an instruction plan that includes individually optimized learning materials and schedules. Throughout this process, the plan is presented in a user-friendly format.

[0125] Step 4:

[0126] The server transmits the generated lesson plan to the educational support device, which presents it to the user and students via display and audio. The input is the lesson plan generated in step 3, and the output is the presentation of teaching materials visually and audibly. In this step, the terminal dynamically displays the teaching materials and assists students in their learning.

[0127] Step 5:

[0128] The server utilizes feedback from support devices to collect student responses as sensor data and monitor learning progress in real time. Input is real-time student response data transmitted from the support devices, and output is an evaluation result indicating the student's level of understanding and engagement. The terminal then uses this information to modify the instructional plan as needed.

[0129] Step 6:

[0130] The server readjusts the lesson plan as needed based on the updated learning data and sends it back to the educational support device. The input is the latest comprehension assessment based on student feedback, and the output is the improved lesson plan delivered to the support device. This continuously optimizes the learning of individual students.

[0131] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0132] This invention provides a system for comprehensively managing students' learning progress and emotional state in educational institutions. The system comprises a server, terminals, and an emotion recognition engine.

[0133] First, the server accesses the school's learning management database to automatically collect student test results, homework submission status, and attendance information. The server also integrates an emotion engine that infers emotions from students' facial expressions, tone of voice, and text-based feedback.

[0134] Teachers, as users, can access both learning data and emotional data through their devices. This allows them to see how students feel about the lesson content and to quantitatively grasp their level of understanding and interest.

[0135] The server analyzes collected learning and emotional data to automatically generate personalized instruction plans for each student. These plans are customized to take emotional states into account and propose an effective educational approach.

[0136] Parent reports are automatically generated by the server and delivered via email. These reports include not only information on learning progress but also emotional feedback, making it easier for parents to understand their child's overall situation.

[0137] The server also analyzes the learning trends of the entire class, including sentiment data, and provides this information to the teachers, who are the users of the server. This information helps to adjust the content and methods of lessons and serves as a guide to improve the learning efficiency of the entire class.

[0138] For example, if a student expresses anxiety about mathematics, the server assesses this emotion, suggests teaching methods and materials to alleviate their anxiety, and notifies the teacher via their device. Based on this, the teacher can then follow up with the student.

[0139] The following describes the processing flow.

[0140] Step 1:

[0141] The server automatically and periodically collects student test results, homework submission status, and attendance information from the learning management database. Additionally, the server activates an emotion engine to detect and collect students' facial expressions, voices, and text comments during online activities.

[0142] Step 2:

[0143] On their devices, teachers can access a dashboard that displays real-time learning and emotional data of students, allowing them to check each student's level of understanding and emotional state. This enables teachers to intuitively grasp the students' learning progress.

[0144] Step 3:

[0145] The server analyzes the collected learning and sentiment data. This analysis process uses AI algorithms to detect correlations between students' emotions and learning progress, identifying areas of misunderstanding or learning delays.

[0146] Step 4:

[0147] The server automatically generates individualized instruction plans for each student based on the analysis results. These plans are customized to take into account the student's emotional state and include recommended materials, teaching methods, and suggestions for emotional support.

[0148] Step 5:

[0149] Users can review the generated individualized tutoring plans on their devices and apply the most effective teaching approach to each student. If necessary, they can fine-tune the proposed plan to optimize the student's learning experience.

[0150] Step 6:

[0151] The server automatically generates reports for parents regarding their child's learning progress and emotional state. These reports are delivered via email, helping parents gain a comprehensive understanding of their child's situation at school.

[0152] Step 7:

[0153] The server analyzes the learning trends and sentiments of the entire class and provides the results to the teacher. Based on this information, the user can improve lesson content and adjust how they interact with students, thereby improving the overall learning effectiveness of the class.

[0154] (Example 2)

[0155] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0156] In educational settings, it is not easy to grasp each student's learning progress and emotional state in real time and provide appropriate guidance based on that information. In particular, quantitatively measuring students' comprehension, interests, or stress levels, and developing individualized instruction plans based on that information, is a time-consuming and laborious task using conventional methods. Furthermore, when parents try to understand their child's learning situation, it is difficult to obtain information that takes emotional aspects into account.

[0157] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0158] In this invention, the server includes means for automatically collecting learning data from a learning management database, means for analyzing the collected learning data and emotional data to evaluate students' comprehension and emotional states, and means for automatically generating adaptive instruction plans for each student based on the analysis results. This makes it possible for educational settings to comprehensively grasp each student's learning progress and emotional tendencies, and to quickly formulate individualized instruction plans based on this. Furthermore, it helps parents to more accurately understand their child's overall condition by providing a comprehensive learning status report that includes emotional feedback.

[0159] "Learning data" is a general term for information related to educational activities, including students' test results, homework submission status, and attendance information.

[0160] "Emotional data" refers to information about a student's emotional state, inferred from their facial expressions, tone of voice, and text-based feedback.

[0161] An "adaptive learning plan" is an individualized educational strategy designed to take into account each student's learning progress and emotional state.

[0162] "Automatic generation" refers to the process of creating or generating an object using a pre-configured program or algorithm, minimizing human intervention.

[0163] A "server" is a computer system or part thereof that provides services to other devices or software via a network.

[0164] "Information presentation devices" are devices used by students and teachers to visually or audibly perceive data, including computer monitors and tablet devices.

[0165] A "report" is a document created to organize and present specific information, in this case including learning progress and emotional feedback.

[0166] This invention provides a system for comprehensively managing students' learning progress and emotional state in educational institutions. This system is implemented with a configuration including a server, terminals, and an emotion recognition engine.

[0167] The server automatically collects student learning data by accessing the learning management database within educational institutions. This learning data includes information such as test results, homework submission status, and attendance records. The server incorporates an emotion recognition engine, which allows it to extract emotional data from students' facial expressions, tone of voice, and text-based feedback. The server aggregates this data and analyzes it using a generative AI model.

[0168] This system allows teachers, as users, to access both learning data and emotional data via their devices. Using dedicated software, teachers can monitor students' understanding and emotions in real time and develop appropriate lesson plans. The teachers' devices are equipped with information display devices to show information delivered from the server.

[0169] This system allows the server to analyze individual student data and automatically generate adaptive learning plans. These plans are customized and present educational approaches tailored to the student's cognitive and emotional needs. The server also automatically generates and sends reports to parents via email, including learning progress and emotional feedback.

[0170] For example, if a student expresses anxiety about learning mathematics, the server detects this using an emotion recognition engine and suggests teaching methods and materials to the teacher to help the student feel more at ease. This suggestion is then sent to the teacher's terminal, and the teacher uses this information to follow up with the student.

[0171] An example of a prompt is, "Create a description of a system that proposes individualized instruction plans that take into account students' emotions and levels of understanding." Based on this prompt, the generative AI model can generate a detailed description of the system.

[0172] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0173] Step 1:

[0174] The server accesses the educational institution's learning management database to collect student learning data. Inputs include student test results, homework submission status, and attendance information. The server queries these database entries in a specified format, formats the data, and stores it. The output is the most recent learning data for each student. Specifically, the server periodically runs automated scripts to retrieve information from the database.

[0175] Step 2:

[0176] The server analyzes students' emotional data via an emotion recognition engine. Inputs include student facial images, voice recordings, and text feedback. The server feeds this data to an emotion recognition algorithm to estimate the student's emotional state. Outputs include evaluation values ​​and categorical data representing each student's emotional state. Specifically, it performs real-time image processing and voice analysis, and records the emotion estimation results in a database.

[0177] Step 3:

[0178] The server integrates collected training data and sentiment data and performs analysis using a generative AI model. Both training data and sentiment data are used as input. The server runs the AI ​​model to evaluate each student's level of understanding and emotional tendencies. The output is a proposal for an individualized instruction plan suitable for each student. Specifically, the AI ​​model utilizes machine learning techniques to perform analysis that derives relationships from diverse data.

[0179] Step 4:

[0180] Through the terminal, teachers receive instruction plans and use them to guide their students. Inputs include individual instruction plans and their background data, delivered from the server. The terminal has software installed to display this information, allowing teachers to adjust their instruction based on visual information. The output is the teacher's effective instructional practice. Specifically, teachers can print the plans or share them with students through the school's online platform.

[0181] Step 5:

[0182] The server automatically generates and emails reports to parents containing learning progress and emotional feedback. The inputs used are integrated learning and emotional data, and instructional plan information. The server creates a report based on the data and sends it to the designated parent's email address. The output is a detailed learning report received by the parent. Specifically, report generation software collects data and creates a report based on a template.

[0183] Step 6:

[0184] The server analyzes the learning and emotional tendencies of the entire class and provides this information to the user, the teacher. Input includes learning and emotional data for all students in the class. The server performs statistical analysis to reveal group trends. The output is a report on the overall learning and emotional state of the class, which the teacher can evaluate. Specifically, it generates graphs and charts to provide information visually.

[0185] (Application Example 2)

[0186] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0187] In educational institutions, traditionally, only academic performance was emphasized when assessing students' learning progress, making it difficult to develop educational plans that considered the emotional state of learners. Furthermore, there was a lack of concrete guidelines to help parents understand their children's learning and emotional states, and to enable educational institutions and instructors to quickly adjust learning methods.

[0188] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0189] In this invention, the server includes means for automatically collecting learning progress information from a learning management database, means for analyzing the collected learning progress information and student emotional information to generate individualized instruction plans, means for presenting the generated instruction plans to a decision support device, means for automatically generating reports on learning and emotional states for parents, means for analyzing group learning and emotional tendencies and proposing adjustments to teaching methods, and means for evaluating learners' facial expressions and vocal characteristics using an emotion recognition engine. This enables educational institutions to comprehensively manage students' learning progress and emotional states, and to formulate effective educational plans and respond immediately.

[0190] A "learning management database" is an information management system for electronically storing and managing students' learning progress information and evaluation data.

[0191] "Learning progress information" refers to data that quantitatively shows each student's progress in learning activities, such as their academic performance, homework submission status, and attendance record.

[0192] "Emotional information" refers to information that indicates a student's feelings or emotional state, inferred from their facial expressions, voice, text data, etc.

[0193] A "teaching plan" is a plan that outlines individually customized educational guidelines and material selections based on students' learning progress and emotional state.

[0194] A "decision support device" is a tool or display device used by educators to make educational decisions based on students' learning data and emotional data.

[0195] A "report" is a document or electronic record that outlines a student's learning progress and emotional state, and is used to share this information with parents.

[0196] "Group-based learning and emotional tendencies" refer to statistical trends in students' learning progress and emotional patterns within a class or school as a whole.

[0197] An "emotion recognition engine" is a computational system or algorithm that infers a learner's emotional state through facial expression analysis, speech analysis, and text processing.

[0198] The system that implements this application consists of a server, a user terminal, and an emotion recognition engine. The server is designed using a Python framework (e.g., Django), and its database uses PostgreSQL. The server automatically collects learning progress information in real time from the learning management database and analyzes the collected information along with the students' emotion information.

[0199] The emotion recognition engine uses deep learning frameworks such as TensorFlow and PyTorch to process facial expression and audio data. This engine acquires data using the camera and microphone of a smartphone connected through the user's device. The acquired data is immediately sent to the server.

[0200] The server analyzes the received data and generates individualized instruction plans based on learning progress and sentiment data. The generated instruction plans are displayed on the user's terminal, assisting educators in decision-making.

[0201] Furthermore, reports based on this data will be regularly distributed to parents via email. These reports will include quantitative information on students' learning progress and emotional state, increasing transparency between educational institutions and parents.

[0202] For example, if a student shows interest or anxiety during a history lesson, the server analyzes this emotional data and generates a lesson plan to provide appropriate materials and support. This information is then notified to the educator on their device.

[0203] An example of a prompt using a generative AI model is: "Analyze student A's facial expression data and text comments, and propose teaching materials to keep them interested in history."

[0204] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0205] Step 1:

[0206] The server connects to a learning management database and automatically collects student learning progress information. The input is academic data retrieved from the learning management database, and the output is the collected learning progress information. It uses database queries to extract the necessary information and stores it on the server.

[0207] Step 2:

[0208] The device uses the smartphone's camera and microphone to collect students' facial expressions and voices. The input is the student's real-time emotional expression captured via the smartphone, and the output is processed emotional data. This data is sent to a server, enabling the emotion recognition engine to perform subsequent processing.

[0209] Step 3:

[0210] The server integrates and analyzes the received learning progress information and sentiment data. The inputs are the learning progress information collected in Step 1 and the sentiment data obtained in Step 2. The output is the integrated analysis result. Statistical methods and machine learning models are used to comprehensively evaluate learning progress and emotional state.

[0211] Step 4:

[0212] The server generates individualized instruction plans based on the analysis results. The input is the integrated analysis results, and the output is a customized instruction plan. Using a generative AI model, it generates prompts that suggest the most suitable learning support for each student.

[0213] Step 5:

[0214] The user, the educator, receives the lesson plan generated through the terminal and uses it for decision-making. The input is the lesson plan presented on the terminal, and the output is the educational measures selected by the educator. Based on the presented information, the educator adjusts the actual educational activities.

[0215] Step 6:

[0216] The server automatically generates and sends reports to parents via email. The input is integrated data on learning and emotional states, and the output is a report to parents. It periodically sends information using the email system.

[0217] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0218] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0219] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0220] [Second Embodiment]

[0221] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0222] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0223] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0224] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0225] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0226] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0227] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0228] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0229] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0230] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0231] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0232] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0233] This invention provides a system for efficiently managing the learning progress of each student in an educational institution. The system consists of a server, user terminals, and necessary software components.

[0234] First, the server accesses the school's learning management database and periodically collects learning progress data such as students' test results, homework submission status, and attendance information. This ensures that the data is always up-to-date.

[0235] Teachers, as users, can access learning data through their devices and view individual students' levels of understanding and areas of difficulty in graph and chart format. This allows them to intuitively grasp the current learning situation of their students.

[0236] The server analyzes the collected data and, if a particular student is showing signs of learning delays, uses AI to automatically generate a personalized tutoring plan tailored to that student. This plan includes recommended learning materials and methods for managing learning progress, allowing users to provide specific guidance to the student based on it.

[0237] In addition, the server generates reports for parents that show the student's learning progress and distributes them via electronic means such as email. Through these reports, parents can understand their child's learning activities at school and provide necessary support at home.

[0238] Furthermore, the server analyzes the learning trends of the entire class and provides teachers with information to help improve and adjust their lesson content. This overall analysis allows teachers to develop strategies to improve the learning effectiveness of the entire class.

[0239] For example, if a student's understanding of a specific area of ​​physics is analyzed as insufficient, the server automatically proposes a lesson plan for that student, including specific video materials and practice problems, and notifies the teacher via their device. The teacher can then review this plan and apply it to the student, enabling effective individualized instruction.

[0240] The following describes the processing flow.

[0241] Step 1:

[0242] The server accesses the learning management database to automatically collect student test results, homework submission status, and attendance information.

[0243] Step 2:

[0244] Based on the learning progress data collected by the server, AI is used to analyze each student's learning situation. This analysis utilizes statistical methods and machine learning techniques to detect students' comprehension levels and learning delays.

[0245] Step 3:

[0246] The server uses the analysis results to identify each student's level of understanding and areas of weakness, and then processes this information into graphs and charts.

[0247] Step 4:

[0248] Through the terminal, teachers can access visualized learning progress information and check the learning status of individual students. This information serves as a reference for teachers when making educational decisions.

[0249] Step 5:

[0250] The server automatically generates individualized instruction plans based on the analysis results and proposes specific teaching materials and methods for students who are experiencing learning delays.

[0251] Step 6:

[0252] On their devices, users can review the generated individualized instruction plans and use them as a reference when creating instructional plans for students.

[0253] Step 7:

[0254] The server automatically generates learning progress reports for parents and sends them via email. This allows parents to understand their child's learning progress and provide necessary support at home.

[0255] Step 8:

[0256] The server collects and analyzes learning data for the entire class, providing the teacher (user) with information on the class's overall understanding and learning trends. Based on this information, the teacher can adjust the lesson content.

[0257] (Example 1)

[0258] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0259] Traditional educational management systems struggled to provide appropriate educational plans tailored to the individual progress of learners, and also failed to adequately inform parents and teachers. This resulted in the failure to detect learners' lack of understanding or falling behind early, leading to missed educational opportunities. Furthermore, there was a lack of systems to analyze the learning trends of the entire group and provide specific information for improving educational content.

[0260] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0261] In this invention, the server includes means for automatically collecting information from a data management structure, means for analyzing the collected information to detect learners' level of understanding and progress delays, and means for automatically generating learning plans for learners based on the analysis results. This enables the provision of appropriate learning plans tailored to the individual circumstances of learners and effective information sharing.

[0262] A "data management structure" is a system for systematically storing and managing information, and includes databases and file systems.

[0263] "Means of automatically collecting information" refers to the process of automatically acquiring necessary data from specific sources according to time and conditions.

[0264] "Means of analyzing collected information" refers to methods and techniques for analyzing obtained data and deriving useful insights and patterns.

[0265] "Means for detecting learners' level of understanding and progress delays" refers to analytical methods for determining how well learners understand the learning material and whether their learning is behind schedule.

[0266] "Methods for automatically generating educational plans" refer to the process of creating learning policies and materials optimized for each individual learner based on analyzed data.

[0267] "Means of presenting on a display device" refers to a method of showing generated information or plans to a user through a screen or display.

[0268] "A method for automatically generating learning progress reports for parents" refers to a technique that compiles information on learners' progress and achievements and creates it in a format that is easy for parents to understand.

[0269] "Methods for analyzing the learning trends of an entire group" refer to analytical techniques that clarify common learning trends and challenges among multiple learners and use this information to improve overall education.

[0270] "Methods using generative AI models" refer to technologies that use artificial intelligence to analyze data and generate specific outputs.

[0271] "Methods of displaying data in charts and graphs" refer to methods of displaying data in graph or chart format to make it easier to understand visually.

[0272] "The function of transmitting using electronic communication means" refers to the function of transmitting information using digital communication means such as the internet or email.

[0273] This invention is a system for accurately and efficiently tracking learners' progress in educational management and providing each individual with an optimal educational plan. The invention is implemented as follows.

[0274] The server connects to a data management structure to automatically collect information, periodically retrieving learners' test results, homework submission status, and attendance information. A common database management system (DBMS) is used to connect to the database, and programming languages ​​such as Python or Java, along with their corresponding connection libraries, are employed.

[0275] The acquired information is analyzed on the server to evaluate the learner's level of understanding and any delays in progress. This analysis utilizes libraries such as Pandas and NumPy as data analysis tools. Based on the analysis, the server automates the process of generating customized learning plans for each learner using AI models. TensorFlow and PyTorch are used for these AI models.

[0276] The generated lesson plans are displayed on the display devices used by teachers. These devices are assumed to be computers running Windows or macOS, or smart devices. The program receives data from the server via a REST API and visualizes it. Data visualization technologies such as Matplotlib and Seaborn are used as visualization tools.

[0277] Furthermore, the system automatically generates and sends reports on learning progress to parents via electronic communication. The SMTP protocol is used for email transmission, allowing users to check the information as needed.

[0278] To analyze the learning trends of the entire class, statistical analysis is performed using R or SPSS based on data from the group, and the information is provided to teachers in the form of suggestions for improving the teaching content.

[0279] For example, if a student has a lack of understanding in a specific area of ​​mathematics, the server proposes an educational plan for that student, including video materials and practice problems, and notifies the teacher via the terminal. In this process, a generative AI model participates in selecting the materials, promoting effective learning.

[0280] Example of a prompt sentence:

[0281] "Explain the process of using AI to analyze the progress deficiencies of individual learners in the field of mathematics and propose effective teaching materials and problems."

[0282] The flow of the specific process in Example 1 will be described using Figure 11.

[0283] Step 1:

[0284] The server retrieves the test results, homework submission status, and attendance information of the learners from the data management structure. As input, an SQL query is executed to collect the necessary information from the database. The output is a dataset containing the latest learning data for each learner. This data is formatted using a Python library in preparation for subsequent analysis.

[0285] Step 2:

[0286] The server analyzes the retrieved dataset. The input is the formatted data obtained in Step 1. Using Pandas, a data analysis tool, calculations are performed to evaluate the understanding and progress delays of each learner. The output is the performance metrics of individual learners and their evaluation results. Based on these results, learning obstacles and areas of strength and weakness are clearly identified.

[0287] Step 3:

[0288] The server generates an individualized education plan based on the analysis results. The input is the evaluation results obtained in Step 2. Using a generative AI model, a plan is developed to recommend appropriate teaching materials and learning methods. The output is the content of the education plan optimized for each learner. This plan is automatically generated by a model using TensorFlow.

[0289] Step 4:

[0290] The terminal presents the generated lesson plan to the teacher. The input is the individualized lesson plan received from the server. Information is transferred to the terminal using a REST API and visualized on the display device. Using Matplotlib, graphics are generated that clearly represent the level of understanding and the lesson plan. The output is a well-organized display of the lesson plan.

[0291] Step 5:

[0292] The server automatically generates and sends emails to parents reporting on the student's learning progress. The input consists of the student's latest progress data and the generated report content. The learning status is sent to a pre-configured email address using the SMTP protocol. The output is an email sent in a format that parents can review.

[0293] Step 6:

[0294] The server analyzes the learning trends of the entire group and reports to the teacher. The input is a collection of progress data for all learners. Statistical analysis is performed using R or SPSS to create a report detailing the learning trends and challenges of the entire class. The output is a report that clarifies the approach the teacher should take for the entire class.

[0295] (Application Example 1)

[0296] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0297] In an environment where individualized educational support is needed, traditional systems struggle to track each student's learning progress in real time and provide timely, appropriate instruction. Furthermore, the inability to assess students' understanding in real time and flexibly adjust instruction plans based on that assessment leads to decreased learning effectiveness. Additionally, parents face challenges in accurately understanding their children's learning progress when providing support at home.

[0298] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0299] In this invention, the server includes means for automatically collecting learning progress data from a learning management database, means for analyzing the collected learning progress data and detecting students' comprehension levels and learning delays, means for automatically generating individualized instruction plans based on the analysis results, means for presenting the generated instruction plans to a support device, means for presenting educational content audibly and visually and collecting student responses, and means for grasping the learning situation in real time based on the collected student responses and adjusting the instruction plan. This enables real-time instruction optimized for each individual student and provides effective learning support.

[0300] A "learning management database" is a database used to systematically store and provide educational information such as students' learning progress, test results, and attendance records.

[0301] "Learning progress data" refers to data that shows the progress of each student's learning activities, including their level of understanding and homework submission status.

[0302] "Analysis" is the process of organizing information based on collected data and evaluating the learning situation and trends of a particular student.

[0303] A "teaching plan" is an educational plan optimized for each individual student based on the analysis results, and includes recommended teaching materials and study schedules.

[0304] A "support device" is a hardware or software system that presents educational content to students and assists them in their learning.

[0305] "Presenting educational content aurally and visually" refers to a method of communicating subject matter and instruction to students using audio and video.

[0306] "Student reaction" refers to the responses and actions of students towards guidance, serving as a criterion for measuring understanding and interest.

[0307] "Grasp in real-time and adjust the guidance plan" means evaluating the learning status of students on-site and immediately changing the educational policy and teaching materials as needed.

[0308] To implement this invention, first, the server needs to connect to the learning management database and automatically collect the learning progress data of students. In this process, software that regularly obtains test results and attendance information from the database is used. The collected data is then processed by an analysis module, and AI technology is used to identify the understanding level and learning lag of students. In this analysis, generative AI models such as TensorFlow and PyTorch are utilized.

[0309] Next, based on the analysis results, a guidance plan for each student is automatically generated. This guidance plan includes recommended teaching materials and guidelines for learning progress. The generated plan is presented to the user through an educational support device, and educational content is presented to students via voice and display. In this support device, voice recognition technology and display interfaces are utilized, and robots assist students' learning.

[0310] The user operates these devices to provide educational content optimized for students. During learning, the reactions of students are recorded by sensors and cameras, and the server analyzes this data in real-time and adjusts the guidance plan as needed. By such a method, optimal education can be provided for individual students.

[0311] As a specific example, if a certain student has a difficulty awareness in a specific unit of algebra, the server can detect this and prepare video teaching materials and review questions related to that unit. The user presents this to the student via the support device. If the student still cannot understand, additional practice questions and explanations from different perspectives are automatically proposed as the next step.

[0312] Examples of prompts for a generative AI model:

[0313] "Based on the student's learning progress data, please suggest the next learning materials and assignments they should tackle. The student is in the second year of junior high school and has particular difficulty with algebra in mathematics."

[0314] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0315] Step 1:

[0316] The server accesses the learning management database and periodically collects learning progress data such as students' test results, homework submission status, and attendance information. Current information from the database is used as input, and the collected data is sent to an analysis module on the server. This ensures that the most up-to-date learning progress is always maintained.

[0317] Step 2:

[0318] The server analyzes the collected learning progress data and uses a generative AI model to detect each student's level of understanding and any learning delays. The input includes the learning progress data obtained in step 1, and the output provides an evaluation of understanding and an indicator of learning delays. Specifically, the AI ​​model uses TensorFlow or PyTorch to process the data and generate the evaluation results.

[0319] Step 3:

[0320] The server automatically generates individualized instruction plans for each student based on the analysis results. The input includes the analysis results from step 2, and the output is an instruction plan that includes individually optimized learning materials and schedules. Throughout this process, the plan is presented in a user-friendly format.

[0321] Step 4:

[0322] The server transmits the generated lesson plan to the educational support device, which presents it to the user and students via display and audio. The input is the lesson plan generated in step 3, and the output is the presentation of teaching materials visually and audibly. In this step, the terminal dynamically displays the teaching materials and assists students in their learning.

[0323] Step 5:

[0324] The server utilizes feedback from support devices to collect student responses as sensor data and monitor learning progress in real time. Input is real-time student response data transmitted from the support devices, and output is an evaluation result indicating the student's level of understanding and engagement. The terminal then uses this information to modify the instructional plan as needed.

[0325] Step 6:

[0326] The server readjusts the lesson plan as needed based on the updated learning data and sends it back to the educational support device. The input is the latest comprehension assessment based on student feedback, and the output is the improved lesson plan delivered to the support device. This continuously optimizes the learning of individual students.

[0327] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0328] This invention provides a system for comprehensively managing students' learning progress and emotional state in educational institutions. The system comprises a server, terminals, and an emotion recognition engine.

[0329] First, the server accesses the school's learning management database to automatically collect student test results, homework submission status, and attendance information. The server also integrates an emotion engine that infers emotions from students' facial expressions, tone of voice, and text-based feedback.

[0330] Teachers, as users, can access both learning data and emotional data through their devices. This allows them to see how students feel about the lesson content and to quantitatively grasp their level of understanding and interest.

[0331] The server analyzes collected learning and emotional data to automatically generate personalized instruction plans for each student. These plans are customized to take emotional states into account and propose an effective educational approach.

[0332] Parent reports are automatically generated by the server and delivered via email. These reports include not only information on learning progress but also emotional feedback, making it easier for parents to understand their child's overall situation.

[0333] The server also analyzes the learning trends of the entire class, including sentiment data, and provides this information to the teachers, who are the users of the server. This information helps to adjust the content and methods of lessons and serves as a guide to improve the learning efficiency of the entire class.

[0334] For example, if a student expresses anxiety about mathematics, the server assesses this emotion, suggests teaching methods and materials to alleviate their anxiety, and notifies the teacher via their device. Based on this, the teacher can then follow up with the student.

[0335] The following describes the processing flow.

[0336] Step 1:

[0337] The server automatically and periodically collects student test results, homework submission status, and attendance information from the learning management database. Additionally, the server activates an emotion engine to detect and collect students' facial expressions, voices, and text comments during online activities.

[0338] Step 2:

[0339] On their devices, teachers can access a dashboard that displays real-time learning and emotional data of students, allowing them to check each student's level of understanding and emotional state. This enables teachers to intuitively grasp the students' learning progress.

[0340] Step 3:

[0341] The server analyzes the collected learning and sentiment data. This analysis process uses AI algorithms to detect correlations between students' emotions and learning progress, identifying areas of misunderstanding or learning delays.

[0342] Step 4:

[0343] The server automatically generates individualized instruction plans for each student based on the analysis results. These plans are customized to take into account the student's emotional state and include recommended materials, teaching methods, and suggestions for emotional support.

[0344] Step 5:

[0345] Users can review the generated individualized tutoring plans on their devices and apply the most effective teaching approach to each student. If necessary, they can fine-tune the proposed plan to optimize the student's learning experience.

[0346] Step 6:

[0347] The server automatically generates reports for parents regarding their child's learning progress and emotional state. These reports are delivered via email, helping parents gain a comprehensive understanding of their child's situation at school.

[0348] Step 7:

[0349] The server analyzes the learning trends and sentiments of the entire class and provides the results to the teacher. Based on this information, the user can improve lesson content and adjust how they interact with students, thereby improving the overall learning effectiveness of the class.

[0350] (Example 2)

[0351] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0352] In educational settings, it is not easy to grasp each student's learning progress and emotional state in real time and provide appropriate guidance based on that information. In particular, quantitatively measuring students' comprehension, interests, or stress levels, and developing individualized instruction plans based on that information, is a time-consuming and laborious task using conventional methods. Furthermore, when parents try to understand their child's learning situation, it is difficult to obtain information that takes emotional aspects into account.

[0353] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0354] In this invention, the server includes means for automatically collecting learning data from a learning management database, means for analyzing the collected learning data and emotional data to evaluate students' comprehension and emotional states, and means for automatically generating adaptive instruction plans for each student based on the analysis results. This makes it possible for educational settings to comprehensively grasp each student's learning progress and emotional tendencies, and to quickly formulate individualized instruction plans based on this. Furthermore, it helps parents to more accurately understand their child's overall condition by providing a comprehensive learning status report that includes emotional feedback.

[0355] "Learning data" is a general term for information related to educational activities, including students' test results, homework submission status, and attendance information.

[0356] "Emotional data" refers to information about a student's emotional state, inferred from their facial expressions, tone of voice, and text-based feedback.

[0357] An "adaptive learning plan" is an individualized educational strategy designed to take into account each student's learning progress and emotional state.

[0358] "Automatic generation" refers to the process of creating or generating an object using a pre-configured program or algorithm, minimizing human intervention.

[0359] A "server" is a computer system or part thereof that provides services to other devices or software via a network.

[0360] "Information presentation devices" are devices used by students and teachers to visually or audibly perceive data, including computer monitors and tablet devices.

[0361] A "report" is a document created to organize and present specific information, in this case including learning progress and emotional feedback.

[0362] This invention provides a system for comprehensively managing students' learning progress and emotional state in educational institutions. This system is implemented with a configuration including a server, terminals, and an emotion recognition engine.

[0363] The server automatically collects student learning data by accessing the learning management database within educational institutions. This learning data includes information such as test results, homework submission status, and attendance records. The server incorporates an emotion recognition engine, which allows it to extract emotional data from students' facial expressions, tone of voice, and text-based feedback. The server aggregates this data and analyzes it using a generative AI model.

[0364] This system allows teachers, as users, to access both learning data and emotional data via their devices. Using dedicated software, teachers can monitor students' understanding and emotions in real time and develop appropriate lesson plans. The teachers' devices are equipped with information display devices to show information delivered from the server.

[0365] This system allows the server to analyze individual student data and automatically generate adaptive learning plans. These plans are customized and present educational approaches tailored to the student's cognitive and emotional needs. The server also automatically generates and sends reports to parents via email, including learning progress and emotional feedback.

[0366] For example, if a student expresses anxiety about learning mathematics, the server detects this using an emotion recognition engine and suggests teaching methods and materials to the teacher to help the student feel more at ease. This suggestion is then sent to the teacher's terminal, and the teacher uses this information to follow up with the student.

[0367] An example of a prompt is, "Create a description of a system that proposes individualized instruction plans that take into account students' emotions and levels of understanding." Based on this prompt, the generative AI model can generate a detailed description of the system.

[0368] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0369] Step 1:

[0370] The server accesses the educational institution's learning management database to collect student learning data. Inputs include student test results, homework submission status, and attendance information. The server queries these database entries in a specified format, formats the data, and stores it. The output is the most recent learning data for each student. Specifically, the server periodically runs automated scripts to retrieve information from the database.

[0371] Step 2:

[0372] The server analyzes students' emotional data via an emotion recognition engine. Inputs include student facial images, voice recordings, and text feedback. The server feeds this data to an emotion recognition algorithm to estimate the student's emotional state. Outputs include evaluation values ​​and categorical data representing each student's emotional state. Specifically, it performs real-time image processing and voice analysis, and records the emotion estimation results in a database.

[0373] Step 3:

[0374] The server integrates collected training data and sentiment data and performs analysis using a generative AI model. Both training data and sentiment data are used as input. The server runs the AI ​​model to evaluate each student's level of understanding and emotional tendencies. The output is a proposal for an individualized instruction plan suitable for each student. Specifically, the AI ​​model utilizes machine learning techniques to perform analysis that derives relationships from diverse data.

[0375] Step 4:

[0376] Through the terminal, teachers receive instruction plans and use them to guide their students. Inputs include individual instruction plans and their background data, delivered from the server. The terminal has software installed to display this information, allowing teachers to adjust their instruction based on visual information. The output is the teacher's effective instructional practice. Specifically, teachers can print the plans or share them with students through the school's online platform.

[0377] Step 5:

[0378] The server automatically generates and emails reports to parents containing learning progress and emotional feedback. The inputs used are integrated learning and emotional data, and instructional plan information. The server creates a report based on the data and sends it to the designated parent's email address. The output is a detailed learning report received by the parent. Specifically, report generation software collects data and creates a report based on a template.

[0379] Step 6:

[0380] The server analyzes the learning and emotional tendencies of the entire class and provides this information to the user, the teacher. Input includes learning and emotional data for all students in the class. The server performs statistical analysis to reveal group trends. The output is a report on the overall learning and emotional state of the class, which the teacher can evaluate. Specifically, it generates graphs and charts to provide information visually.

[0381] (Application Example 2)

[0382] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0383] In educational institutions, traditionally, only academic performance was emphasized when assessing students' learning progress, making it difficult to develop educational plans that considered the emotional state of learners. Furthermore, there was a lack of concrete guidelines to help parents understand their children's learning and emotional states, and to enable educational institutions and instructors to quickly adjust learning methods.

[0384] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0385] In this invention, the server includes means for automatically collecting learning progress information from a learning management database, means for analyzing the collected learning progress information and student emotional information to generate individualized instruction plans, means for presenting the generated instruction plans to a decision support device, means for automatically generating reports on learning and emotional states for parents, means for analyzing group learning and emotional tendencies and proposing adjustments to teaching methods, and means for evaluating learners' facial expressions and vocal characteristics using an emotion recognition engine. This enables educational institutions to comprehensively manage students' learning progress and emotional states, and to formulate effective educational plans and respond immediately.

[0386] A "learning management database" is an information management system for electronically storing and managing students' learning progress information and evaluation data.

[0387] "Learning progress information" refers to data that quantitatively shows each student's progress in learning activities, such as their academic performance, homework submission status, and attendance record.

[0388] "Emotional information" refers to information that indicates a student's feelings or emotional state, inferred from their facial expressions, voice, text data, etc.

[0389] A "teaching plan" is a plan that outlines individually customized educational guidelines and material selections based on students' learning progress and emotional state.

[0390] A "decision support device" is a tool or display device used by educators to make educational decisions based on students' learning data and emotional data.

[0391] A "report" is a document or electronic record that outlines a student's learning progress and emotional state, and is used to share this information with parents.

[0392] "Group-based learning and emotional tendencies" refer to statistical trends in students' learning progress and emotional patterns within a class or school as a whole.

[0393] An "emotion recognition engine" is a computational system or algorithm that infers a learner's emotional state through facial expression analysis, speech analysis, and text processing.

[0394] The system that implements this application consists of a server, a user terminal, and an emotion recognition engine. The server is designed using a Python framework (e.g., Django), and its database uses PostgreSQL. The server automatically collects learning progress information in real time from the learning management database and analyzes the collected information along with the students' emotion information.

[0395] The emotion recognition engine uses deep learning frameworks such as TensorFlow and PyTorch to process facial expression and audio data. This engine acquires data using the camera and microphone of a smartphone connected through the user's device. The acquired data is immediately sent to the server.

[0396] The server analyzes the received data and generates individualized instruction plans based on learning progress and sentiment data. The generated instruction plans are displayed on the user's terminal, assisting educators in decision-making.

[0397] Furthermore, reports based on this data will be regularly distributed to parents via email. These reports will include quantitative information on students' learning progress and emotional state, increasing transparency between educational institutions and parents.

[0398] For example, if a student shows interest or anxiety during a history lesson, the server analyzes this emotional data and generates a lesson plan to provide appropriate materials and support. This information is then notified to the educator on their device.

[0399] An example of a prompt using a generative AI model is: "Analyze student A's facial expression data and text comments, and propose teaching materials to keep them interested in history."

[0400] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0401] Step 1:

[0402] The server connects to a learning management database and automatically collects student learning progress information. The input is academic data retrieved from the learning management database, and the output is the collected learning progress information. It uses database queries to extract the necessary information and stores it on the server.

[0403] Step 2:

[0404] The device uses the smartphone's camera and microphone to collect students' facial expressions and voices. The input is the student's real-time emotional expression captured via the smartphone, and the output is processed emotional data. This data is sent to a server, enabling the emotion recognition engine to perform subsequent processing.

[0405] Step 3:

[0406] The server integrates and analyzes the received learning progress information and sentiment data. The inputs are the learning progress information collected in Step 1 and the sentiment data obtained in Step 2. The output is the integrated analysis result. Statistical methods and machine learning models are used to comprehensively evaluate learning progress and emotional state.

[0407] Step 4:

[0408] The server generates individualized instruction plans based on the analysis results. The input is the integrated analysis results, and the output is a customized instruction plan. Using a generative AI model, it generates prompts that suggest the most suitable learning support for each student.

[0409] Step 5:

[0410] The user, the educator, receives the lesson plan generated through the terminal and uses it for decision-making. The input is the lesson plan presented on the terminal, and the output is the educational measures selected by the educator. Based on the presented information, the educator adjusts the actual educational activities.

[0411] Step 6:

[0412] The server automatically generates and sends reports to parents via email. The input is integrated data on learning and emotional states, and the output is a report to parents. It periodically sends information using the email system.

[0413] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0414] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0415] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0416] [Third Embodiment]

[0417] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0418] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0419] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0420] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0421] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0422] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0423] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0424] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0425] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0426] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0427] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0428] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0429] This invention provides a system for efficiently managing the learning progress of each student in an educational institution. The system consists of a server, user terminals, and necessary software components.

[0430] First, the server accesses the school's learning management database and periodically collects learning progress data such as students' test results, homework submission status, and attendance information. This ensures that the data is always up-to-date.

[0431] Teachers, as users, can access learning data through their devices and view individual students' levels of understanding and areas of difficulty in graph and chart format. This allows them to intuitively grasp the current learning situation of their students.

[0432] The server analyzes the collected data and, if a particular student is showing signs of learning delays, uses AI to automatically generate a personalized tutoring plan tailored to that student. This plan includes recommended learning materials and methods for managing learning progress, allowing users to provide specific guidance to the student based on it.

[0433] In addition, the server generates reports for parents that show the student's learning progress and distributes them via electronic means such as email. Through these reports, parents can understand their child's learning activities at school and provide necessary support at home.

[0434] Furthermore, the server analyzes the learning trends of the entire class and provides teachers with information to help improve and adjust their lesson content. This overall analysis allows teachers to develop strategies to improve the learning effectiveness of the entire class.

[0435] For example, if a student's understanding of a specific area of ​​physics is analyzed as insufficient, the server automatically proposes a lesson plan for that student, including specific video materials and practice problems, and notifies the teacher via their device. The teacher can then review this plan and apply it to the student, enabling effective individualized instruction.

[0436] The following describes the processing flow.

[0437] Step 1:

[0438] The server accesses the learning management database to automatically collect student test results, homework submission status, and attendance information.

[0439] Step 2:

[0440] Based on the learning progress data collected by the server, AI is used to analyze each student's learning situation. This analysis utilizes statistical methods and machine learning techniques to detect students' comprehension levels and learning delays.

[0441] Step 3:

[0442] The server uses the analysis results to identify each student's level of understanding and areas of weakness, and then processes this information into graphs and charts.

[0443] Step 4:

[0444] Through the terminal, teachers can access visualized learning progress information and check the learning status of individual students. This information serves as a reference for teachers when making educational decisions.

[0445] Step 5:

[0446] The server automatically generates individualized instruction plans based on the analysis results and proposes specific teaching materials and methods for students who are experiencing learning delays.

[0447] Step 6:

[0448] On their devices, users can review the generated individualized instruction plans and use them as a reference when creating instructional plans for students.

[0449] Step 7:

[0450] The server automatically generates learning progress reports for parents and sends them via email. This allows parents to understand their child's learning progress and provide necessary support at home.

[0451] Step 8:

[0452] The server collects and analyzes learning data for the entire class, providing the teacher (user) with information on the class's overall understanding and learning trends. Based on this information, the teacher can adjust the lesson content.

[0453] (Example 1)

[0454] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0455] Traditional educational management systems struggled to provide appropriate educational plans tailored to the individual progress of learners, and also failed to adequately inform parents and teachers. This resulted in the failure to detect learners' lack of understanding or falling behind early, leading to missed educational opportunities. Furthermore, there was a lack of systems to analyze the learning trends of the entire group and provide specific information for improving educational content.

[0456] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0457] In this invention, the server includes means for automatically collecting information from a data management structure, means for analyzing the collected information to detect learners' level of understanding and progress delays, and means for automatically generating learning plans for learners based on the analysis results. This enables the provision of appropriate learning plans tailored to the individual circumstances of learners and effective information sharing.

[0458] A "data management structure" is a system for systematically storing and managing information, and includes databases and file systems.

[0459] "Means of automatically collecting information" refers to the process of automatically acquiring necessary data from specific sources according to time and conditions.

[0460] "Means of analyzing collected information" refers to methods and techniques for analyzing obtained data and deriving useful insights and patterns.

[0461] "Means for detecting learners' level of understanding and progress delays" refers to analytical methods for determining how well learners understand the learning material and whether their learning is behind schedule.

[0462] "Methods for automatically generating educational plans" refer to the process of creating learning policies and materials optimized for each individual learner based on analyzed data.

[0463] "Means of presenting on a display device" refers to a method of showing generated information or plans to a user through a screen or display.

[0464] "A method for automatically generating learning progress reports for parents" refers to a technique that compiles information on learners' progress and achievements and creates it in a format that is easy for parents to understand.

[0465] "Methods for analyzing the learning trends of an entire group" refer to analytical techniques that clarify common learning trends and challenges among multiple learners and use this information to improve overall education.

[0466] "Methods using generative AI models" refer to technologies that use artificial intelligence to analyze data and generate specific outputs.

[0467] "Methods of displaying data in charts and graphs" refer to methods of displaying data in graph or chart format to make it easier to understand visually.

[0468] "The function of transmitting using electronic communication means" refers to the function of transmitting information using digital communication means such as the internet or email.

[0469] This invention is a system for accurately and efficiently tracking learners' progress in educational management and providing each individual with an optimal educational plan. The invention is implemented as follows.

[0470] The server connects to a data management structure to automatically collect information, periodically retrieving learners' test results, homework submission status, and attendance information. A common database management system (DBMS) is used to connect to the database, and programming languages ​​such as Python or Java, along with their corresponding connection libraries, are employed.

[0471] The acquired information is analyzed on the server to evaluate the learner's level of understanding and any delays in progress. This analysis utilizes libraries such as Pandas and NumPy as data analysis tools. Based on the analysis, the server automates the process of generating customized learning plans for each learner using AI models. TensorFlow and PyTorch are used for these AI models.

[0472] The generated lesson plans are displayed on the display devices used by teachers. These devices are assumed to be computers running Windows or macOS, or smart devices. The program receives data from the server via a REST API and visualizes it. Data visualization technologies such as Matplotlib and Seaborn are used as visualization tools.

[0473] Furthermore, the system automatically generates and sends reports on learning progress to parents via electronic communication. The SMTP protocol is used for email transmission, allowing users to check the information as needed.

[0474] To analyze the learning trends of the entire class, statistical analysis is performed using R or SPSS based on data from the group, and the information is provided to teachers in the form of suggestions for improving the teaching content.

[0475] For example, if a student has a lack of understanding in a specific area of ​​mathematics, the server proposes an educational plan for that student, including video materials and practice problems, and notifies the teacher via the terminal. In this process, a generative AI model participates in selecting the materials, promoting effective learning.

[0476] Example of a prompt:

[0477] "Describe the process of using AI to analyze individual learners' underachievement in mathematics and suggest effective learning materials and problems."

[0478] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0479] Step 1:

[0480] The server retrieves learner test results, homework submission status, and attendance information from the data management structure. SQL queries are executed to collect the necessary information from the database as input. The output is a dataset containing the most recent learning data for each learner. This data is formatted using Python libraries to prepare it for subsequent analysis.

[0481] Step 2:

[0482] The server analyzes the acquired dataset. The input is the organized data obtained in Step 1. Using the data analysis tool Pandas, calculations are performed to evaluate each learner's level of understanding and progress. The output is the performance indicators for each learner and their evaluation results. Based on these results, learning obstacles and areas of strength and weakness are clearly identified.

[0483] Step 3:

[0484] The server generates personalized learning plans based on the analysis results. The input is the evaluation results obtained in step 2. Using a generative AI model, a plan is developed that recommends appropriate teaching materials and learning methods. The output is the content of the learning plan optimized for each learner. This plan is automatically generated by a model using TensorFlow.

[0485] Step 4:

[0486] The terminal presents the generated lesson plan to the teacher. The input is the individualized lesson plan received from the server. Information is transferred to the terminal using a REST API and visualized on the display device. Using Matplotlib, graphics are generated that clearly represent the level of understanding and the lesson plan. The output is a well-organized display of the lesson plan.

[0487] Step 5:

[0488] The server automatically generates and sends emails to parents reporting on the student's learning progress. The input consists of the student's latest progress data and the generated report content. The learning status is sent to a pre-configured email address using the SMTP protocol. The output is an email sent in a format that parents can review.

[0489] Step 6:

[0490] The server analyzes the learning trends of the entire group and reports to the teacher. The input is a collection of progress data for all learners. Statistical analysis is performed using R or SPSS to create a report detailing the learning trends and challenges of the entire class. The output is a report that clarifies the approach the teacher should take for the entire class.

[0491] (Application Example 1)

[0492] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0493] In an environment where individualized educational support is needed, traditional systems struggle to track each student's learning progress in real time and provide timely, appropriate instruction. Furthermore, the inability to assess students' understanding in real time and flexibly adjust instruction plans based on that assessment leads to decreased learning effectiveness. Additionally, parents face challenges in accurately understanding their children's learning progress when providing support at home.

[0494] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0495] In this invention, the server includes means for automatically collecting learning progress data from a learning management database, means for analyzing the collected learning progress data and detecting students' comprehension levels and learning delays, means for automatically generating individualized instruction plans based on the analysis results, means for presenting the generated instruction plans to a support device, means for presenting educational content audibly and visually and collecting student responses, and means for grasping the learning situation in real time based on the collected student responses and adjusting the instruction plan. This enables real-time instruction optimized for each individual student and provides effective learning support.

[0496] A "learning management database" is a database used to systematically store and provide educational information such as students' learning progress, test results, and attendance records.

[0497] "Learning progress data" refers to data that shows the progress of each student's learning activities, including their level of understanding and homework submission status.

[0498] "Analysis" is the process of organizing information based on collected data and evaluating the learning situation and trends of a particular student.

[0499] A "teaching plan" is an educational plan optimized for each individual student based on the analysis results, and includes recommended teaching materials and study schedules.

[0500] A "support device" is a hardware or software system that presents educational content to students and assists them in their learning.

[0501] "Presenting educational content aurally and visually" refers to a method of communicating subject matter and instruction to students using audio and video.

[0502] "Student responses" refer to students' reactions and behaviors towards instruction, and serve as a criterion for measuring their level of understanding and interest.

[0503] "Gathering information in real time and adjusting the teaching plan" means evaluating students' learning progress on the spot and immediately changing educational policies and materials as needed.

[0504] To implement this invention, a server must first connect to a learning management database and automatically collect student learning progress data. This process involves using software that periodically retrieves test results and attendance information from the database. The collected data is then processed by an analysis module, and AI technology is used to identify students' comprehension levels and learning delays. Generative AI models such as TensorFlow and PyTorch are utilized in this analysis.

[0505] Next, based on the analysis results, a personalized instruction plan is automatically generated for each student. This plan includes recommended teaching materials and guidelines for learning progress. The generated plan is presented to the user through an educational support device, which then presents the educational content to the student via voice and display. This support device utilizes voice recognition technology and a display interface, allowing the robot to assist the student's learning.

[0506] Users operate these devices to provide students with personalized learning content. During learning, student responses are recorded by sensors and cameras, and the server analyzes this data in real time, adjusting the lesson plan as needed. In this way, the most suitable education can be provided to each individual student.

[0507] For example, if a student has difficulty with a particular algebra unit, the server can detect this and prepare video materials and review exercises related to that unit. The user then presents these to the student via an assistive device. If the student still doesn't understand, additional practice exercises or explanations from different perspectives are automatically suggested as the next step.

[0508] Examples of prompts for a generative AI model:

[0509] "Based on the student's learning progress data, please suggest the next learning materials and assignments they should tackle. The student is in the second year of junior high school and has particular difficulty with algebra in mathematics."

[0510] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0511] Step 1:

[0512] The server accesses the learning management database and periodically collects learning progress data such as students' test results, homework submission status, and attendance information. Current information from the database is used as input, and the collected data is sent to an analysis module on the server. This ensures that the most up-to-date learning progress is always maintained.

[0513] Step 2:

[0514] The server analyzes the collected learning progress data and uses a generative AI model to detect each student's level of understanding and any learning delays. The input includes the learning progress data obtained in step 1, and the output provides an evaluation of understanding and an indicator of learning delays. Specifically, the AI ​​model uses TensorFlow or PyTorch to process the data and generate the evaluation results.

[0515] Step 3:

[0516] The server automatically generates individualized instruction plans for each student based on the analysis results. The input includes the analysis results from step 2, and the output is an instruction plan that includes individually optimized learning materials and schedules. Throughout this process, the plan is presented in a user-friendly format.

[0517] Step 4:

[0518] The server transmits the generated lesson plan to the educational support device, which presents it to the user and students via display and audio. The input is the lesson plan generated in step 3, and the output is the presentation of teaching materials visually and audibly. In this step, the terminal dynamically displays the teaching materials and assists students in their learning.

[0519] Step 5:

[0520] The server utilizes feedback from support devices to collect student responses as sensor data and monitor learning progress in real time. Input is real-time student response data transmitted from the support devices, and output is an evaluation result indicating the student's level of understanding and engagement. The terminal then uses this information to modify the instructional plan as needed.

[0521] Step 6:

[0522] The server readjusts the lesson plan as needed based on the updated learning data and sends it back to the educational support device. The input is the latest comprehension assessment based on student feedback, and the output is the improved lesson plan delivered to the support device. This continuously optimizes the learning of individual students.

[0523] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0524] This invention provides a system for comprehensively managing students' learning progress and emotional state in educational institutions. The system comprises a server, terminals, and an emotion recognition engine.

[0525] First, the server accesses the school's learning management database to automatically collect student test results, homework submission status, and attendance information. The server also integrates an emotion engine that infers emotions from students' facial expressions, tone of voice, and text-based feedback.

[0526] Teachers, as users, can access both learning data and emotional data through their devices. This allows them to see how students feel about the lesson content and to quantitatively grasp their level of understanding and interest.

[0527] The server analyzes collected learning and emotional data to automatically generate personalized instruction plans for each student. These plans are customized to take emotional states into account and propose an effective educational approach.

[0528] Parent reports are automatically generated by the server and delivered via email. These reports include not only information on learning progress but also emotional feedback, making it easier for parents to understand their child's overall situation.

[0529] The server also analyzes the learning trends of the entire class, including sentiment data, and provides this information to the teachers, who are the users of the server. This information helps to adjust the content and methods of lessons and serves as a guide to improve the learning efficiency of the entire class.

[0530] For example, if a student expresses anxiety about mathematics, the server assesses this emotion, suggests teaching methods and materials to alleviate their anxiety, and notifies the teacher via their device. Based on this, the teacher can then follow up with the student.

[0531] The following describes the processing flow.

[0532] Step 1:

[0533] The server automatically and periodically collects student test results, homework submission status, and attendance information from the learning management database. Additionally, the server activates an emotion engine to detect and collect students' facial expressions, voices, and text comments during online activities.

[0534] Step 2:

[0535] On their devices, teachers can access a dashboard that displays real-time learning and emotional data of students, allowing them to check each student's level of understanding and emotional state. This enables teachers to intuitively grasp the students' learning progress.

[0536] Step 3:

[0537] The server analyzes the collected learning and sentiment data. This analysis process uses AI algorithms to detect correlations between students' emotions and learning progress, identifying areas of misunderstanding or learning delays.

[0538] Step 4:

[0539] The server automatically generates individualized instruction plans for each student based on the analysis results. These plans are customized to take into account the student's emotional state and include recommended materials, teaching methods, and suggestions for emotional support.

[0540] Step 5:

[0541] Users can review the generated individualized tutoring plans on their devices and apply the most effective teaching approach to each student. If necessary, they can fine-tune the proposed plan to optimize the student's learning experience.

[0542] Step 6:

[0543] The server automatically generates reports for parents regarding their child's learning progress and emotional state. These reports are delivered via email, helping parents gain a comprehensive understanding of their child's situation at school.

[0544] Step 7:

[0545] The server analyzes the learning trends and sentiments of the entire class and provides the results to the teacher. Based on this information, the user can improve lesson content and adjust how they interact with students, thereby improving the overall learning effectiveness of the class.

[0546] (Example 2)

[0547] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0548] In educational settings, it is not easy to grasp each student's learning progress and emotional state in real time and provide appropriate guidance based on that information. In particular, quantitatively measuring students' comprehension, interests, or stress levels, and developing individualized instruction plans based on that information, is a time-consuming and laborious task using conventional methods. Furthermore, when parents try to understand their child's learning situation, it is difficult to obtain information that takes emotional aspects into account.

[0549] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0550] In this invention, the server includes means for automatically collecting learning data from a learning management database, means for analyzing the collected learning data and emotional data to evaluate students' comprehension and emotional states, and means for automatically generating adaptive instruction plans for each student based on the analysis results. This makes it possible for educational settings to comprehensively grasp each student's learning progress and emotional tendencies, and to quickly formulate individualized instruction plans based on this. Furthermore, it helps parents to more accurately understand their child's overall condition by providing a comprehensive learning status report that includes emotional feedback.

[0551] "Learning data" is a general term for information related to educational activities, including students' test results, homework submission status, and attendance information.

[0552] "Emotional data" refers to information about a student's emotional state, inferred from their facial expressions, tone of voice, and text-based feedback.

[0553] An "adaptive learning plan" is an individualized educational strategy designed to take into account each student's learning progress and emotional state.

[0554] "Automatic generation" refers to the process of creating or generating an object using a pre-configured program or algorithm, minimizing human intervention.

[0555] A "server" is a computer system or part thereof that provides services to other devices or software via a network.

[0556] "Information presentation devices" are devices used by students and teachers to visually or audibly perceive data, including computer monitors and tablet devices.

[0557] A "report" is a document created to organize and present specific information, in this case including learning progress and emotional feedback.

[0558] This invention provides a system for comprehensively managing students' learning progress and emotional state in educational institutions. This system is implemented with a configuration including a server, terminals, and an emotion recognition engine.

[0559] The server automatically collects student learning data by accessing the learning management database within educational institutions. This learning data includes information such as test results, homework submission status, and attendance records. The server incorporates an emotion recognition engine, which allows it to extract emotional data from students' facial expressions, tone of voice, and text-based feedback. The server aggregates this data and analyzes it using a generative AI model.

[0560] This system allows teachers, as users, to access both learning data and emotional data via their devices. Using dedicated software, teachers can monitor students' understanding and emotions in real time and develop appropriate lesson plans. The teachers' devices are equipped with information display devices to show information delivered from the server.

[0561] This system allows the server to analyze individual student data and automatically generate adaptive learning plans. These plans are customized and present educational approaches tailored to the student's cognitive and emotional needs. The server also automatically generates and sends reports to parents via email, including learning progress and emotional feedback.

[0562] For example, if a student expresses anxiety about learning mathematics, the server detects this using an emotion recognition engine and suggests teaching methods and materials to the teacher to help the student feel more at ease. This suggestion is then sent to the teacher's terminal, and the teacher uses this information to follow up with the student.

[0563] An example of a prompt is, "Create a description of a system that proposes individualized instruction plans that take into account students' emotions and levels of understanding." Based on this prompt, the generative AI model can generate a detailed description of the system.

[0564] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0565] Step 1:

[0566] The server accesses the educational institution's learning management database to collect student learning data. Inputs include student test results, homework submission status, and attendance information. The server queries these database entries in a specified format, formats the data, and stores it. The output is the most recent learning data for each student. Specifically, the server periodically runs automated scripts to retrieve information from the database.

[0567] Step 2:

[0568] The server analyzes students' emotional data via an emotion recognition engine. Inputs include student facial images, voice recordings, and text feedback. The server feeds this data to an emotion recognition algorithm to estimate the student's emotional state. Outputs include evaluation values ​​and categorical data representing each student's emotional state. Specifically, it performs real-time image processing and voice analysis, and records the emotion estimation results in a database.

[0569] Step 3:

[0570] The server integrates collected training data and sentiment data and performs analysis using a generative AI model. Both training data and sentiment data are used as input. The server runs the AI ​​model to evaluate each student's level of understanding and emotional tendencies. The output is a proposal for an individualized instruction plan suitable for each student. Specifically, the AI ​​model utilizes machine learning techniques to perform analysis that derives relationships from diverse data.

[0571] Step 4:

[0572] Through the terminal, teachers receive instruction plans and use them to guide their students. Inputs include individual instruction plans and their background data, delivered from the server. The terminal has software installed to display this information, allowing teachers to adjust their instruction based on visual information. The output is the teacher's effective instructional practice. Specifically, teachers can print the plans or share them with students through the school's online platform.

[0573] Step 5:

[0574] The server automatically generates and emails reports to parents containing learning progress and emotional feedback. The inputs used are integrated learning and emotional data, and instructional plan information. The server creates a report based on the data and sends it to the designated parent's email address. The output is a detailed learning report received by the parent. Specifically, report generation software collects data and creates a report based on a template.

[0575] Step 6:

[0576] The server analyzes the learning and emotional tendencies of the entire class and provides this information to the user, the teacher. Input includes learning and emotional data for all students in the class. The server performs statistical analysis to reveal group trends. The output is a report on the overall learning and emotional state of the class, which the teacher can evaluate. Specifically, it generates graphs and charts to provide information visually.

[0577] (Application Example 2)

[0578] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0579] In educational institutions, traditionally, only academic performance was emphasized when assessing students' learning progress, making it difficult to develop educational plans that considered the emotional state of learners. Furthermore, there was a lack of concrete guidelines to help parents understand their children's learning and emotional states, and to enable educational institutions and instructors to quickly adjust learning methods.

[0580] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0581] In this invention, the server includes means for automatically collecting learning progress information from a learning management database, means for analyzing the collected learning progress information and student emotional information to generate individualized instruction plans, means for presenting the generated instruction plans to a decision support device, means for automatically generating reports on learning and emotional states for parents, means for analyzing group learning and emotional tendencies and proposing adjustments to teaching methods, and means for evaluating learners' facial expressions and vocal characteristics using an emotion recognition engine. This enables educational institutions to comprehensively manage students' learning progress and emotional states, and to formulate effective educational plans and respond immediately.

[0582] A "learning management database" is an information management system for electronically storing and managing students' learning progress information and evaluation data.

[0583] "Learning progress information" refers to data that quantitatively shows each student's progress in learning activities, such as their academic performance, homework submission status, and attendance record.

[0584] "Emotional information" refers to information that indicates a student's feelings or emotional state, inferred from their facial expressions, voice, text data, etc.

[0585] A "teaching plan" is a plan that outlines individually customized educational guidelines and material selections based on students' learning progress and emotional state.

[0586] A "decision support device" is a tool or display device used by educators to make educational decisions based on students' learning data and emotional data.

[0587] A "report" is a document or electronic record that outlines a student's learning progress and emotional state, and is used to share this information with parents.

[0588] "Group-based learning and emotional tendencies" refer to statistical trends in students' learning progress and emotional patterns within a class or school as a whole.

[0589] An "emotion recognition engine" is a computational system or algorithm that infers a learner's emotional state through facial expression analysis, speech analysis, and text processing.

[0590] The system that implements this application consists of a server, a user terminal, and an emotion recognition engine. The server is designed using a Python framework (e.g., Django), and its database uses PostgreSQL. The server automatically collects learning progress information in real time from the learning management database and analyzes the collected information along with the students' emotion information.

[0591] The emotion recognition engine uses deep learning frameworks such as TensorFlow and PyTorch to process facial expression and audio data. This engine acquires data using the camera and microphone of a smartphone connected through the user's device. The acquired data is immediately sent to the server.

[0592] The server analyzes the received data and generates individualized instruction plans based on learning progress and sentiment data. The generated instruction plans are displayed on the user's terminal, assisting educators in decision-making.

[0593] Furthermore, reports based on this data will be regularly distributed to parents via email. These reports will include quantitative information on students' learning progress and emotional state, increasing transparency between educational institutions and parents.

[0594] For example, if a student shows interest or anxiety during a history lesson, the server analyzes this emotional data and generates a lesson plan to provide appropriate materials and support. This information is then notified to the educator on their device.

[0595] An example of a prompt using a generative AI model is: "Analyze student A's facial expression data and text comments, and propose teaching materials to keep them interested in history."

[0596] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0597] Step 1:

[0598] The server connects to a learning management database and automatically collects student learning progress information. The input is academic data retrieved from the learning management database, and the output is the collected learning progress information. It uses database queries to extract the necessary information and stores it on the server.

[0599] Step 2:

[0600] The device uses the smartphone's camera and microphone to collect students' facial expressions and voices. The input is the student's real-time emotional expression captured via the smartphone, and the output is processed emotional data. This data is sent to a server, enabling the emotion recognition engine to perform subsequent processing.

[0601] Step 3:

[0602] The server integrates and analyzes the received learning progress information and sentiment data. The inputs are the learning progress information collected in Step 1 and the sentiment data obtained in Step 2. The output is the integrated analysis result. Statistical methods and machine learning models are used to comprehensively evaluate learning progress and emotional state.

[0603] Step 4:

[0604] The server generates individualized instruction plans based on the analysis results. The input is the integrated analysis results, and the output is a customized instruction plan. Using a generative AI model, it generates prompts that suggest the most suitable learning support for each student.

[0605] Step 5:

[0606] The user, the educator, receives the lesson plan generated through the terminal and uses it for decision-making. The input is the lesson plan presented on the terminal, and the output is the educational measures selected by the educator. Based on the presented information, the educator adjusts the actual educational activities.

[0607] Step 6:

[0608] The server automatically generates and sends reports to parents via email. The input is integrated data on learning and emotional states, and the output is a report to parents. It periodically sends information using the email system.

[0609] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0610] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0611] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0612] [Fourth Embodiment]

[0613] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0614] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0615] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0616] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0617] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0618] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0619] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0620] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0621] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0622] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0623] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0624] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0625] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0626] This invention provides a system for efficiently managing the learning progress of each student in an educational institution. The system consists of a server, user terminals, and necessary software components.

[0627] First, the server accesses the school's learning management database and periodically collects learning progress data such as students' test results, homework submission status, and attendance information. This ensures that the data is always up-to-date.

[0628] Teachers, as users, can access learning data through their devices and view individual students' levels of understanding and areas of difficulty in graph and chart format. This allows them to intuitively grasp the current learning situation of their students.

[0629] The server analyzes the collected data and, if a particular student is showing signs of learning delays, uses AI to automatically generate a personalized tutoring plan tailored to that student. This plan includes recommended learning materials and methods for managing learning progress, allowing users to provide specific guidance to the student based on it.

[0630] In addition, the server generates reports for parents that show the student's learning progress and distributes them via electronic means such as email. Through these reports, parents can understand their child's learning activities at school and provide necessary support at home.

[0631] Furthermore, the server analyzes the learning trends of the entire class and provides teachers with information to help improve and adjust their lesson content. This overall analysis allows teachers to develop strategies to improve the learning effectiveness of the entire class.

[0632] For example, if a student's understanding of a specific area of ​​physics is analyzed as insufficient, the server automatically proposes a lesson plan for that student, including specific video materials and practice problems, and notifies the teacher via their device. The teacher can then review this plan and apply it to the student, enabling effective individualized instruction.

[0633] The following describes the processing flow.

[0634] Step 1:

[0635] The server accesses the learning management database to automatically collect student test results, homework submission status, and attendance information.

[0636] Step 2:

[0637] Based on the learning progress data collected by the server, AI is used to analyze each student's learning situation. This analysis utilizes statistical methods and machine learning techniques to detect students' comprehension levels and learning delays.

[0638] Step 3:

[0639] The server uses the analysis results to identify each student's level of understanding and areas of weakness, and then processes this information into graphs and charts.

[0640] Step 4:

[0641] Through the terminal, teachers can access visualized learning progress information and check the learning status of individual students. This information serves as a reference for teachers when making educational decisions.

[0642] Step 5:

[0643] The server automatically generates individualized instruction plans based on the analysis results and proposes specific teaching materials and methods for students who are experiencing learning delays.

[0644] Step 6:

[0645] On their devices, users can review the generated individualized instruction plans and use them as a reference when creating instructional plans for students.

[0646] Step 7:

[0647] The server automatically generates learning progress reports for parents and sends them via email. This allows parents to understand their child's learning progress and provide necessary support at home.

[0648] Step 8:

[0649] The server collects and analyzes learning data for the entire class, providing the teacher (user) with information on the class's overall understanding and learning trends. Based on this information, the teacher can adjust the lesson content.

[0650] (Example 1)

[0651] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0652] Traditional educational management systems struggled to provide appropriate educational plans tailored to the individual progress of learners, and also failed to adequately inform parents and teachers. This resulted in the failure to detect learners' lack of understanding or falling behind early, leading to missed educational opportunities. Furthermore, there was a lack of systems to analyze the learning trends of the entire group and provide specific information for improving educational content.

[0653] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0654] In this invention, the server includes means for automatically collecting information from a data management structure, means for analyzing the collected information to detect learners' level of understanding and progress delays, and means for automatically generating learning plans for learners based on the analysis results. This enables the provision of appropriate learning plans tailored to the individual circumstances of learners and effective information sharing.

[0655] A "data management structure" is a system for systematically storing and managing information, and includes databases and file systems.

[0656] "Means of automatically collecting information" refers to the process of automatically acquiring necessary data from specific sources according to time and conditions.

[0657] "Means of analyzing collected information" refers to methods and techniques for analyzing obtained data and deriving useful insights and patterns.

[0658] "Means for detecting learners' level of understanding and progress delays" refers to analytical methods for determining how well learners understand the learning material and whether their learning is behind schedule.

[0659] "Methods for automatically generating educational plans" refer to the process of creating learning policies and materials optimized for each individual learner based on analyzed data.

[0660] "Means of presenting on a display device" refers to a method of showing generated information or plans to a user through a screen or display.

[0661] "A method for automatically generating learning progress reports for parents" refers to a technique that compiles information on learners' progress and achievements and creates it in a format that is easy for parents to understand.

[0662] "Methods for analyzing the learning trends of an entire group" refer to analytical techniques that clarify common learning trends and challenges among multiple learners and use this information to improve overall education.

[0663] "Methods using generative AI models" refer to technologies that use artificial intelligence to analyze data and generate specific outputs.

[0664] "Methods of displaying data in charts and graphs" refer to methods of displaying data in graph or chart format to make it easier to understand visually.

[0665] "The function of transmitting using electronic communication means" refers to the function of transmitting information using digital communication means such as the internet or email.

[0666] This invention is a system for accurately and efficiently tracking learners' progress in educational management and providing each individual with an optimal educational plan. The invention is implemented as follows.

[0667] The server connects to a data management structure to automatically collect information, periodically retrieving learners' test results, homework submission status, and attendance information. A common database management system (DBMS) is used to connect to the database, and programming languages ​​such as Python or Java, along with their corresponding connection libraries, are employed.

[0668] The acquired information is analyzed on the server to evaluate the learner's level of understanding and any delays in progress. This analysis utilizes libraries such as Pandas and NumPy as data analysis tools. Based on the analysis, the server automates the process of generating customized learning plans for each learner using AI models. TensorFlow and PyTorch are used for these AI models.

[0669] The generated lesson plans are displayed on the display devices used by teachers. These devices are assumed to be computers running Windows or macOS, or smart devices. The program receives data from the server via a REST API and visualizes it. Data visualization technologies such as Matplotlib and Seaborn are used as visualization tools.

[0670] Furthermore, the system automatically generates and sends reports on learning progress to parents via electronic communication. The SMTP protocol is used for email transmission, allowing users to check the information as needed.

[0671] To analyze the learning trends of the entire class, statistical analysis is performed using R or SPSS based on data from the group, and the information is provided to teachers in the form of suggestions for improving the teaching content.

[0672] For example, if a student has a lack of understanding in a specific area of ​​mathematics, the server proposes an educational plan for that student, including video materials and practice problems, and notifies the teacher via the terminal. In this process, a generative AI model participates in selecting the materials, promoting effective learning.

[0673] Example of a prompt:

[0674] "Describe the process of using AI to analyze individual learners' underachievement in mathematics and suggest effective learning materials and problems."

[0675] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0676] Step 1:

[0677] The server retrieves learner test results, homework submission status, and attendance information from the data management structure. SQL queries are executed to collect the necessary information from the database as input. The output is a dataset containing the most recent learning data for each learner. This data is formatted using Python libraries to prepare it for subsequent analysis.

[0678] Step 2:

[0679] The server analyzes the acquired dataset. The input is the organized data obtained in Step 1. Using the data analysis tool Pandas, calculations are performed to evaluate each learner's level of understanding and progress. The output is the performance indicators for each learner and their evaluation results. Based on these results, learning obstacles and areas of strength and weakness are clearly identified.

[0680] Step 3:

[0681] The server generates personalized learning plans based on the analysis results. The input is the evaluation results obtained in step 2. Using a generative AI model, a plan is developed that recommends appropriate teaching materials and learning methods. The output is the content of the learning plan optimized for each learner. This plan is automatically generated by a model using TensorFlow.

[0682] Step 4:

[0683] The terminal presents the generated lesson plan to the teacher. The input is the individualized lesson plan received from the server. Information is transferred to the terminal using a REST API and visualized on the display device. Using Matplotlib, graphics are generated that clearly represent the level of understanding and the lesson plan. The output is a well-organized display of the lesson plan.

[0684] Step 5:

[0685] The server automatically generates and sends emails to parents reporting on the student's learning progress. The input consists of the student's latest progress data and the generated report content. The learning status is sent to a pre-configured email address using the SMTP protocol. The output is an email sent in a format that parents can review.

[0686] Step 6:

[0687] The server analyzes the learning trends of the entire group and reports to the teacher. The input is a collection of progress data for all learners. Statistical analysis is performed using R or SPSS to create a report detailing the learning trends and challenges of the entire class. The output is a report that clarifies the approach the teacher should take for the entire class.

[0688] (Application Example 1)

[0689] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0690] In an environment where individualized educational support is needed, traditional systems struggle to track each student's learning progress in real time and provide timely, appropriate instruction. Furthermore, the inability to assess students' understanding in real time and flexibly adjust instruction plans based on that assessment leads to decreased learning effectiveness. Additionally, parents face challenges in accurately understanding their children's learning progress when providing support at home.

[0691] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0692] In this invention, the server includes means for automatically collecting learning progress data from a learning management database, means for analyzing the collected learning progress data and detecting students' comprehension levels and learning delays, means for automatically generating individualized instruction plans based on the analysis results, means for presenting the generated instruction plans to a support device, means for presenting educational content audibly and visually and collecting student responses, and means for grasping the learning situation in real time based on the collected student responses and adjusting the instruction plan. This enables real-time instruction optimized for each individual student and provides effective learning support.

[0693] A "learning management database" is a database used to systematically store and provide educational information such as students' learning progress, test results, and attendance records.

[0694] "Learning progress data" refers to data that shows the progress of each student's learning activities, including their level of understanding and homework submission status.

[0695] "Analysis" is the process of organizing information based on collected data and evaluating the learning situation and trends of a particular student.

[0696] A "teaching plan" is an educational plan optimized for each individual student based on the analysis results, and includes recommended teaching materials and study schedules.

[0697] A "support device" is a hardware or software system that presents educational content to students and assists them in their learning.

[0698] "Presenting educational content aurally and visually" refers to a method of communicating subject matter and instruction to students using audio and video.

[0699] "Student responses" refer to students' reactions and behaviors towards instruction, and serve as a criterion for measuring their level of understanding and interest.

[0700] "Gathering information in real time and adjusting the teaching plan" means evaluating students' learning progress on the spot and immediately changing educational policies and materials as needed.

[0701] To implement this invention, a server must first connect to a learning management database and automatically collect student learning progress data. This process involves using software that periodically retrieves test results and attendance information from the database. The collected data is then processed by an analysis module, and AI technology is used to identify students' comprehension levels and learning delays. Generative AI models such as TensorFlow and PyTorch are utilized in this analysis.

[0702] Next, based on the analysis results, a personalized instruction plan is automatically generated for each student. This plan includes recommended teaching materials and guidelines for learning progress. The generated plan is presented to the user through an educational support device, which then presents the educational content to the student via voice and display. This support device utilizes voice recognition technology and a display interface, allowing the robot to assist the student's learning.

[0703] Users operate these devices to provide students with personalized learning content. During learning, student responses are recorded by sensors and cameras, and the server analyzes this data in real time, adjusting the lesson plan as needed. In this way, the most suitable education can be provided to each individual student.

[0704] For example, if a student has difficulty with a particular algebra unit, the server can detect this and prepare video materials and review exercises related to that unit. The user then presents these to the student via an assistive device. If the student still doesn't understand, additional practice exercises or explanations from different perspectives are automatically suggested as the next step.

[0705] Examples of prompts for a generative AI model:

[0706] "Based on the student's learning progress data, please suggest the next learning materials and assignments they should tackle. The student is in the second year of junior high school and has particular difficulty with algebra in mathematics."

[0707] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0708] Step 1:

[0709] The server accesses the learning management database and periodically collects learning progress data such as students' test results, homework submission status, and attendance information. Current information from the database is used as input, and the collected data is sent to an analysis module on the server. This ensures that the most up-to-date learning progress is always maintained.

[0710] Step 2:

[0711] The server analyzes the collected learning progress data and uses a generative AI model to detect each student's level of understanding and any learning delays. The input includes the learning progress data obtained in step 1, and the output provides an evaluation of understanding and an indicator of learning delays. Specifically, the AI ​​model uses TensorFlow or PyTorch to process the data and generate the evaluation results.

[0712] Step 3:

[0713] The server automatically generates individualized instruction plans for each student based on the analysis results. The input includes the analysis results from step 2, and the output is an instruction plan that includes individually optimized learning materials and schedules. Throughout this process, the plan is presented in a user-friendly format.

[0714] Step 4:

[0715] The server transmits the generated lesson plan to the educational support device, which presents it to the user and students via display and audio. The input is the lesson plan generated in step 3, and the output is the presentation of teaching materials visually and audibly. In this step, the terminal dynamically displays the teaching materials and assists students in their learning.

[0716] Step 5:

[0717] The server utilizes feedback from support devices to collect student responses as sensor data and monitor learning progress in real time. Input is real-time student response data transmitted from the support devices, and output is an evaluation result indicating the student's level of understanding and engagement. The terminal then uses this information to modify the instructional plan as needed.

[0718] Step 6:

[0719] The server readjusts the lesson plan as needed based on the updated learning data and sends it back to the educational support device. The input is the latest comprehension assessment based on student feedback, and the output is the improved lesson plan delivered to the support device. This continuously optimizes the learning of individual students.

[0720] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0721] This invention provides a system for comprehensively managing students' learning progress and emotional state in educational institutions. The system comprises a server, terminals, and an emotion recognition engine.

[0722] First, the server accesses the school's learning management database to automatically collect student test results, homework submission status, and attendance information. The server also integrates an emotion engine that infers emotions from students' facial expressions, tone of voice, and text-based feedback.

[0723] Teachers, as users, can access both learning data and emotional data through their devices. This allows them to see how students feel about the lesson content and to quantitatively grasp their level of understanding and interest.

[0724] The server analyzes collected learning and emotional data to automatically generate personalized instruction plans for each student. These plans are customized to take emotional states into account and propose an effective educational approach.

[0725] Parent reports are automatically generated by the server and delivered via email. These reports include not only information on learning progress but also emotional feedback, making it easier for parents to understand their child's overall situation.

[0726] The server also analyzes the learning trends of the entire class, including sentiment data, and provides this information to the teachers, who are the users of the server. This information helps to adjust the content and methods of lessons and serves as a guide to improve the learning efficiency of the entire class.

[0727] For example, if a student expresses anxiety about mathematics, the server assesses this emotion, suggests teaching methods and materials to alleviate their anxiety, and notifies the teacher via their device. Based on this, the teacher can then follow up with the student.

[0728] The following describes the processing flow.

[0729] Step 1:

[0730] The server automatically and periodically collects student test results, homework submission status, and attendance information from the learning management database. Additionally, the server activates an emotion engine to detect and collect students' facial expressions, voices, and text comments during online activities.

[0731] Step 2:

[0732] On their devices, teachers can access a dashboard that displays real-time learning and emotional data of students, allowing them to check each student's level of understanding and emotional state. This enables teachers to intuitively grasp the students' learning progress.

[0733] Step 3:

[0734] The server analyzes the collected learning and sentiment data. This analysis process uses AI algorithms to detect correlations between students' emotions and learning progress, identifying areas of misunderstanding or learning delays.

[0735] Step 4:

[0736] The server automatically generates individualized instruction plans for each student based on the analysis results. These plans are customized to take into account the student's emotional state and include recommended materials, teaching methods, and suggestions for emotional support.

[0737] Step 5:

[0738] Users can review the generated individualized tutoring plans on their devices and apply the most effective teaching approach to each student. If necessary, they can fine-tune the proposed plan to optimize the student's learning experience.

[0739] Step 6:

[0740] The server automatically generates reports for parents regarding their child's learning progress and emotional state. These reports are delivered via email, helping parents gain a comprehensive understanding of their child's situation at school.

[0741] Step 7:

[0742] The server analyzes the learning trends and sentiments of the entire class and provides the results to the teacher. Based on this information, the user can improve lesson content and adjust how they interact with students, thereby improving the overall learning effectiveness of the class.

[0743] (Example 2)

[0744] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0745] In educational settings, it is not easy to grasp each student's learning progress and emotional state in real time and provide appropriate guidance based on that information. In particular, quantitatively measuring students' comprehension, interests, or stress levels, and developing individualized instruction plans based on that information, is a time-consuming and laborious task using conventional methods. Furthermore, when parents try to understand their child's learning situation, it is difficult to obtain information that takes emotional aspects into account.

[0746] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0747] In this invention, the server includes means for automatically collecting learning data from a learning management database, means for analyzing the collected learning data and emotional data to evaluate students' comprehension and emotional states, and means for automatically generating adaptive instruction plans for each student based on the analysis results. This makes it possible for educational settings to comprehensively grasp each student's learning progress and emotional tendencies, and to quickly formulate individualized instruction plans based on this. Furthermore, it helps parents to more accurately understand their child's overall condition by providing a comprehensive learning status report that includes emotional feedback.

[0748] "Learning data" is a general term for information related to educational activities, including students' test results, homework submission status, and attendance information.

[0749] "Emotional data" refers to information about a student's emotional state, inferred from their facial expressions, tone of voice, and text-based feedback.

[0750] An "adaptive learning plan" is an individualized educational strategy designed to take into account each student's learning progress and emotional state.

[0751] "Automatic generation" refers to the process of creating or generating an object using a pre-configured program or algorithm, minimizing human intervention.

[0752] A "server" is a computer system or part thereof that provides services to other devices or software via a network.

[0753] "Information presentation devices" are devices used by students and teachers to visually or audibly perceive data, including computer monitors and tablet devices.

[0754] A "report" is a document created to organize and present specific information, in this case including learning progress and emotional feedback.

[0755] This invention provides a system for comprehensively managing students' learning progress and emotional state in educational institutions. This system is implemented with a configuration including a server, terminals, and an emotion recognition engine.

[0756] The server automatically collects student learning data by accessing the learning management database within educational institutions. This learning data includes information such as test results, homework submission status, and attendance records. The server incorporates an emotion recognition engine, which allows it to extract emotional data from students' facial expressions, tone of voice, and text-based feedback. The server aggregates this data and analyzes it using a generative AI model.

[0757] This system allows teachers, as users, to access both learning data and emotional data via their devices. Using dedicated software, teachers can monitor students' understanding and emotions in real time and develop appropriate lesson plans. The teachers' devices are equipped with information display devices to show information delivered from the server.

[0758] This system allows the server to analyze individual student data and automatically generate adaptive learning plans. These plans are customized and present educational approaches tailored to the student's cognitive and emotional needs. The server also automatically generates and sends reports to parents via email, including learning progress and emotional feedback.

[0759] For example, if a student expresses anxiety about learning mathematics, the server detects this using an emotion recognition engine and suggests teaching methods and materials to the teacher to help the student feel more at ease. This suggestion is then sent to the teacher's terminal, and the teacher uses this information to follow up with the student.

[0760] An example of a prompt is, "Create a description of a system that proposes individualized instruction plans that take into account students' emotions and levels of understanding." Based on this prompt, the generative AI model can generate a detailed description of the system.

[0761] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0762] Step 1:

[0763] The server accesses the educational institution's learning management database to collect student learning data. Inputs include student test results, homework submission status, and attendance information. The server queries these database entries in a specified format, formats the data, and stores it. The output is the most recent learning data for each student. Specifically, the server periodically runs automated scripts to retrieve information from the database.

[0764] Step 2:

[0765] The server analyzes students' emotional data via an emotion recognition engine. Inputs include student facial images, voice recordings, and text feedback. The server feeds this data to an emotion recognition algorithm to estimate the student's emotional state. Outputs include evaluation values ​​and categorical data representing each student's emotional state. Specifically, it performs real-time image processing and voice analysis, and records the emotion estimation results in a database.

[0766] Step 3:

[0767] The server integrates collected training data and sentiment data and performs analysis using a generative AI model. Both training data and sentiment data are used as input. The server runs the AI ​​model to evaluate each student's level of understanding and emotional tendencies. The output is a proposal for an individualized instruction plan suitable for each student. Specifically, the AI ​​model utilizes machine learning techniques to perform analysis that derives relationships from diverse data.

[0768] Step 4:

[0769] Through the terminal, teachers receive instruction plans and use them to guide their students. Inputs include individual instruction plans and their background data, delivered from the server. The terminal has software installed to display this information, allowing teachers to adjust their instruction based on visual information. The output is the teacher's effective instructional practice. Specifically, teachers can print the plans or share them with students through the school's online platform.

[0770] Step 5:

[0771] The server automatically generates and emails reports to parents containing learning progress and emotional feedback. The inputs used are integrated learning and emotional data, and instructional plan information. The server creates a report based on the data and sends it to the designated parent's email address. The output is a detailed learning report received by the parent. Specifically, report generation software collects data and creates a report based on a template.

[0772] Step 6:

[0773] The server analyzes the learning and emotional tendencies of the entire class and provides this information to the user, the teacher. Input includes learning and emotional data for all students in the class. The server performs statistical analysis to reveal group trends. The output is a report on the overall learning and emotional state of the class, which the teacher can evaluate. Specifically, it generates graphs and charts to provide information visually.

[0774] (Application Example 2)

[0775] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0776] In educational institutions, traditionally, only academic performance was emphasized when assessing students' learning progress, making it difficult to develop educational plans that considered the emotional state of learners. Furthermore, there was a lack of concrete guidelines to help parents understand their children's learning and emotional states, and to enable educational institutions and instructors to quickly adjust learning methods.

[0777] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0778] In this invention, the server includes means for automatically collecting learning progress information from a learning management database, means for analyzing the collected learning progress information and student emotional information to generate individualized instruction plans, means for presenting the generated instruction plans to a decision support device, means for automatically generating reports on learning and emotional states for parents, means for analyzing group learning and emotional tendencies and proposing adjustments to teaching methods, and means for evaluating learners' facial expressions and vocal characteristics using an emotion recognition engine. This enables educational institutions to comprehensively manage students' learning progress and emotional states, and to formulate effective educational plans and respond immediately.

[0779] A "learning management database" is an information management system for electronically storing and managing students' learning progress information and evaluation data.

[0780] "Learning progress information" refers to data that quantitatively shows each student's progress in learning activities, such as their academic performance, homework submission status, and attendance record.

[0781] "Emotional information" refers to information that indicates a student's feelings or emotional state, inferred from their facial expressions, voice, text data, etc.

[0782] A "teaching plan" is a plan that outlines individually customized educational guidelines and material selections based on students' learning progress and emotional state.

[0783] A "decision support device" is a tool or display device used by educators to make educational decisions based on students' learning data and emotional data.

[0784] A "report" is a document or electronic record that outlines a student's learning progress and emotional state, and is used to share this information with parents.

[0785] "Group-based learning and emotional tendencies" refer to statistical trends in students' learning progress and emotional patterns within a class or school as a whole.

[0786] An "emotion recognition engine" is a computational system or algorithm that infers a learner's emotional state through facial expression analysis, speech analysis, and text processing.

[0787] The system that implements this application consists of a server, a user terminal, and an emotion recognition engine. The server is designed using a Python framework (e.g., Django), and its database uses PostgreSQL. The server automatically collects learning progress information in real time from the learning management database and analyzes the collected information along with the students' emotion information.

[0788] The emotion recognition engine uses deep learning frameworks such as TensorFlow and PyTorch to process facial expression and audio data. This engine acquires data using the camera and microphone of a smartphone connected through the user's device. The acquired data is immediately sent to the server.

[0789] The server analyzes the received data and generates individualized instruction plans based on learning progress and sentiment data. The generated instruction plans are displayed on the user's terminal, assisting educators in decision-making.

[0790] Furthermore, reports based on this data will be regularly distributed to parents via email. These reports will include quantitative information on students' learning progress and emotional state, increasing transparency between educational institutions and parents.

[0791] For example, if a student shows interest or anxiety during a history lesson, the server analyzes this emotional data and generates a lesson plan to provide appropriate materials and support. This information is then notified to the educator on their device.

[0792] An example of a prompt using a generative AI model is: "Analyze student A's facial expression data and text comments, and propose teaching materials to keep them interested in history."

[0793] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0794] Step 1:

[0795] The server connects to a learning management database and automatically collects student learning progress information. The input is academic data retrieved from the learning management database, and the output is the collected learning progress information. It uses database queries to extract the necessary information and stores it on the server.

[0796] Step 2:

[0797] The device uses the smartphone's camera and microphone to collect students' facial expressions and voices. The input is the student's real-time emotional expression captured via the smartphone, and the output is processed emotional data. This data is sent to a server, enabling the emotion recognition engine to perform subsequent processing.

[0798] Step 3:

[0799] The server integrates and analyzes the received learning progress information and sentiment data. The inputs are the learning progress information collected in Step 1 and the sentiment data obtained in Step 2. The output is the integrated analysis result. Statistical methods and machine learning models are used to comprehensively evaluate learning progress and emotional state.

[0800] Step 4:

[0801] The server generates individualized instruction plans based on the analysis results. The input is the integrated analysis results, and the output is a customized instruction plan. Using a generative AI model, it generates prompts that suggest the most suitable learning support for each student.

[0802] Step 5:

[0803] The user, the educator, receives the lesson plan generated through the terminal and uses it for decision-making. The input is the lesson plan presented on the terminal, and the output is the educational measures selected by the educator. Based on the presented information, the educator adjusts the actual educational activities.

[0804] Step 6:

[0805] The server automatically generates and sends reports to parents via email. The input is integrated data on learning and emotional states, and the output is a report to parents. It periodically sends information using the email system.

[0806] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0807] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0808] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0809] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0810] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0811] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0812] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0813] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0814] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0815] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0816] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0817] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0818] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0819] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0820] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0821] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0822] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0823] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0824] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0825] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0826] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0827] The following is further disclosed regarding the embodiments described above.

[0828] (Claim 1)

[0829] A means of automatically collecting learning progress data from a learning management database,

[0830] A means of analyzing collected learning progress data to detect students' level of understanding and learning delays,

[0831] A means for automatically generating individualized instruction plans for each student based on the analysis results,

[0832] A means for displaying the generated lesson plan on a display device,

[0833] A method for automatically generating reports on learning progress for parents,

[0834] A means of analyzing the learning trends of the entire class,

[0835] A system that includes this.

[0836] (Claim 2)

[0837] The system according to claim 1, comprising means for visualizing students' level of understanding in a graph or chart based on analyzed learning progress data.

[0838] (Claim 3)

[0839] The system according to claim 1, which has a function to deliver learning progress reports to parents using electronic communication means.

[0840] "Example 1"

[0841] (Claim 1)

[0842] A means of automatically collecting information from a data management structure,

[0843] A means of analyzing collected information to detect learners' level of understanding and any delays in progress,

[0844] A means for automatically generating an educational plan for learners based on the analysis results,

[0845] A means for displaying the generated educational plan on a display device,

[0846] A method for automatically generating reports on learning progress for parents,

[0847] Methods for analyzing the learning trends of the entire group,

[0848] A means of using a generative AI model to propose personalized learning materials and training problems based on learner information,

[0849] Based on the learning trend data of the analyzed group, a means of proposing improvements to educational content,

[0850] A system that includes this.

[0851] (Claim 2)

[0852] The system according to claim 1, comprising means for displaying the learner's level of understanding in a chart or graph based on the analyzed information.

[0853] (Claim 3)

[0854] The system according to claim 1, which has a function to transmit learning progress reports to parents using electronic communication means.

[0855] "Application Example 1"

[0856] (Claim 1)

[0857] A means of automatically collecting learning progress data from a learning management database,

[0858] A means of analyzing collected learning progress data to detect students' level of understanding and learning delays,

[0859] A means for automatically generating individual student instruction plans based on analysis results,

[0860] A means for displaying the generated lesson plan on a display device,

[0861] A method for automatically generating reports on learning progress for parents,

[0862] A means of analyzing the learning trends of the entire class,

[0863] A means of presenting information to a support device for providing individualized instruction based on a learning plan,

[0864] A means of presenting educational content audibly and visually, and collecting student responses,

[0865] A means of understanding students' learning progress in real time based on collected responses and adjusting instructional plans accordingly.

[0866] A system that includes this.

[0867] (Claim 2)

[0868] The system according to claim 1, further comprising means for visualizing students' level of understanding in a graph or chart based on analyzed learning progress data and presenting it to students via an educational support device.

[0869] (Claim 3)

[0870] The system according to claim 1, which has a function to distribute learning progress reports to parents using electronic communication means and to encourage their use in supporting learning at home.

[0871] "Example 2 of combining an emotion engine"

[0872] (Claim 1)

[0873] A means of automatically collecting learning data from a learning management database,

[0874] A means of analyzing collected learning data and emotional data to evaluate students' comprehension and emotional state,

[0875] A means for automatically generating an adaptive instruction plan for each student based on the analysis results,

[0876] A means for displaying the generated lesson plan on an information display device,

[0877] A means of automatically generating reports for parents that include learning progress and emotional feedback,

[0878] A means of analyzing learning and emotional tendencies at the group level,

[0879] A system that includes this.

[0880] (Claim 2)

[0881] The system according to claim 1, comprising means for visualizing students' understanding and emotions based on analyzed learning data and emotion data.

[0882] (Claim 3)

[0883] The system according to claim 1, which has a function to deliver a report to parents using electronic communication means, including learning progress and emotional feedback.

[0884] "Application example 2 when combining with an emotional engine"

[0885] (Claim 1)

[0886] A means of automatically collecting learning progress information from a learning management database,

[0887] A means for analyzing collected learning progress information and student emotional information to generate individualized instruction plans,

[0888] A means for presenting the generated instruction plan to a decision support device,

[0889] A means of automatically generating reports for parents regarding learning and emotional states,

[0890] A means of analyzing the learning and emotional tendencies of groups and proposing adjustments to educational methods,

[0891] A means for evaluating the facial expressions and vocal characteristics of learners using an emotion recognition engine,

[0892] A system that includes this.

[0893] (Claim 2)

[0894] The system according to claim 1, comprising means for visualizing students' level of understanding and emotional state based on analyzed learning progress information and emotional information.

[0895] (Claim 3)

[0896] The system according to claim 1, which has a function to distribute reports on learning and emotional states to parents and educators using electronic communication means. [Explanation of symbols]

[0897] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. A means of automatically collecting learning progress data from a learning management database, A means of analyzing collected learning progress data to detect students' level of understanding and learning delays, A means for automatically generating individualized instruction plans for each student based on the analysis results, A means for displaying the generated lesson plan on a display device, A method for automatically generating reports on learning progress for parents, A means of analyzing the learning trends of the entire class, A system that includes this.

2. The system according to claim 1, comprising means for visualizing students' level of understanding in a graph or chart based on analyzed learning progress data.

3. The system according to claim 1, which has a function to deliver learning progress reports to parents using electronic communication means.

Images