system

Abstract

We provide the system. [Solution] A means of connecting to an educational database to collect the latest information and generating teaching materials based on predetermined standards, A means of distributing generated learning materials to learner devices and tracking and recording learning progress, A means of analyzing questions from learners and providing relevant learning materials, A means of analyzing learning data and updating the content of the learning materials, A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor, the method including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a character of the chatbot, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance as a response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In modern educational settings, the workload of educators has increased, and it is difficult for them to improve their own IT skills. Also, due to regional differences, the quality of education is uneven, creating barriers to improving students' information and communication technology (ICT) skills. It is necessary to solve these problems and provide students with high-quality IT education of consistent quality.

Means for Solving the Problems

[0005] This invention provides a system that connects to an educational database to automatically collect the latest information and generates teaching materials based on the collected information and educational standards. This system distributes the generated teaching materials to learner terminals and includes functions for tracking and recording learner progress. Furthermore, it analyzes learner questions in real time and provides relevant teaching materials to aid student understanding. It also optimizes educational content by analyzing collected learning data and updating the teaching material content. Through these means, the system reduces the workload of teachers, provides consistently high-quality IT education to students, and supports the improvement of their ICT skills.

[0006] An "educational database" is a collection of data that stores and manages educational information necessary for generating teaching materials and supporting learning, in an accessible format.

[0007] "Gathering the latest information" means automatically obtaining the latest IT-related data and educational trends from external sources.

[0008] "Generating educational materials" refers to the act of automatically constructing educational content for learners based on collected information and existing educational standards.

[0009] A "learner terminal" is a computing device used by learners to access learning materials and engage in learning activities.

[0010] "Tracking and recording progress" is the process of monitoring a learner's learning process and saving data on their level of achievement and understanding.

[0011] "Analyzing questions and providing relevant materials" means understanding the questions received from learners and presenting appropriate educational resources accordingly.

[0012] "Analyzing learning data and updating the course materials" means evaluating the collected learning data and improving or updating the course materials based on the results.

[0013] "Optimizing educational content" is the process of adjusting the content and structure of educational materials in order to improve their quality and effectiveness. [Brief explanation of the drawing]

[0014] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11] This is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] This is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] This is a sequence diagram showing the processing flow of the data processing system in Example 2, which incorporates an emotion engine. [Figure 14]It is a sequence diagram showing the processing flow of a data processing system in Application Example 2 when a sentiment engine is combined.

Embodiments for Carrying Out the Invention

[0015] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0016] First, the terms used in the following description will be explained.

[0017] In the following embodiments, a numbered processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0018] In the following embodiments, a numbered RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0019] In the following embodiments, a numbered storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, etc.

[0020] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0021] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0022] [First Embodiment]

[0023] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0024] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0025] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0026] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0027] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0028] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0029] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0030] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0031] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0032] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0033] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0034] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0035] This invention relates to an educational support system for improving learners' ICT skills in educational institutions. The system includes a server, learner terminals, and an AI component with advanced data processing capabilities.

[0036] Course material generation and distribution

[0037] The server is programmed to access educational databases and regularly collect the latest educational information. Based on this collected information, the server automatically generates teaching materials that conform to the curriculum guidelines and current IT trends, and is designed to be appropriate for each grade level and level. These materials are customized according to specific educational needs and, after generation, are delivered to learners' devices.

[0038] Tracking and recording learning progress

[0039] The learner terminal provides learners with learning materials received through a user interface. As learners use the materials, the terminal tracks their progress in real time and records data to evaluate their level of understanding. This data is sent to a server to generate feedback tailored to each learner's level of understanding and pace.

[0040] Question answering function

[0041] Users can ask questions or express concerns about specific topics during their learning process. The server receives these questions, analyzes them using an AI program, and provides appropriate learning materials and answers. This process is fast and supports users in continuing their learning without interruption.

[0042] Data analysis and course material updates

[0043] The server analyzes collected learning data and provides insights to improve the quality of learning materials. Based on continuous data analysis, the learning materials are updated as needed, providing learners with the most effective and up-to-date educational content. This enables consistent IT education across educational institutions, aiming to reduce the workload of teachers while supporting the improvement of students' ICT skills.

[0044] This coordinated operation of the entire system will bring innovation to the educational setting and enable it to address the diverse learning needs of students.

[0045] The following describes the processing flow.

[0046] Step 1:

[0047] The server establishes a connection to the educational database and regularly collects the latest educational information and IT-related data. This ensures that the server is always up-to-date with the latest trends and educational standards.

[0048] Step 2:

[0049] The server analyzes the collected information and uses AI algorithms to automatically generate learning materials tailored to each grade level and educational standard. These materials consist of text, images, videos, and other formats, and are dynamically customized according to the learning content.

[0050] Step 3:

[0051] The server distributes the generated learning materials to the learner's device. The learner's device displays these materials to the student through its user interface and prepares to begin learning.

[0052] Step 4:

[0053] Users engage in learning activities using learning materials presented on their learning devices. The learning devices track learning progress in real time, evaluate understanding and results, and report them to the server.

[0054] Step 5:

[0055] The server receives questions from users and analyzes their content using AI. Based on the analyzed questions, it generates appropriate explanations and supplementary materials, and presents them to the user's terminal as needed.

[0056] Step 6:

[0057] The server analyzes learner progress data and accumulated learning data. This allows it to identify areas for improvement in the learning materials and gain insights that can be implemented in future lessons.

[0058] Step 7:

[0059] Based on the analysis results, the server adjusts and updates the content and structure of the teaching materials to prepare more effective educational content. This ensures improved learning effectiveness and consistency in education.

[0060] (Example 1)

[0061] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0062] Traditional educational support systems have problems with providing learners with effective and efficient learning materials, tracking their progress, and providing personalized feedback. Furthermore, they have challenges in responding quickly and accurately to learners' questions, and insufficient optimization of learning materials based on data.

[0063] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0064] In this invention, the server includes means for generating learning materials based on educational information, means for distributing the generated learning materials to terminals and recording learning progress, and means for analyzing inquiries from learners and providing relevant learning materials. This enables the provision of optimal learning materials tailored to each learner's progress and prompt and accurate question answering, thereby providing an overall efficient learning environment.

[0065] "Educational information" refers to all data and materials related to educational activities, including information related to curriculum guidelines and learner progress.

[0066] "Learning materials" are content created for learners to use for educational purposes and are provided in formats such as text, images, and videos.

[0067] "Terminal" refers to a device used by learners to receive and manipulate learning materials, and includes personal computers, tablets, and smartphones.

[0068] "Progress" refers to indicators or records that show how much progress learners have made in educational activities.

[0069] An "inquiry" refers to a question that a learner submits to the system seeking answers or information they want to resolve.

[0070] "Natural language" refers to the language that humans use in everyday life, and the system is required to understand questions and instructions by analyzing this language.

[0071] A "generative AI model" refers to a collection of programs and algorithms that use artificial intelligence to automatically create text and content.

[0072] "Optimizing educational materials" refers to the process of adjusting the content and structure of educational materials so that they are more effective and appropriate for learners.

[0073] This invention provides an educational support system that improves learners' ICT skills. The system consists of a server, learner terminals, and an AI component with advanced data processing capabilities.

[0074] The server connects to the database and uses Python scripts and SQL queries to collect educational information. The collected data is passed to an AI component, where a generative AI model automatically generates learning materials based on the curriculum guidelines. A general-purpose AI platform is used for this process. The generated materials are created in HTML format and adjusted according to specific educational needs.

[0075] The server delivers generated learning materials to the device using the HTTP protocol. The device displays the learning materials through an interface using React or similar technologies, allowing users to interactively learn from the received content. As learners use the learning materials, the device tracks user actions in real time using JavaScript® and other technologies, and records their progress. This makes it possible to monitor learners' progress in detail.

[0076] Users can ask questions via voice or text if they encounter difficulties during their learning process. The server analyzes the received questions using natural language processing techniques and provides relevant learning materials and answers. This communication utilizes standard network connectivity technologies.

[0077] The server uses Python analysis tools to analyze learner progress and collected learning data. The analysis results are visualized using frameworks such as Django, and used to retrain the AI ​​and continuously improve the learning materials.

[0078] As a concrete example, a server might use the prompt "Mathematics materials for first-year high school students" to generate learning materials on "Fundamentals of Functions." These materials are delivered as HTML content combined with images and graphs for easy understanding. If a user asks, "I don't understand functions," the server immediately suggests reviewing the relevant section.

[0079] By coordinating the entire system in this way, it becomes possible to create an efficient and effective learning environment in educational settings.

[0080] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0081] Step 1:

[0082] The server accesses an educational database and collects educational information. It uses Python scripts and SQL queries to gather the latest curriculum guidelines and teaching materials. This data is provided to the server as input and serves as the source data for generating learning materials. Specifically, it retrieves data such as "Mathematics Curriculum for First-Year High School Students" from the database. As output, it sends a set of educational information to the AI ​​model for generating learning materials.

[0083] Step 2:

[0084] The server automatically generates learning materials using a generative AI model. The educational information collected in Step 1 is used as input. The server provides this information to the AI ​​component, which analyzes it using natural language processing techniques. As a result of the data processing, HTML-formatted learning materials tailored to the learner's level are output. Specifically, it generates learning materials on "Fundamentals of Functions," incorporating diagrams and practice problems.

[0085] Step 3:

[0086] The server delivers the generated learning materials to the terminal. The HTML-formatted learning materials created in step 2 are used as input. The server sends these to the terminal via the HTTP protocol. As output, learning materials that can be displayed on the learner's terminal are prepared. Specifically, when the user opens the learning materials, they are displayed interactively in the terminal's browser.

[0087] Step 4:

[0088] The terminal tracks the learner's progress through a user interface. The learner's operation data is recorded as input on the terminal. Real-time data tracking is performed using JavaScript. Progress and answer results are sent to the server as output. Specifically, each time the learner solves a problem, the result is logged, and the accuracy rate and response time are analyzed.

[0089] Step 5:

[0090] The user enters their question into the terminal. This is sent to the server as input, and the server analyzes the question using natural language processing technology. As a result, the appropriate learning material or answer is output and presented back to the terminal. For example, if the user asks "I don't understand this function," the relevant learning material section is immediately displayed.

[0091] Step 6:

[0092] The server continuously analyzes learning data and updates the learning materials. Progress data collected in step 4 is used as input. By analyzing the data using Python analysis tools and visualizing it with Django or similar tools, a list of improvement suggestions is obtained as output. Specifically, if a particular section of a learning course is deemed incomprehensible, the content of that material is improved and redistributed.

[0093] (Application Example 1)

[0094] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0095] In today's educational environment, there is a need to provide education tailored to individual progress and understanding, and to efficiently improve ICT skills. Furthermore, there is a demand for educational systems that can quickly respond to learners' questions and provide real-time educational feedback. Traditional educational systems struggle to optimize this individualized approach, limiting their ability to efficiently facilitate learning.

[0096] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0097] In this invention, the server includes means for connecting to educational information sources to collect the latest data and generate educational materials based on predetermined criteria; means for distributing the generated educational materials to learner devices and tracking and recording educational progress; and means for providing real-time educational feedback through eye-tracking or voice input. This enables the provision of up-to-date educational content tailored to individual learners, improves learning efficiency, and allows for immediate and responsive educational support.

[0098] "Educational information sources" are databases and online resources that provide data and materials related to learning.

[0099] "Educational materials" refer to a collection of information provided to learners for the purpose of learning, and include textbooks and reference materials.

[0100] "Learner equipment" refers to electronic devices used by learners to receive and utilize educational materials.

[0101] "Educational progress" is an indicator that shows the extent to which learners understand and have acquired specific educational objectives.

[0102] "Eye-tracking" is a technology that tracks the movement of a learner's eyes, and uses this data to manipulate interfaces or analyze situations.

[0103] "Voice input" is a method of inputting instructions or data into a device using voice.

[0104] "Educational feedback" refers to advice and suggestions provided based on a learner's learning progress, with the aim of improving their learning methods and understanding.

[0105] The system for realizing this application consists of a server, a learner terminal, smart glasses, and an AI component. The server periodically collects data from educational information sources and automatically generates educational materials based on predetermined criteria. The generated educational materials are delivered to learners via the learner terminal or smart glasses.

[0106] By using smart glasses, learners can have their gaze detected and be effectively guided through educational materials. Using voice input, users can ask questions, and the server provides immediate, relevant educational feedback. Furthermore, learner progress is recorded in real time, and the server generates individually tailored educational feedback based on this data.

[0107] The hardware used will be smart glasses, and Google's cloud-based speech recognition service will be utilized. Additionally, the Django framework, running on a server, will be used for progress tracking and material distribution, while Tensorflow (registered trademark) will be used for AI analysis. A concrete example is a citizen wearing smart glasses in a public place and receiving ICT education while taking a walk. An example of a prompt might be: "A new citizen wearing smart glasses will view educational content while walking in a park and ask questions to an AI assistant in real time. Please explain how they are using eye-tracking and voice to learn."

[0108] In this way, the system implementing the invention can support the improvement of learners' ICT skills through the generation, distribution, progress tracking, and provision of real-time feedback of educational materials.

[0109] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0110] Step 1:

[0111] The server connects to educational information sources and collects the latest educational data. Input includes information from external databases and online platforms via the internet. This data is then processed and converted into a format that conforms to specified educational standards. This enables the generation of educational materials tailored to learners.

[0112] Step 2:

[0113] The server uses a generative AI model to create educational materials based on the collected data. The input is the educational data processed in Step 1, and by processing this data with the generative AI model, it outputs customized educational materials tailored to each grade level and level. The output materials are saved as educational content and distributed in the next step.

[0114] Step 3:

[0115] The server delivers educational materials to the learner's device or smart glasses. The input is the educational material created in step 2, which is then sent to the learner's device. During this process, a communication protocol is used to transfer the data, making the material available for download on the device. The output is the learner's device with the educational material downloaded.

[0116] Step 4:

[0117] The terminal presents the delivered educational materials through its user interface function. The input is the educational materials downloaded in step 3, and these materials are provided to the user via a display device or audio output. No data calculation is required; the primary operation is the presentation of data.

[0118] Step 5:

[0119] The user learns while wearing smart glasses and asks questions using eye-tracking and voice input functions. The input consists of the user's voice and eye-tracking data. This data is collected by the device and sent to the server. The output is information on the user's learning progress and the content of the questions, which are processed in the next step.

[0120] Step 6:

[0121] The server analyzes the learning data and questions submitted by the user and provides relevant feedback in real time. The input is the user data collected in step 5, which is processed by the AI ​​analysis engine to generate optimal feedback and additional educational materials. The output is the feedback information returned to the learner.

[0122] Step 7:

[0123] The server continuously monitors learning data and tracks progress to optimize educational materials. The input is accumulated learning progress data, which is then analyzed to extract insights necessary for the next material update. The output is improvement suggestions that will be reflected in the next material update.

[0124] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0125] This invention provides an educational support system that incorporates an emotion engine for recognizing user emotions, thereby offering an educational experience tailored to the individual learner's needs. This system is constructed through the collaboration of a server, a learner terminal, and the emotion engine.

[0126] Emotion recognition and content adjustment

[0127] The device recognizes the learner's emotions through camera and microphone sensors and analyzes that data. For example, if a user shows signs of impatience during learning, the emotion engine analyzes this information and adjusts the difficulty level of the learning content or displays an encouraging message.

[0128] Learning progress and feedback

[0129] The server combines emotional data with learning progress to generate more personalized feedback. Based on emotional data, if the user is experiencing stress, it adjusts learning time or provides content to help them relax. This kind of feedback improves the user experience and enhances learning efficiency.

[0130] Analysis and improvement of the educational environment

[0131] The server collects and analyzes sentiment data from multiple users to evaluate the overall learning environment. Based on this information, it provides insights that help improve educational policies and course materials. For example, for specific topics that many users find difficult simultaneously, the course materials are re-evaluated and revised as needed.

[0132] Specific example

[0133] If the system detects that the user is unfamiliar with the development environment and shows signs of anxiety, it will provide a simple user guide and recommend video learning materials to aid understanding. Furthermore, if the system determines that the user's understanding is low, it will provide links to more detailed materials and additional practice problems on the learner's device.

[0134] Systems with these functions aim to maximize learning efficiency and provide a comfortable learning environment by responding appropriately in real time according to the individual circumstances of each learner. In this way, the present invention proposes a new approach to enhancing the learning experience by combining emotion recognition technology and educational content.

[0135] The following describes the processing flow.

[0136] Step 1:

[0137] The user logs into the learner's device and accesses the designated educational content. The device uses its camera and microphone to capture the user's facial expressions and voice tone in real time.

[0138] Step 2:

[0139] The device sends the captured audio and video data to the emotion engine. The emotion engine analyzes this data to identify the user's emotional state (e.g., concentration, joy, stress).

[0140] Step 3:

[0141] The server receives emotion data from the emotion engine and integrates it with the user's current learning progress data. Based on this integrated data, it considers customizing the learning content and feedback.

[0142] Step 4:

[0143] The device provides the user with appropriate feedback and tailored learning content based on instructions received from the server. For example, if the user is detected as being in a state of agitation, a message prompting them to take a break will be displayed.

[0144] Step 5:

[0145] As the user continues to learn, emotion and progress data are continuously collected and sent to the server by the device. A real-time feedback loop is formed.

[0146] Step 6:

[0147] The server analyzes long-term sentiment data to generate insights for improving the quality of educational content. This information is used to revise teaching materials and review educational policies.

[0148] Step 7:

[0149] Throughout the entire system, the terminal and server continuously work together to enable flexible responses tailored to the user's emotions and learning progress. This provides the optimal educational environment for learners.

[0150] (Example 2)

[0151] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0152] Traditional educational support systems often fail to adequately address the individual needs of learners because they provide standardized learning content and progress management without considering the learner's emotional state. In particular, there is a lack of feedback that responds to the stress and changes in understanding that learners experience, making it difficult to maximize learning efficiency.

[0153] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0154] In this invention, the server includes means for collecting and analyzing video and audio data to identify the user's emotional state, means for adjusting the difficulty level of learning materials or presenting supplementary information based on the analyzed emotional state, and means for combining the emotional data with the learning progress to generate individualized feedback for the learner. This enables the provision of an educational experience tailored to the individual emotional state of the learner, thereby improving learning efficiency.

[0155] "User emotional state" refers to the psychological state exhibited by users during learning, such as joy, impatience, and stress, and is identified through video and audio data.

[0156] "Video and audio data" refers to digital data including the user's captured facial expressions and recorded voice, which is used to analyze their emotional state.

[0157] "Means of analysis" refers to technical methods and devices used to identify and evaluate emotional states from collected video and audio data.

[0158] "Adjusting the difficulty level of learning materials" refers to the process of changing the content and difficulty level of learning materials according to the user's emotional state.

[0159] "Supplementary information" refers to additional learning materials and support information provided to assist users in their learning, with the aim of improving learning efficiency.

[0160] "Individualized feedback" refers to specialized advice and instructions provided based on each learner's emotional state and learning progress.

[0161] "Learner progress" refers to a standard used to measure the degree of progress a learner has achieved during the educational process and to evaluate their growth based on that progress.

[0162] "Educational experience" is a collective term for the series of learning processes that learners acquire through an educational system, and the emotional and intellectual responses that accompany them.

[0163] This invention provides an educational support system that recognizes the emotional state of learners and adjusts the educational experience based on that state. This system is realized through the cooperation of a learner terminal, a server, and an emotion engine.

[0164] Terminal role

[0165] The device collects video and audio data from learners through sensors. Specifically, it records the learner's facial expressions with a camera and acquires audio with a microphone. This data is preprocessed using image processing libraries and speech analysis tools (e.g., OpenCV or Google Cloud Speech-to-Text).

[0166] Emotion recognition and analysis

[0167] The emotion engine analyzes data received from the device and uses machine learning models (e.g., TensorFlow or PyTorch-based models) to identify the learner's emotional state. This process makes it possible to recognize the learner's emotions in real time.

[0168] Server Role

[0169] Based on analyzed sentiment data, the server adjusts the difficulty level of learning materials to provide personalized educational content for each learner. It also provides supplementary materials as needed and generates individualized feedback. Furthermore, the server aggregates sentiment data from multiple users to gain insights that guide improvements to the overall learning environment.

[0170] Examples of specific cases and prompt statements

[0171] For example, if a user is feeling anxious about using a new programming environment, the device will detect their anxiety and the server will provide a simple user guide or instructional video. Also, if a user is stressed about a particular topic, the server will suggest relaxing content.

[0172] An example of a prompt for a generative AI model is: "Design a system that provides appropriate learning materials and feedback based on emotions, so that students can learn the fundamentals of programming effectively."

[0173] This system is designed to ensure that all learners have the optimal learning experience, and it provides personalized learning support using emotion recognition technology.

[0174] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0175] Step 1:

[0176] The device collects video and audio data from learners using sensors. It takes facial expressions captured by the camera and audio recorded by the microphone as input, and preprocesses this data using image processing libraries and audio analysis tools. This provides foundational data for analyzing the learners' emotional states.

[0177] Step 2:

[0178] The device sends pre-processed data to the emotion engine, which analyzes the learner's emotional state. The input data is analyzed using a machine learning model, and the emotional state is determined as output. Specifically, the model assigns emotional labels such as joy, anxiety, and stress.

[0179] Step 3:

[0180] The server receives the analyzed emotional state data and adjusts the learning content. Based on the input emotional data, it decides whether to adjust the difficulty level of the learning materials or provide supplementary information. For example, it might generate materials with a lower difficulty level or video links to aid understanding.

[0181] Step 4:

[0182] The server integrates emotional data and learning progress data to generate personalized feedback. Based on the input data, it calculates what kind of feedback is most effective and presents the user with specialized advice and suggestions for breaks as output.

[0183] Step 5:

[0184] The server aggregates emotional data collected from multiple learners to gain insights into the overall educational environment. Based on the accumulated data, suggestions for improving educational policies and materials are made, and reports and plans for optimizing the educational environment are presented as output.

[0185] (Application Example 2)

[0186] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0187] In today's learning environment, a challenge exists in that learners often struggle to receive education that is tailored to their individual emotional states. Traditional education systems respond to learning progress but rarely directly consider learners' emotions, thus limiting effective individualized instruction. Furthermore, when students experience stress or anxiety during learning, they often do not receive appropriate support, which leads to decreased learning efficiency.

[0188] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0189] In this invention, the server includes means for connecting to an educational information repository to collect the latest information and generating teaching materials based on predetermined standards; means for distributing the generated teaching materials to student terminals and tracking and recording learning progress; and means for identifying the emotions of students and providing learning support in accordance with those emotions. This makes it possible to provide immediate individualized instruction that takes into account the emotional state of students, thereby improving the effectiveness of learning.

[0190] The "Educational Information Library" is a data storage system that stores and makes accessible the latest data on learning materials, subject matter materials, and educational guidance provided to students.

[0191] "Student devices" refer to electronic devices used by learners to access educational content and progress in their studies, and include tablets, personal computers, and smartphones.

[0192] "Means for identifying emotions" refers to functions that include technical elements for analyzing the participant's facial expressions, tone of voice, etc., to determine the participant's emotional state.

[0193] "Means of providing learning support" refers to functions that provide adjustments to learning content, advice, and encouragement according to the learner's progress and emotions.

[0194] "Means for tracking and recording learning progress" refers to systems and functions for monitoring how far a student has progressed in their ongoing learning and recording that data.

[0195] The system that realizes this invention includes an educational information repository, student terminals, and an emotion recognition engine, each of which functions in cooperation with one another.

[0196] The server connects to the educational information repository and collects and updates the latest learning content to provide to students. This content is automatically generated based on pre-configured educational standards and is designed to meet diverse learning needs.

[0197] The learner's device collects the learner's facial expressions and voice in real time using a camera and microphone, and transmits this data to an emotion recognition engine. The emotion recognition engine incorporates commercially available technological elements, which analyze the learner's emotional state. If the learner shows signs of stress or anxiety, the device can automatically adjust the difficulty level of the content or display encouraging messages.

[0198] Furthermore, the server combines learning progress and sentiment data to generate feedback tailored to each learner's individual situation. As a result, learners can continue learning at a pace that is optimal for them. The data in the educational information repository is regularly updated, providing highly accurate feedback that accommodates various learning patterns.

[0199] As a concrete example, suppose a student is working on a new mathematical proposition and a confused expression is detected. In this case, the system analyzes the expression and immediately displays links to relevant video tutorials and practice problems to aid understanding.

[0200] An example of a prompt would be: "Please tell me how to adjust the learning content based on the emotions the learner is experiencing while working on a particular problem. Currently, the learner is not concentrating."

[0201] This system can significantly improve the quality of education by combining emotion recognition with personalized learning support.

[0202] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0203] Step 1:

[0204] The device captures the participant's facial expressions and voice using a camera and microphone. Real-time video and audio data are obtained as input. This data is sent to an emotion recognition engine, where it is converted into emotional states using facial recognition algorithms and voice analysis. The output is digital data indicating the participant's emotional state.

[0205] Step 2:

[0206] The server receives emotional state data and combines it with the learner's learning progress data. In this step, an adaptive learning algorithm is used to determine which learning content is appropriate based on the learner's current learning status and emotional state. The input is numerical data of learning progress and emotional state data, and the output is a list of adjusted learning content.

[0207] Step 3:

[0208] The device presents the learner with customized learning content. Specifically, video tutorials, supplementary materials, or new practice exercises are displayed on the learner's device. The input is a list of learning content received from the server, and the output is the learning material displayed on the learner's screen.

[0209] Step 4:

[0210] The user (learner) engages in learning activities based on the information presented. If a new question arises during this process, the device analyzes the question and presents additional relevant learning materials. The input is the user's question, and the output is a list of relevant learning materials. A question analysis algorithm uses a generative AI model to create prompts to identify appropriate learning materials.

[0211] Step 5:

[0212] The server accumulates learning history and sentiment data, and generates statistical data for future system improvements. This data serves as a starting point for developing new algorithms and contributes to the continuous optimization of educational content. The input is past learning session data, and the output is statistical data and improvement suggestions.

[0213] This will enable the entire system to provide a flexible learning experience tailored to the specific needs of each student.

[0214] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0215] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0216] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0217] [Second Embodiment]

[0218] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0219] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0220] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0221] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0222] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0223] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0224] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0225] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0226] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0227] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0228] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0229] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0230] This invention relates to an educational support system for improving learners' ICT skills in educational institutions. The system includes a server, learner terminals, and an AI component with advanced data processing capabilities.

[0231] Course material generation and distribution

[0232] The server is programmed to access educational databases and regularly collect the latest educational information. Based on this collected information, the server automatically generates teaching materials that conform to the curriculum guidelines and current IT trends, and is designed to be appropriate for each grade level and level. These materials are customized according to specific educational needs and, after generation, are delivered to learners' devices.

[0233] Tracking and recording learning progress

[0234] The learner terminal provides learners with learning materials received through a user interface. As learners use the materials, the terminal tracks their progress in real time and records data to evaluate their level of understanding. This data is sent to a server to generate feedback tailored to each learner's level of understanding and pace.

[0235] Question answering function

[0236] Users can ask questions or express concerns about specific topics during their learning process. The server receives these questions, analyzes them using an AI program, and provides appropriate learning materials and answers. This process is fast and supports users in continuing their learning without interruption.

[0237] Data analysis and course material updates

[0238] The server analyzes collected learning data and provides insights to improve the quality of learning materials. Based on continuous data analysis, the learning materials are updated as needed, providing learners with the most effective and up-to-date educational content. This enables consistent IT education across educational institutions, aiming to reduce the workload of teachers while supporting the improvement of students' ICT skills.

[0239] This coordinated operation of the entire system will bring innovation to the educational setting and enable it to address the diverse learning needs of students.

[0240] The following describes the processing flow.

[0241] Step 1:

[0242] The server establishes a connection to the educational database and regularly collects the latest educational information and IT-related data. This ensures that the server is always up-to-date with the latest trends and educational standards.

[0243] Step 2:

[0244] The server analyzes the collected information and uses AI algorithms to automatically generate learning materials tailored to each grade level and educational standard. These materials consist of text, images, videos, and other formats, and are dynamically customized according to the learning content.

[0245] Step 3:

[0246] The server distributes the generated learning materials to the learner's device. The learner's device displays these materials to the student through its user interface and prepares to begin learning.

[0247] Step 4:

[0248] Users engage in learning activities using learning materials presented on their learning devices. The learning devices track learning progress in real time, evaluate understanding and results, and report them to the server.

[0249] Step 5:

[0250] The server receives questions from users and analyzes their content using AI. Based on the analyzed questions, it generates appropriate explanations and supplementary materials, and presents them to the user's terminal as needed.

[0251] Step 6:

[0252] The server analyzes learner progress data and accumulated learning data. This allows it to identify areas for improvement in the learning materials and gain insights that can be implemented in future lessons.

[0253] Step 7:

[0254] Based on the analysis results, the server adjusts and updates the content and structure of the teaching materials to prepare more effective educational content. This ensures improved learning effectiveness and consistency in education.

[0255] (Example 1)

[0256] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0257] Traditional educational support systems have problems with providing learners with effective and efficient learning materials, tracking their progress, and providing personalized feedback. Furthermore, they have challenges in responding quickly and accurately to learners' questions, and insufficient optimization of learning materials based on data.

[0258] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0259] In this invention, the server includes means for generating learning materials based on educational information, means for distributing the generated learning materials to terminals and recording learning progress, and means for analyzing inquiries from learners and providing relevant learning materials. This enables the provision of optimal learning materials tailored to each learner's progress and prompt and accurate question answering, thereby providing an overall efficient learning environment.

[0260] "Educational information" refers to all data and materials related to educational activities, including information related to curriculum guidelines and learner progress.

[0261] "Learning materials" are content created for learners to use for educational purposes and are provided in formats such as text, images, and videos.

[0262] "Terminal" refers to a device used by learners to receive and manipulate learning materials, and includes personal computers, tablets, and smartphones.

[0263] "Progress" refers to indicators or records that show how much progress learners have made in educational activities.

[0264] An "inquiry" refers to a question that a learner submits to the system seeking answers or information they want to resolve.

[0265] "Natural language" refers to the language that humans use in everyday life, and the system is required to understand questions and instructions by analyzing this language.

[0266] A "generative AI model" refers to a collection of programs and algorithms that use artificial intelligence to automatically create text and content.

[0267] "Optimizing educational materials" refers to the process of adjusting the content and structure of educational materials so that they are more effective and appropriate for learners.

[0268] This invention provides an educational support system that improves learners' ICT skills. The system consists of a server, learner terminals, and an AI component with advanced data processing capabilities.

[0269] The server connects to the database and uses Python scripts and SQL queries to collect educational information. The collected data is passed to an AI component, where a generative AI model automatically generates learning materials based on the curriculum guidelines. A general-purpose AI platform is used for this process. The generated materials are created in HTML format and adjusted according to specific educational needs.

[0270] The server delivers generated learning materials to the device using the HTTP protocol. The device displays the learning materials through an interface using React or similar technologies, allowing users to interactively learn from the received content. As learners use the materials, the device tracks user actions in real time using JavaScript and other technologies, recording their progress. This makes it possible to monitor learners' progress in detail.

[0271] Users can ask questions via voice or text if they encounter difficulties during their learning process. The server analyzes the received questions using natural language processing techniques and provides relevant learning materials and answers. This communication utilizes standard network connectivity technologies.

[0272] The server uses Python analysis tools to analyze learner progress and collected learning data. The analysis results are visualized using frameworks such as Django, and used to retrain the AI ​​and continuously improve the learning materials.

[0273] As a concrete example, a server might use the prompt "Mathematics materials for first-year high school students" to generate learning materials on "Fundamentals of Functions." These materials are delivered as HTML content combined with images and graphs for easy understanding. If a user asks, "I don't understand functions," the server immediately suggests reviewing the relevant section.

[0274] By coordinating the entire system in this way, it becomes possible to create an efficient and effective learning environment in educational settings.

[0275] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0276] Step 1:

[0277] The server accesses the education database and collects educational information. Using Python scripts and SQL queries, it collects the latest curriculum guidelines and teaching material information. This data is provided to the server as input and serves as the raw data for generating learning materials. Specifically, it retrieves data named "Mathematics Curriculum for First-Year High School Students" from the database. As output, it sends a set of educational information to the generation AI model.

[0278] Step 2:

[0279] The server automatically generates teaching materials using the generation AI model. As input, the educational information collected in Step 1 is used. The server provides this information to the AI component and analyzes it using natural language processing technology. As a result of data processing, HTML-formatted teaching materials suitable for the learner's level are output. Specifically, it generates learning materials related to "The Basics of Functions" in a format that incorporates figures and practice questions.

[0280] Step 3:

[0281] The server distributes the generated teaching materials to the terminal. As input, the HTML-formatted teaching materials created in Step 2 are used. The server sends this to the terminal via the HTTP protocol. As output, teaching materials that can be displayed on the learner's terminal are prepared. Specifically, when the user opens the teaching materials, they are interactively displayed on the browser of the terminal.

[0282] Step 4:

[0283] The terminal tracks the learner's progress through the user interface. As input, the learner's operation data is recorded on the terminal. Here, real-time data tracking is performed using JavaScript. The progress status and answer results are sent to the server as output. Specifically, each time the learner solves a problem, the result is logged, and the correct answer rate and response time are analyzed.

[0284] Step 5:

[0285] The user inputs questions into the terminal. These are sent as input to the server, and the server analyzes the questions using natural language processing technology. As a result, appropriate teaching material sections and answers are output and presented to the terminal again. As a specific operation, when the user asks "I don't understand functions", the relevant teaching material sections are immediately displayed.

[0286] Step 6:

[0287] The server continuously analyzes the learning data and updates the teaching materials. As input, the progress data collected in Step 4 is used. The data is analyzed using Python analysis tools and visualized with Django or the like, and as a result, a list of improvement plans is obtained. As a specific operation, when it is determined that a specific section of the learning course is not understood, the content of the teaching material is improved and redistributed.

[0288] (Application Example 1)

[0289] Next, Application Example 1 will be described. In the following description, the data processing device 12 is referred to as the "server", and the smart glasses 214 are referred to as the "terminal".

[0290] In the modern educational environment, there is a demand to provide education according to an individual's progress and understanding level and to efficiently realize the improvement of ICT skills. Also, there is a demand to realize an educational system that can quickly respond to learners' questions and provide educational feedback in real time. In conventional educational systems, it is difficult to optimize such individual responses, and there are limitations in promoting learning efficiently.

[0291] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0292] In this invention, the server includes means for connecting to educational information sources to collect the latest data and generate educational materials based on predetermined criteria; means for distributing the generated educational materials to learner devices and tracking and recording educational progress; and means for providing real-time educational feedback through eye-tracking or voice input. This enables the provision of up-to-date educational content tailored to individual learners, improves learning efficiency, and allows for immediate and responsive educational support.

[0293] "Educational information sources" are databases and online resources that provide data and materials related to learning.

[0294] "Educational materials" refer to a collection of information provided to learners for the purpose of learning, and include textbooks and reference materials.

[0295] "Learner equipment" refers to electronic devices used by learners to receive and utilize educational materials.

[0296] "Educational progress" is an indicator that shows the extent to which learners understand and have acquired specific educational objectives.

[0297] "Eye-tracking" is a technology that tracks the movement of a learner's eyes, and uses this data to manipulate interfaces or analyze situations.

[0298] "Voice input" is a method of inputting instructions or data into a device using voice.

[0299] "Educational feedback" refers to advice and suggestions provided based on a learner's learning progress, with the aim of improving their learning methods and understanding.

[0300] The system for realizing this application consists of a server, a learner terminal, smart glasses, and an AI component. The server periodically collects data from educational information sources and automatically generates educational materials based on predetermined criteria. The generated educational materials are delivered to learners via the learner terminal or smart glasses.

[0301] By using smart glasses, learners can have their gaze detected and be effectively guided through educational materials. Using voice input, users can ask questions, and the server provides immediate, relevant educational feedback. Furthermore, learner progress is recorded in real time, and the server generates individually tailored educational feedback based on this data.

[0302] The hardware used will consist of smart glasses, and Google's cloud-based speech conversion service will be used for voice recognition. The Django framework will run on a server for tracking progress and distributing materials, and TensorFlow will be used for AI analysis. As a concrete example, one can imagine a citizen wearing smart glasses in a public place and receiving ICT education while taking a walk. An example of a prompt would be: "A new citizen wearing smart glasses will view educational content while walking in a park and ask questions to an AI assistant in real time. Explain how it is using eye tracking and voice to learn."

[0303] In this way, the system implementing the invention can support the improvement of learners' ICT skills through the generation, distribution, progress tracking, and provision of real-time feedback of educational materials.

[0304] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0305] Step 1:

[0306] The server connects to an educational information source and collects the latest educational data. The inputs include information from external databases and online platforms through the Internet. Based on this data, data processing is performed to convert it into a format that conforms to predetermined educational standards. Thereby, it becomes possible to generate educational materials suitable for learners.

[0307] Step 2:

[0308] The server uses a generative AI model to create educational materials based on the collected data. The input is the educational data processed in Step 1, and by processing this data with the generative AI model, customized educational materials corresponding to each academic year and level are output. The output materials are saved as educational content and distributed in the next step.

[0309] Step 3:

[0310] The server distributes the educational materials to the learners' terminals or smart glasses. The input is the educational materials created in Step 2, and it is sent to the learners' devices. At that time, the data is transferred using a communication protocol to make the materials downloadable on the terminal side. The output is the learners' terminals with the educational materials downloaded.

[0311] Step 4:

[0312] The terminal presents the distributed educational materials through the interface function with the user. The input is the educational materials downloaded in Step 3, and the materials are provided to the user through a display device or voice output. Data calculation is not required, and the operation of presenting the data is the main operation.

[0313] Step 5:

[0314] The user learns while wearing smart glasses and asks questions using eye-tracking and voice input functions. The input consists of the user's voice and eye-tracking data. This data is collected by the device and sent to the server. The output is information on the user's learning progress and the content of the questions, which are processed in the next step.

[0315] Step 6:

[0316] The server analyzes the learning data and questions submitted by the user and provides relevant feedback in real time. The input is the user data collected in step 5, which is processed by the AI ​​analysis engine to generate optimal feedback and additional educational materials. The output is the feedback information returned to the learner.

[0317] Step 7:

[0318] The server continuously monitors learning data and tracks progress to optimize educational materials. The input is accumulated learning progress data, which is then analyzed to extract insights necessary for the next material update. The output is improvement suggestions that will be reflected in the next material update.

[0319] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0320] This invention provides an educational support system that incorporates an emotion engine for recognizing user emotions, thereby offering an educational experience tailored to the individual learner's needs. This system is constructed through the collaboration of a server, a learner terminal, and the emotion engine.

[0321] Emotion recognition and content adjustment

[0322] The device recognizes the learner's emotions through camera and microphone sensors and analyzes that data. For example, if a user shows signs of impatience during learning, the emotion engine analyzes this information and adjusts the difficulty level of the learning content or displays an encouraging message.

[0323] Learning progress and feedback

[0324] The server combines emotional data with learning progress to generate more personalized feedback. Based on emotional data, if the user is experiencing stress, it adjusts learning time or provides content to help them relax. This kind of feedback improves the user experience and enhances learning efficiency.

[0325] Analysis and improvement of the educational environment

[0326] The server collects and analyzes sentiment data from multiple users to evaluate the overall learning environment. Based on this information, it provides insights that help improve educational policies and course materials. For example, for specific topics that many users find difficult simultaneously, the course materials are re-evaluated and revised as needed.

[0327] Specific example

[0328] If the system detects that the user is unfamiliar with the development environment and shows signs of anxiety, it will provide a simple user guide and recommend video learning materials to aid understanding. Furthermore, if the system determines that the user's understanding is low, it will provide links to more detailed materials and additional practice problems on the learner's device.

[0329] Systems with these functions aim to maximize learning efficiency and provide a comfortable learning environment by responding appropriately in real time according to the individual circumstances of each learner. In this way, the present invention proposes a new approach to enhancing the learning experience by combining emotion recognition technology and educational content.

[0330] The following describes the processing flow.

[0331] Step 1:

[0332] The user logs into the learner's device and accesses the designated educational content. The device uses its camera and microphone to capture the user's facial expressions and voice tone in real time.

[0333] Step 2:

[0334] The device sends the captured audio and video data to the emotion engine. The emotion engine analyzes this data to identify the user's emotional state (e.g., concentration, joy, stress).

[0335] Step 3:

[0336] The server receives emotion data from the emotion engine and integrates it with the user's current learning progress data. Based on this integrated data, it considers customizing the learning content and feedback.

[0337] Step 4:

[0338] The device provides the user with appropriate feedback and tailored learning content based on instructions received from the server. For example, if the user is detected as being in a state of agitation, a message prompting them to take a break will be displayed.

[0339] Step 5:

[0340] As the user continues to learn, emotion and progress data are continuously collected and sent to the server by the device. A real-time feedback loop is formed.

[0341] Step 6:

[0342] The server analyzes long-term sentiment data to generate insights for improving the quality of educational content. This information is used to revise teaching materials and review educational policies.

[0343] Step 7:

[0344] Throughout the entire system, the terminal and server continuously work together to enable flexible responses tailored to the user's emotions and learning progress. This provides the optimal educational environment for learners.

[0345] (Example 2)

[0346] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0347] Traditional educational support systems often fail to adequately address the individual needs of learners because they provide standardized learning content and progress management without considering the learner's emotional state. In particular, there is a lack of feedback that responds to the stress and changes in understanding that learners experience, making it difficult to maximize learning efficiency.

[0348] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0349] In this invention, the server includes means for collecting and analyzing video and audio data to identify the user's emotional state, means for adjusting the difficulty level of learning materials or presenting supplementary information based on the analyzed emotional state, and means for combining the emotional data with the learning progress to generate individualized feedback for the learner. This enables the provision of an educational experience tailored to the individual emotional state of the learner, thereby improving learning efficiency.

[0350] "User emotional state" refers to the psychological state exhibited by users during learning, such as joy, impatience, and stress, and is identified through video and audio data.

[0351] "Video and audio data" refers to digital data including the user's captured facial expressions and recorded voice, which is used to analyze their emotional state.

[0352] "Means of analysis" refers to technical methods and devices used to identify and evaluate emotional states from collected video and audio data.

[0353] "Adjusting the difficulty level of learning materials" refers to the process of changing the content and difficulty level of learning materials according to the user's emotional state.

[0354] "Supplementary information" refers to additional learning materials and support information provided to assist users in their learning, with the aim of improving learning efficiency.

[0355] "Individualized feedback" refers to specialized advice and instructions provided based on each learner's emotional state and learning progress.

[0356] "Learner progress" refers to a standard used to measure the degree of progress a learner has achieved during the educational process and to evaluate their growth based on that progress.

[0357] "Educational experience" is a collective term for the series of learning processes that learners acquire through an educational system, and the emotional and intellectual responses that accompany them.

[0358] This invention provides an educational support system that recognizes the emotional state of learners and adjusts the educational experience based on that state. This system is realized through the cooperation of a learner terminal, a server, and an emotion engine.

[0359] Terminal role

[0360] The device collects video and audio data from learners through sensors. Specifically, it records the learner's facial expressions with a camera and acquires audio with a microphone. This data is preprocessed using image processing libraries and speech analysis tools (e.g., OpenCV or Google Cloud Speech-to-Text).

[0361] Emotion recognition and analysis

[0362] The emotion engine analyzes data received from the device and uses machine learning models (e.g., TensorFlow or PyTorch-based models) to identify the learner's emotional state. This process makes it possible to recognize the learner's emotions in real time.

[0363] Server Role

[0364] Based on analyzed sentiment data, the server adjusts the difficulty level of learning materials to provide personalized educational content for each learner. It also provides supplementary materials as needed and generates individualized feedback. Furthermore, the server aggregates sentiment data from multiple users to gain insights that guide improvements to the overall learning environment.

[0365] Examples of specific cases and prompt statements

[0366] For example, if a user is feeling anxious about using a new programming environment, the device will detect their anxiety and the server will provide a simple user guide or instructional video. Also, if a user is stressed about a particular topic, the server will suggest relaxing content.

[0367] An example of a prompt for a generative AI model is: "Design a system that provides appropriate learning materials and feedback based on emotions, so that students can learn the fundamentals of programming effectively."

[0368] This system is designed to ensure that all learners have the optimal learning experience, and it provides personalized learning support using emotion recognition technology.

[0369] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0370] Step 1:

[0371] The device collects video and audio data from learners using sensors. It takes facial expressions captured by the camera and audio recorded by the microphone as input, and preprocesses this data using image processing libraries and audio analysis tools. This provides foundational data for analyzing the learners' emotional states.

[0372] Step 2:

[0373] The device sends pre-processed data to the emotion engine, which analyzes the learner's emotional state. The input data is analyzed using a machine learning model, and the emotional state is determined as output. Specifically, the model assigns emotional labels such as joy, anxiety, and stress.

[0374] Step 3:

[0375] The server receives the analyzed emotional state data and adjusts the learning content. Based on the input emotional data, it decides whether to adjust the difficulty level of the learning materials or provide supplementary information. For example, it might generate materials with a lower difficulty level or video links to aid understanding.

[0376] Step 4:

[0377] The server integrates emotional data and learning progress data to generate personalized feedback. Based on the input data, it calculates what kind of feedback is most effective and presents the user with specialized advice and suggestions for breaks as output.

[0378] Step 5:

[0379] The server aggregates emotional data collected from multiple learners to gain insights into the overall educational environment. Based on the accumulated data, suggestions for improving educational policies and materials are made, and reports and plans for optimizing the educational environment are presented as output.

[0380] (Application Example 2)

[0381] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0382] In today's learning environment, a challenge exists in that learners often struggle to receive education that is tailored to their individual emotional states. Traditional education systems respond to learning progress but rarely directly consider learners' emotions, thus limiting effective individualized instruction. Furthermore, when students experience stress or anxiety during learning, they often do not receive appropriate support, which leads to decreased learning efficiency.

[0383] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0384] In this invention, the server includes means for connecting to an educational information repository to collect the latest information and generating teaching materials based on predetermined standards; means for distributing the generated teaching materials to student terminals and tracking and recording learning progress; and means for identifying the emotions of students and providing learning support in accordance with those emotions. This makes it possible to provide immediate individualized instruction that takes into account the emotional state of students, thereby improving the effectiveness of learning.

[0385] The "Educational Information Library" is a data storage system that stores and makes accessible the latest data on learning materials, subject matter materials, and educational guidance provided to students.

[0386] "Student devices" refer to electronic devices used by learners to access educational content and progress in their studies, and include tablets, personal computers, and smartphones.

[0387] "Means for identifying emotions" refers to functions that include technical elements for analyzing the participant's facial expressions, tone of voice, etc., to determine the participant's emotional state.

[0388] "Means of providing learning support" refers to functions that provide adjustments to learning content, advice, and encouragement according to the learner's progress and emotions.

[0389] "Means for tracking and recording learning progress" refers to systems and functions for monitoring how far a student has progressed in their ongoing learning and recording that data.

[0390] The system that realizes this invention includes an educational information repository, student terminals, and an emotion recognition engine, each of which functions in cooperation with one another.

[0391] The server connects to the educational information repository and collects and updates the latest learning content to provide to students. This content is automatically generated based on pre-configured educational standards and is designed to meet diverse learning needs.

[0392] The learner's device collects the learner's facial expressions and voice in real time using a camera and microphone, and transmits this data to an emotion recognition engine. The emotion recognition engine incorporates commercially available technological elements, which analyze the learner's emotional state. If the learner shows signs of stress or anxiety, the device can automatically adjust the difficulty level of the content or display encouraging messages.

[0393] Furthermore, the server combines learning progress and sentiment data to generate feedback tailored to each learner's individual situation. As a result, learners can continue learning at a pace that is optimal for them. The data in the educational information repository is regularly updated, providing highly accurate feedback that accommodates various learning patterns.

[0394] As a concrete example, suppose a student is working on a new mathematical proposition and a confused expression is detected. In this case, the system analyzes the expression and immediately displays links to relevant video tutorials and practice problems to aid understanding.

[0395] An example of a prompt would be: "Please tell me how to adjust the learning content based on the emotions the learner is experiencing while working on a particular problem. Currently, the learner is not concentrating."

[0396] This system can significantly improve the quality of education by combining emotion recognition with personalized learning support.

[0397] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0398] Step 1:

[0399] The device captures the participant's facial expressions and voice using a camera and microphone. Real-time video and audio data are obtained as input. This data is sent to an emotion recognition engine, where it is converted into emotional states using facial recognition algorithms and voice analysis. The output is digital data indicating the participant's emotional state.

[0400] Step 2:

[0401] The server receives emotional state data and combines it with the learner's learning progress data. In this step, an adaptive learning algorithm is used to determine which learning content is appropriate based on the learner's current learning status and emotional state. The input is numerical data of learning progress and emotional state data, and the output is a list of adjusted learning content.

[0402] Step 3:

[0403] The device presents the learner with customized learning content. Specifically, video tutorials, supplementary materials, or new practice exercises are displayed on the learner's device. The input is a list of learning content received from the server, and the output is the learning material displayed on the learner's screen.

[0404] Step 4:

[0405] The user (learner) engages in learning activities based on the information presented. If a new question arises during this process, the device analyzes the question and presents additional relevant learning materials. The input is the user's question, and the output is a list of relevant learning materials. A question analysis algorithm uses a generative AI model to create prompts to identify appropriate learning materials.

[0406] Step 5:

[0407] The server accumulates learning history and sentiment data, and generates statistical data for future system improvements. This data serves as a starting point for developing new algorithms and contributes to the continuous optimization of educational content. The input is past learning session data, and the output is statistical data and improvement suggestions.

[0408] This will enable the entire system to provide a flexible learning experience tailored to the specific needs of each student.

[0409] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0410] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0411] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0412] [Third Embodiment]

[0413] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0414] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0415] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0416] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0417] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0418] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0419] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0420] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0421] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0422] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0423] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0424] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0425] This invention relates to an educational support system for improving learners' ICT skills in educational institutions. The system includes a server, learner terminals, and an AI component with advanced data processing capabilities.

[0426] Course material generation and distribution

[0427] The server is programmed to access educational databases and regularly collect the latest educational information. Based on this collected information, the server automatically generates teaching materials that conform to the curriculum guidelines and current IT trends, and is designed to be appropriate for each grade level and level. These materials are customized according to specific educational needs and, after generation, are delivered to learners' devices.

[0428] Tracking and recording learning progress

[0429] The learner terminal provides learners with learning materials received through a user interface. As learners use the materials, the terminal tracks their progress in real time and records data to evaluate their level of understanding. This data is sent to a server to generate feedback tailored to each learner's level of understanding and pace.

[0430] Question answering function

[0431] Users can ask questions or express concerns about specific topics during their learning process. The server receives these questions, analyzes them using an AI program, and provides appropriate learning materials and answers. This process is fast and supports users in continuing their learning without interruption.

[0432] Data analysis and course material updates

[0433] The server analyzes collected learning data and provides insights to improve the quality of learning materials. Based on continuous data analysis, the learning materials are updated as needed, providing learners with the most effective and up-to-date educational content. This enables consistent IT education across educational institutions, aiming to reduce the workload of teachers while supporting the improvement of students' ICT skills.

[0434] This coordinated operation of the entire system will bring innovation to the educational setting and enable it to address the diverse learning needs of students.

[0435] The following describes the processing flow.

[0436] Step 1:

[0437] The server establishes a connection to the educational database and regularly collects the latest educational information and IT-related data. This ensures that the server is always up-to-date with the latest trends and educational standards.

[0438] Step 2:

[0439] The server analyzes the collected information and uses AI algorithms to automatically generate learning materials tailored to each grade level and educational standard. These materials consist of text, images, videos, and other formats, and are dynamically customized according to the learning content.

[0440] Step 3:

[0441] The server distributes the generated learning materials to the learner's device. The learner's device displays these materials to the student through its user interface and prepares to begin learning.

[0442] Step 4:

[0443] Users engage in learning activities using learning materials presented on their learning devices. The learning devices track learning progress in real time, evaluate understanding and results, and report them to the server.

[0444] Step 5:

[0445] The server receives questions from users and analyzes their content using AI. Based on the analyzed questions, it generates appropriate explanations and supplementary materials, and presents them to the user's terminal as needed.

[0446] Step 6:

[0447] The server analyzes learner progress data and accumulated learning data. This allows it to identify areas for improvement in the learning materials and gain insights that can be implemented in future lessons.

[0448] Step 7:

[0449] Based on the analysis results, the server adjusts and updates the content and structure of the teaching materials to prepare more effective educational content. This ensures improved learning effectiveness and consistency in education.

[0450] (Example 1)

[0451] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0452] Traditional educational support systems have problems with providing learners with effective and efficient learning materials, tracking their progress, and providing personalized feedback. Furthermore, they have challenges in responding quickly and accurately to learners' questions, and insufficient optimization of learning materials based on data.

[0453] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0454] In this invention, the server includes means for generating learning materials based on educational information, means for distributing the generated learning materials to terminals and recording learning progress, and means for analyzing inquiries from learners and providing relevant learning materials. This enables the provision of optimal learning materials tailored to each learner's progress and prompt and accurate question answering, thereby providing an overall efficient learning environment.

[0455] "Educational information" refers to all data and materials related to educational activities, including information related to curriculum guidelines and learner progress.

[0456] "Learning materials" are content created for learners to use for educational purposes and are provided in formats such as text, images, and videos.

[0457] "Terminal" refers to a device used by learners to receive and manipulate learning materials, and includes personal computers, tablets, and smartphones.

[0458] "Progress" refers to indicators or records that show how much progress learners have made in educational activities.

[0459] An "inquiry" refers to a question that a learner submits to the system seeking answers or information they want to resolve.

[0460] "Natural language" refers to the language that humans use in everyday life, and the system is required to understand questions and instructions by analyzing this language.

[0461] A "generative AI model" refers to a collection of programs and algorithms that use artificial intelligence to automatically create text and content.

[0462] "Optimizing educational materials" refers to the process of adjusting the content and structure of educational materials so that they are more effective and appropriate for learners.

[0463] This invention provides an educational support system that improves learners' ICT skills. The system consists of a server, learner terminals, and an AI component with advanced data processing capabilities.

[0464] The server connects to the database and uses Python scripts and SQL queries to collect educational information. The collected data is passed to an AI component, where a generative AI model automatically generates learning materials based on the curriculum guidelines. A general-purpose AI platform is used for this process. The generated materials are created in HTML format and adjusted according to specific educational needs.

[0465] The server delivers generated learning materials to the device using the HTTP protocol. The device displays the learning materials through an interface using React or similar technologies, allowing users to interactively learn from the received content. As learners use the materials, the device tracks user actions in real time using JavaScript and other technologies, recording their progress. This makes it possible to monitor learners' progress in detail.

[0466] Users can ask questions via voice or text if they encounter difficulties during their learning process. The server analyzes the received questions using natural language processing techniques and provides relevant learning materials and answers. This communication utilizes standard network connectivity technologies.

[0467] The server uses Python analysis tools to analyze learner progress and collected learning data. The analysis results are visualized using frameworks such as Django, and used to retrain the AI ​​and continuously improve the learning materials.

[0468] As a concrete example, a server might use the prompt "Mathematics materials for first-year high school students" to generate learning materials on "Fundamentals of Functions." These materials are delivered as HTML content combined with images and graphs for easy understanding. If a user asks, "I don't understand functions," the server immediately suggests reviewing the relevant section.

[0469] By coordinating the entire system in this way, it becomes possible to create an efficient and effective learning environment in educational settings.

[0470] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0471] Step 1:

[0472] The server accesses an educational database and collects educational information. It uses Python scripts and SQL queries to gather the latest curriculum guidelines and teaching materials. This data is provided to the server as input and serves as the source data for generating learning materials. Specifically, it retrieves data such as "Mathematics Curriculum for First-Year High School Students" from the database. As output, it sends a set of educational information to the AI ​​model for generating learning materials.

[0473] Step 2:

[0474] The server automatically generates learning materials using a generative AI model. The educational information collected in Step 1 is used as input. The server provides this information to the AI ​​component, which analyzes it using natural language processing techniques. As a result of the data processing, HTML-formatted learning materials tailored to the learner's level are output. Specifically, it generates learning materials on "Fundamentals of Functions," incorporating diagrams and practice problems.

[0475] Step 3:

[0476] The server delivers the generated learning materials to the terminal. The HTML-formatted learning materials created in step 2 are used as input. The server sends these to the terminal via the HTTP protocol. As output, learning materials that can be displayed on the learner's terminal are prepared. Specifically, when the user opens the learning materials, they are displayed interactively in the terminal's browser.

[0477] Step 4:

[0478] The terminal tracks the learner's progress through a user interface. The learner's operation data is recorded as input on the terminal. Real-time data tracking is performed using JavaScript. Progress and answer results are sent to the server as output. Specifically, each time the learner solves a problem, the result is logged, and the accuracy rate and response time are analyzed.

[0479] Step 5:

[0480] The user enters their question into the terminal. This is sent to the server as input, and the server analyzes the question using natural language processing technology. As a result, the appropriate learning material or answer is output and presented back to the terminal. For example, if the user asks "I don't understand this function," the relevant learning material section is immediately displayed.

[0481] Step 6:

[0482] The server continuously analyzes learning data and updates the learning materials. Progress data collected in step 4 is used as input. By analyzing the data using Python analysis tools and visualizing it with Django or similar tools, a list of improvement suggestions is obtained as output. Specifically, if a particular section of a learning course is deemed incomprehensible, the content of that material is improved and redistributed.

[0483] (Application Example 1)

[0484] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0485] In today's educational environment, there is a need to provide education tailored to individual progress and understanding, and to efficiently improve ICT skills. Furthermore, there is a demand for educational systems that can quickly respond to learners' questions and provide real-time educational feedback. Traditional educational systems struggle to optimize this individualized approach, limiting their ability to efficiently facilitate learning.

[0486] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0487] In this invention, the server includes means for connecting to educational information sources to collect the latest data and generate educational materials based on predetermined criteria; means for distributing the generated educational materials to learner devices and tracking and recording educational progress; and means for providing real-time educational feedback through eye-tracking or voice input. This enables the provision of up-to-date educational content tailored to individual learners, improves learning efficiency, and allows for immediate and responsive educational support.

[0488] "Educational information sources" are databases and online resources that provide data and materials related to learning.

[0489] "Educational materials" refer to a collection of information provided to learners for the purpose of learning, and include textbooks and reference materials.

[0490] "Learner equipment" refers to electronic devices used by learners to receive and utilize educational materials.

[0491] "Educational progress" is an indicator that shows the extent to which learners understand and have acquired specific educational objectives.

[0492] "Eye-tracking" is a technology that tracks the movement of a learner's eyes, and uses this data to manipulate interfaces or analyze situations.

[0493] "Voice input" is a method of inputting instructions or data into a device using voice.

[0494] "Educational feedback" refers to advice and suggestions provided based on a learner's learning progress, with the aim of improving their learning methods and understanding.

[0495] The system for realizing this application consists of a server, a learner terminal, smart glasses, and an AI component. The server periodically collects data from educational information sources and automatically generates educational materials based on predetermined criteria. The generated educational materials are delivered to learners via the learner terminal or smart glasses.

[0496] By using smart glasses, learners can have their gaze detected and be effectively guided through educational materials. Using voice input, users can ask questions, and the server provides immediate, relevant educational feedback. Furthermore, learner progress is recorded in real time, and the server generates individually tailored educational feedback based on this data.

[0497] The hardware used will consist of smart glasses, and Google's cloud-based speech conversion service will be used for voice recognition. The Django framework will run on a server for tracking progress and distributing materials, and TensorFlow will be used for AI analysis. As a concrete example, one can imagine a citizen wearing smart glasses in a public place and receiving ICT education while taking a walk. An example of a prompt would be: "A new citizen wearing smart glasses will view educational content while walking in a park and ask questions to an AI assistant in real time. Explain how it is using eye tracking and voice to learn."

[0498] In this way, the system implementing the invention can support the improvement of learners' ICT skills through the generation, distribution, progress tracking, and provision of real-time feedback of educational materials.

[0499] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0500] Step 1:

[0501] The server connects to educational information sources and collects the latest educational data. Input includes information from external databases and online platforms via the internet. This data is then processed and converted into a format that conforms to specified educational standards. This enables the generation of educational materials tailored to learners.

[0502] Step 2:

[0503] The server uses a generative AI model to create educational materials based on the collected data. The input is the educational data processed in Step 1, and by processing this data with the generative AI model, it outputs customized educational materials tailored to each grade level and level. The output materials are saved as educational content and distributed in the next step.

[0504] Step 3:

[0505] The server delivers educational materials to the learner's device or smart glasses. The input is the educational material created in step 2, which is then sent to the learner's device. During this process, a communication protocol is used to transfer the data, making the material available for download on the device. The output is the learner's device with the educational material downloaded.

[0506] Step 4:

[0507] The terminal presents the delivered educational materials through its user interface function. The input is the educational materials downloaded in step 3, and these materials are provided to the user via a display device or audio output. No data calculation is required; the primary operation is the presentation of data.

[0508] Step 5:

[0509] The user learns while wearing smart glasses and asks questions using eye-tracking and voice input functions. The input consists of the user's voice and eye-tracking data. This data is collected by the device and sent to the server. The output is information on the user's learning progress and the content of the questions, which are processed in the next step.

[0510] Step 6:

[0511] The server analyzes the learning data and questions submitted by the user and provides relevant feedback in real time. The input is the user data collected in step 5, which is processed by the AI ​​analysis engine to generate optimal feedback and additional educational materials. The output is the feedback information returned to the learner.

[0512] Step 7:

[0513] The server continuously monitors learning data and tracks progress to optimize educational materials. The input is accumulated learning progress data, which is then analyzed to extract insights necessary for the next material update. The output is improvement suggestions that will be reflected in the next material update.

[0514] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0515] This invention provides an educational support system that incorporates an emotion engine for recognizing user emotions, thereby offering an educational experience tailored to the individual learner's needs. This system is constructed through the collaboration of a server, a learner terminal, and the emotion engine.

[0516] Emotion recognition and content adjustment

[0517] The device recognizes the learner's emotions through camera and microphone sensors and analyzes that data. For example, if a user shows signs of impatience during learning, the emotion engine analyzes this information and adjusts the difficulty level of the learning content or displays an encouraging message.

[0518] Learning progress and feedback

[0519] The server combines emotional data with learning progress to generate more personalized feedback. Based on emotional data, if the user is experiencing stress, it adjusts learning time or provides content to help them relax. This kind of feedback improves the user experience and enhances learning efficiency.

[0520] Analysis and improvement of the educational environment

[0521] The server collects and analyzes sentiment data from multiple users to evaluate the overall learning environment. Based on this information, it provides insights that help improve educational policies and course materials. For example, for specific topics that many users find difficult simultaneously, the course materials are re-evaluated and revised as needed.

[0522] Specific example

[0523] If the system detects that the user is unfamiliar with the development environment and shows signs of anxiety, it will provide a simple user guide and recommend video learning materials to aid understanding. Furthermore, if the system determines that the user's understanding is low, it will provide links to more detailed materials and additional practice problems on the learner's device.

[0524] Systems with these functions aim to maximize learning efficiency and provide a comfortable learning environment by responding appropriately in real time according to the individual circumstances of each learner. In this way, the present invention proposes a new approach to enhancing the learning experience by combining emotion recognition technology and educational content.

[0525] The following describes the processing flow.

[0526] Step 1:

[0527] The user logs into the learner's device and accesses the designated educational content. The device uses its camera and microphone to capture the user's facial expressions and voice tone in real time.

[0528] Step 2:

[0529] The device sends the captured audio and video data to the emotion engine. The emotion engine analyzes this data to identify the user's emotional state (e.g., concentration, joy, stress).

[0530] Step 3:

[0531] The server receives emotion data from the emotion engine and integrates it with the user's current learning progress data. Based on this integrated data, it considers customizing the learning content and feedback.

[0532] Step 4:

[0533] The device provides the user with appropriate feedback and tailored learning content based on instructions received from the server. For example, if the user is detected as being in a state of agitation, a message prompting them to take a break will be displayed.

[0534] Step 5:

[0535] As the user continues to learn, emotion and progress data are continuously collected and sent to the server by the device. A real-time feedback loop is formed.

[0536] Step 6:

[0537] The server analyzes long-term sentiment data to generate insights for improving the quality of educational content. This information is used to revise teaching materials and review educational policies.

[0538] Step 7:

[0539] Throughout the entire system, the terminal and server continuously work together to enable flexible responses tailored to the user's emotions and learning progress. This provides the optimal educational environment for learners.

[0540] (Example 2)

[0541] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0542] Traditional educational support systems often fail to adequately address the individual needs of learners because they provide standardized learning content and progress management without considering the learner's emotional state. In particular, there is a lack of feedback that responds to the stress and changes in understanding that learners experience, making it difficult to maximize learning efficiency.

[0543] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0544] In this invention, the server includes means for collecting and analyzing video and audio data to identify the user's emotional state, means for adjusting the difficulty level of learning materials or presenting supplementary information based on the analyzed emotional state, and means for combining the emotional data with the learning progress to generate individualized feedback for the learner. This enables the provision of an educational experience tailored to the individual emotional state of the learner, thereby improving learning efficiency.

[0545] "User emotional state" refers to the psychological state exhibited by users during learning, such as joy, impatience, and stress, and is identified through video and audio data.

[0546] "Video and audio data" refers to digital data including the user's captured facial expressions and recorded voice, which is used to analyze their emotional state.

[0547] "Means of analysis" refers to technical methods and devices used to identify and evaluate emotional states from collected video and audio data.

[0548] "Adjusting the difficulty level of learning materials" refers to the process of changing the content and difficulty level of learning materials according to the user's emotional state.

[0549] "Supplementary information" refers to additional learning materials and support information provided to assist users in their learning, with the aim of improving learning efficiency.

[0550] "Individualized feedback" refers to specialized advice and instructions provided based on each learner's emotional state and learning progress.

[0551] "Learner progress" refers to a standard used to measure the degree of progress a learner has achieved during the educational process and to evaluate their growth based on that progress.

[0552] "Educational experience" is a collective term for the series of learning processes that learners acquire through an educational system, and the emotional and intellectual responses that accompany them.

[0553] This invention provides an educational support system that recognizes the emotional state of learners and adjusts the educational experience based on that state. This system is realized through the cooperation of a learner terminal, a server, and an emotion engine.

[0554] Terminal role

[0555] The device collects video and audio data from learners through sensors. Specifically, it records the learner's facial expressions with a camera and acquires audio with a microphone. This data is preprocessed using image processing libraries and speech analysis tools (e.g., OpenCV or Google Cloud Speech-to-Text).

[0556] Emotion recognition and analysis

[0557] The emotion engine analyzes data received from the device and uses machine learning models (e.g., TensorFlow or PyTorch-based models) to identify the learner's emotional state. This process makes it possible to recognize the learner's emotions in real time.

[0558] Server Role

[0559] Based on analyzed sentiment data, the server adjusts the difficulty level of learning materials to provide personalized educational content for each learner. It also provides supplementary materials as needed and generates individualized feedback. Furthermore, the server aggregates sentiment data from multiple users to gain insights that guide improvements to the overall learning environment.

[0560] Examples of specific cases and prompt statements

[0561] For example, if a user is feeling anxious about using a new programming environment, the device will detect their anxiety and the server will provide a simple user guide or instructional video. Also, if a user is stressed about a particular topic, the server will suggest relaxing content.

[0562] An example of a prompt for a generative AI model is: "Design a system that provides appropriate learning materials and feedback based on emotions, so that students can learn the fundamentals of programming effectively."

[0563] This system is designed to ensure that all learners have the optimal learning experience, and it provides personalized learning support using emotion recognition technology.

[0564] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0565] Step 1:

[0566] The device collects video and audio data from learners using sensors. It takes facial expressions captured by the camera and audio recorded by the microphone as input, and preprocesses this data using image processing libraries and audio analysis tools. This provides foundational data for analyzing the learners' emotional states.

[0567] Step 2:

[0568] The device sends pre-processed data to the emotion engine, which analyzes the learner's emotional state. The input data is analyzed using a machine learning model, and the emotional state is determined as output. Specifically, the model assigns emotional labels such as joy, anxiety, and stress.

[0569] Step 3:

[0570] The server receives the analyzed emotional state data and adjusts the learning content. Based on the input emotional data, it decides whether to adjust the difficulty level of the learning materials or provide supplementary information. For example, it might generate materials with a lower difficulty level or video links to aid understanding.

[0571] Step 4:

[0572] The server integrates emotional data and learning progress data to generate personalized feedback. Based on the input data, it calculates what kind of feedback is most effective and presents the user with specialized advice and suggestions for breaks as output.

[0573] Step 5:

[0574] The server aggregates emotional data collected from multiple learners to gain insights into the overall educational environment. Based on the accumulated data, suggestions for improving educational policies and materials are made, and reports and plans for optimizing the educational environment are presented as output.

[0575] (Application Example 2)

[0576] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0577] In today's learning environment, a challenge exists in that learners often struggle to receive education that is tailored to their individual emotional states. Traditional education systems respond to learning progress but rarely directly consider learners' emotions, thus limiting effective individualized instruction. Furthermore, when students experience stress or anxiety during learning, they often do not receive appropriate support, which leads to decreased learning efficiency.

[0578] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0579] In this invention, the server includes means for connecting to an educational information repository to collect the latest information and generating teaching materials based on predetermined standards; means for distributing the generated teaching materials to student terminals and tracking and recording learning progress; and means for identifying the emotions of students and providing learning support in accordance with those emotions. This makes it possible to provide immediate individualized instruction that takes into account the emotional state of students, thereby improving the effectiveness of learning.

[0580] The "Educational Information Library" is a data storage system that stores and makes accessible the latest data on learning materials, subject matter materials, and educational guidance provided to students.

[0581] "Student devices" refer to electronic devices used by learners to access educational content and progress in their studies, and include tablets, personal computers, and smartphones.

[0582] "Means for identifying emotions" refers to functions that include technical elements for analyzing the participant's facial expressions, tone of voice, etc., to determine the participant's emotional state.

[0583] "Means of providing learning support" refers to functions that provide adjustments to learning content, advice, and encouragement according to the learner's progress and emotions.

[0584] "Means for tracking and recording learning progress" refers to systems and functions for monitoring how far a student has progressed in their ongoing learning and recording that data.

[0585] The system that realizes this invention includes an educational information repository, student terminals, and an emotion recognition engine, each of which functions in cooperation with one another.

[0586] The server connects to the educational information repository and collects and updates the latest learning content to provide to students. This content is automatically generated based on pre-configured educational standards and is designed to meet diverse learning needs.

[0587] The learner's device collects the learner's facial expressions and voice in real time using a camera and microphone, and transmits this data to an emotion recognition engine. The emotion recognition engine incorporates commercially available technological elements, which analyze the learner's emotional state. If the learner shows signs of stress or anxiety, the device can automatically adjust the difficulty level of the content or display encouraging messages.

[0588] Furthermore, the server combines learning progress and sentiment data to generate feedback tailored to each learner's individual situation. As a result, learners can continue learning at a pace that is optimal for them. The data in the educational information repository is regularly updated, providing highly accurate feedback that accommodates various learning patterns.

[0589] As a concrete example, suppose a student is working on a new mathematical proposition and a confused expression is detected. In this case, the system analyzes the expression and immediately displays links to relevant video tutorials and practice problems to aid understanding.

[0590] An example of a prompt would be: "Please tell me how to adjust the learning content based on the emotions the learner is experiencing while working on a particular problem. Currently, the learner is not concentrating."

[0591] This system can significantly improve the quality of education by combining emotion recognition with personalized learning support.

[0592] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0593] Step 1:

[0594] The device captures the participant's facial expressions and voice using a camera and microphone. Real-time video and audio data are obtained as input. This data is sent to an emotion recognition engine, where it is converted into emotional states using facial recognition algorithms and voice analysis. The output is digital data indicating the participant's emotional state.

[0595] Step 2:

[0596] The server receives emotional state data and combines it with the learner's learning progress data. In this step, an adaptive learning algorithm is used to determine which learning content is appropriate based on the learner's current learning status and emotional state. The input is numerical data of learning progress and emotional state data, and the output is a list of adjusted learning content.

[0597] Step 3:

[0598] The device presents the learner with customized learning content. Specifically, video tutorials, supplementary materials, or new practice exercises are displayed on the learner's device. The input is a list of learning content received from the server, and the output is the learning material displayed on the learner's screen.

[0599] Step 4:

[0600] The user (learner) engages in learning activities based on the information presented. If a new question arises during this process, the device analyzes the question and presents additional relevant learning materials. The input is the user's question, and the output is a list of relevant learning materials. A question analysis algorithm uses a generative AI model to create prompts to identify appropriate learning materials.

[0601] Step 5:

[0602] The server accumulates learning history and sentiment data, and generates statistical data for future system improvements. This data serves as a starting point for developing new algorithms and contributes to the continuous optimization of educational content. The input is past learning session data, and the output is statistical data and improvement suggestions.

[0603] This will enable the entire system to provide a flexible learning experience tailored to the specific needs of each student.

[0604] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0605] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0606] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0607] [Fourth Embodiment]

[0608] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0609] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0610] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0611] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0612] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0613] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0614] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0615] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0616] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0617] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0618] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0619] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0620] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0621] This invention relates to an educational support system for improving learners' ICT skills in educational institutions. The system includes a server, learner terminals, and an AI component with advanced data processing capabilities.

[0622] Course material generation and distribution

[0623] The server is programmed to access educational databases and regularly collect the latest educational information. Based on this collected information, the server automatically generates teaching materials that conform to the curriculum guidelines and current IT trends, and is designed to be appropriate for each grade level and level. These materials are customized according to specific educational needs and, after generation, are delivered to learners' devices.

[0624] Tracking and recording learning progress

[0625] The learner terminal provides learners with learning materials received through a user interface. As learners use the materials, the terminal tracks their progress in real time and records data to evaluate their level of understanding. This data is sent to a server to generate feedback tailored to each learner's level of understanding and pace.

[0626] Question answering function

[0627] Users can ask questions or express concerns about specific topics during their learning process. The server receives these questions, analyzes them using an AI program, and provides appropriate learning materials and answers. This process is fast and supports users in continuing their learning without interruption.

[0628] Data analysis and course material updates

[0629] The server analyzes collected learning data and provides insights to improve the quality of learning materials. Based on continuous data analysis, the learning materials are updated as needed, providing learners with the most effective and up-to-date educational content. This enables consistent IT education across educational institutions, aiming to reduce the workload of teachers while supporting the improvement of students' ICT skills.

[0630] This coordinated operation of the entire system will bring innovation to the educational setting and enable it to address the diverse learning needs of students.

[0631] The following describes the processing flow.

[0632] Step 1:

[0633] The server establishes a connection to the educational database and regularly collects the latest educational information and IT-related data. This ensures that the server is always up-to-date with the latest trends and educational standards.

[0634] Step 2:

[0635] The server analyzes the collected information and uses AI algorithms to automatically generate learning materials tailored to each grade level and educational standard. These materials consist of text, images, videos, and other formats, and are dynamically customized according to the learning content.

[0636] Step 3:

[0637] The server distributes the generated learning materials to the learner's device. The learner's device displays these materials to the student through its user interface and prepares to begin learning.

[0638] Step 4:

[0639] Users engage in learning activities using learning materials presented on their learning devices. The learning devices track learning progress in real time, evaluate understanding and results, and report them to the server.

[0640] Step 5:

[0641] The server receives questions from users and analyzes their content using AI. Based on the analyzed questions, it generates appropriate explanations and supplementary materials, and presents them to the user's terminal as needed.

[0642] Step 6:

[0643] The server analyzes learner progress data and accumulated learning data. This allows it to identify areas for improvement in the learning materials and gain insights that can be implemented in future lessons.

[0644] Step 7:

[0645] Based on the analysis results, the server adjusts and updates the content and structure of the teaching materials to prepare more effective educational content. This ensures improved learning effectiveness and consistency in education.

[0646] (Example 1)

[0647] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0648] Traditional educational support systems have problems with providing learners with effective and efficient learning materials, tracking their progress, and providing personalized feedback. Furthermore, they have challenges in responding quickly and accurately to learners' questions, and insufficient optimization of learning materials based on data.

[0649] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0650] In this invention, the server includes means for generating learning materials based on educational information, means for distributing the generated learning materials to terminals and recording learning progress, and means for analyzing inquiries from learners and providing relevant learning materials. This enables the provision of optimal learning materials tailored to each learner's progress and prompt and accurate question answering, thereby providing an overall efficient learning environment.

[0651] "Educational information" refers to all data and materials related to educational activities, including information related to curriculum guidelines and learner progress.

[0652] "Learning materials" are content created for learners to use for educational purposes and are provided in formats such as text, images, and videos.

[0653] "Terminal" refers to a device used by learners to receive and manipulate learning materials, and includes personal computers, tablets, and smartphones.

[0654] "Progress" refers to indicators or records that show how much progress learners have made in educational activities.

[0655] An "inquiry" refers to a question that a learner submits to the system seeking answers or information they want to resolve.

[0656] "Natural language" refers to the language that humans use in everyday life, and the system is required to understand questions and instructions by analyzing this language.

[0657] A "generative AI model" refers to a collection of programs and algorithms that use artificial intelligence to automatically create text and content.

[0658] "Optimizing educational materials" refers to the process of adjusting the content and structure of educational materials so that they are more effective and appropriate for learners.

[0659] This invention provides an educational support system that improves learners' ICT skills. The system consists of a server, learner terminals, and an AI component with advanced data processing capabilities.

[0660] The server connects to the database and uses Python scripts and SQL queries to collect educational information. The collected data is passed to an AI component, where a generative AI model automatically generates learning materials based on the curriculum guidelines. A general-purpose AI platform is used for this process. The generated materials are created in HTML format and adjusted according to specific educational needs.

[0661] The server delivers generated learning materials to the device using the HTTP protocol. The device displays the learning materials through an interface using React or similar technologies, allowing users to interactively learn from the received content. As learners use the materials, the device tracks user actions in real time using JavaScript and other technologies, recording their progress. This makes it possible to monitor learners' progress in detail.

[0662] Users can ask questions via voice or text if they encounter difficulties during their learning process. The server analyzes the received questions using natural language processing techniques and provides relevant learning materials and answers. This communication utilizes standard network connectivity technologies.

[0663] The server uses Python analysis tools to analyze learner progress and collected learning data. The analysis results are visualized using frameworks such as Django, and used to retrain the AI ​​and continuously improve the learning materials.

[0664] As a concrete example, a server might use the prompt "Mathematics materials for first-year high school students" to generate learning materials on "Fundamentals of Functions." These materials are delivered as HTML content combined with images and graphs for easy understanding. If a user asks, "I don't understand functions," the server immediately suggests reviewing the relevant section.

[0665] By coordinating the entire system in this way, it becomes possible to create an efficient and effective learning environment in educational settings.

[0666] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0667] Step 1:

[0668] The server accesses an educational database and collects educational information. It uses Python scripts and SQL queries to gather the latest curriculum guidelines and teaching materials. This data is provided to the server as input and serves as the source data for generating learning materials. Specifically, it retrieves data such as "Mathematics Curriculum for First-Year High School Students" from the database. As output, it sends a set of educational information to the AI ​​model for generating learning materials.

[0669] Step 2:

[0670] The server automatically generates learning materials using a generative AI model. The educational information collected in Step 1 is used as input. The server provides this information to the AI ​​component, which analyzes it using natural language processing techniques. As a result of the data processing, HTML-formatted learning materials tailored to the learner's level are output. Specifically, it generates learning materials on "Fundamentals of Functions," incorporating diagrams and practice problems.

[0671] Step 3:

[0672] The server delivers the generated learning materials to the terminal. The HTML-formatted learning materials created in step 2 are used as input. The server sends these to the terminal via the HTTP protocol. As output, learning materials that can be displayed on the learner's terminal are prepared. Specifically, when the user opens the learning materials, they are displayed interactively in the terminal's browser.

[0673] Step 4:

[0674] The terminal tracks the learner's progress through a user interface. The learner's operation data is recorded as input on the terminal. Real-time data tracking is performed using JavaScript. Progress and answer results are sent to the server as output. Specifically, each time the learner solves a problem, the result is logged, and the accuracy rate and response time are analyzed.

[0675] Step 5:

[0676] The user enters their question into the terminal. This is sent to the server as input, and the server analyzes the question using natural language processing technology. As a result, the appropriate learning material or answer is output and presented back to the terminal. For example, if the user asks "I don't understand this function," the relevant learning material section is immediately displayed.

[0677] Step 6:

[0678] The server continuously analyzes learning data and updates the learning materials. Progress data collected in step 4 is used as input. By analyzing the data using Python analysis tools and visualizing it with Django or similar tools, a list of improvement suggestions is obtained as output. Specifically, if a particular section of a learning course is deemed incomprehensible, the content of that material is improved and redistributed.

[0679] (Application Example 1)

[0680] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0681] In today's educational environment, there is a need to provide education tailored to individual progress and understanding, and to efficiently improve ICT skills. Furthermore, there is a demand for educational systems that can quickly respond to learners' questions and provide real-time educational feedback. Traditional educational systems struggle to optimize this individualized approach, limiting their ability to efficiently facilitate learning.

[0682] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0683] In this invention, the server includes means for connecting to educational information sources to collect the latest data and generate educational materials based on predetermined criteria; means for distributing the generated educational materials to learner devices and tracking and recording educational progress; and means for providing real-time educational feedback through eye-tracking or voice input. This enables the provision of up-to-date educational content tailored to individual learners, improves learning efficiency, and allows for immediate and responsive educational support.

[0684] "Educational information sources" are databases and online resources that provide data and materials related to learning.

[0685] "Educational materials" refer to a collection of information provided to learners for the purpose of learning, and include textbooks and reference materials.

[0686] "Learner equipment" refers to electronic devices used by learners to receive and utilize educational materials.

[0687] "Educational progress" is an indicator that shows the extent to which learners understand and have acquired specific educational objectives.

[0688] "Eye-tracking" is a technology that tracks the movement of a learner's eyes, and uses this data to manipulate interfaces or analyze situations.

[0689] "Voice input" is a method of inputting instructions or data into a device using voice.

[0690] "Educational feedback" refers to advice and suggestions provided based on a learner's learning progress, with the aim of improving their learning methods and understanding.

[0691] The system for realizing this application consists of a server, a learner terminal, smart glasses, and an AI component. The server periodically collects data from educational information sources and automatically generates educational materials based on predetermined criteria. The generated educational materials are delivered to learners via the learner terminal or smart glasses.

[0692] By using smart glasses, learners can have their gaze detected and be effectively guided through educational materials. Using voice input, users can ask questions, and the server provides immediate, relevant educational feedback. Furthermore, learner progress is recorded in real time, and the server generates individually tailored educational feedback based on this data.

[0693] The hardware used will consist of smart glasses, and Google's cloud-based speech conversion service will be used for voice recognition. The Django framework will run on a server for tracking progress and distributing materials, and TensorFlow will be used for AI analysis. As a concrete example, one can imagine a citizen wearing smart glasses in a public place and receiving ICT education while taking a walk. An example of a prompt would be: "A new citizen wearing smart glasses will view educational content while walking in a park and ask questions to an AI assistant in real time. Explain how it is using eye tracking and voice to learn."

[0694] In this way, the system implementing the invention can support the improvement of learners' ICT skills through the generation, distribution, progress tracking, and provision of real-time feedback of educational materials.

[0695] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0696] Step 1:

[0697] The server connects to educational information sources and collects the latest educational data. Input includes information from external databases and online platforms via the internet. This data is then processed and converted into a format that conforms to specified educational standards. This enables the generation of educational materials tailored to learners.

[0698] Step 2:

[0699] The server uses a generative AI model to create educational materials based on the collected data. The input is the educational data processed in Step 1, and by processing this data with the generative AI model, it outputs customized educational materials tailored to each grade level and level. The output materials are saved as educational content and distributed in the next step.

[0700] Step 3:

[0701] The server delivers educational materials to the learner's device or smart glasses. The input is the educational material created in step 2, which is then sent to the learner's device. During this process, a communication protocol is used to transfer the data, making the material available for download on the device. The output is the learner's device with the educational material downloaded.

[0702] Step 4:

[0703] The terminal presents the delivered educational materials through its user interface function. The input is the educational materials downloaded in step 3, and these materials are provided to the user via a display device or audio output. No data calculation is required; the primary operation is the presentation of data.

[0704] Step 5:

[0705] The user learns while wearing smart glasses and asks questions using eye-tracking and voice input functions. The input consists of the user's voice and eye-tracking data. This data is collected by the device and sent to the server. The output is information on the user's learning progress and the content of the questions, which are processed in the next step.

[0706] Step 6:

[0707] The server analyzes the learning data and questions submitted by the user and provides relevant feedback in real time. The input is the user data collected in step 5, which is processed by the AI ​​analysis engine to generate optimal feedback and additional educational materials. The output is the feedback information returned to the learner.

[0708] Step 7:

[0709] The server continuously monitors learning data and tracks progress to optimize educational materials. The input is accumulated learning progress data, which is then analyzed to extract insights necessary for the next material update. The output is improvement suggestions that will be reflected in the next material update.

[0710] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0711] This invention provides an educational support system that incorporates an emotion engine for recognizing user emotions, thereby offering an educational experience tailored to the individual learner's needs. This system is constructed through the collaboration of a server, a learner terminal, and the emotion engine.

[0712] Emotion recognition and content adjustment

[0713] The device recognizes the learner's emotions through camera and microphone sensors and analyzes that data. For example, if a user shows signs of impatience during learning, the emotion engine analyzes this information and adjusts the difficulty level of the learning content or displays an encouraging message.

[0714] Learning progress and feedback

[0715] The server combines emotional data with learning progress to generate more personalized feedback. Based on emotional data, if the user is experiencing stress, it adjusts learning time or provides content to help them relax. This kind of feedback improves the user experience and enhances learning efficiency.

[0716] Analysis and improvement of the educational environment

[0717] The server collects and analyzes sentiment data from multiple users to evaluate the overall learning environment. Based on this information, it provides insights that help improve educational policies and course materials. For example, for specific topics that many users find difficult simultaneously, the course materials are re-evaluated and revised as needed.

[0718] Specific example

[0719] If the system detects that the user is unfamiliar with the development environment and shows signs of anxiety, it will provide a simple user guide and recommend video learning materials to aid understanding. Furthermore, if the system determines that the user's understanding is low, it will provide links to more detailed materials and additional practice problems on the learner's device.

[0720] Systems with these functions aim to maximize learning efficiency and provide a comfortable learning environment by responding appropriately in real time according to the individual circumstances of each learner. In this way, the present invention proposes a new approach to enhancing the learning experience by combining emotion recognition technology and educational content.

[0721] The following describes the processing flow.

[0722] Step 1:

[0723] The user logs into the learner's device and accesses the designated educational content. The device uses its camera and microphone to capture the user's facial expressions and voice tone in real time.

[0724] Step 2:

[0725] The device sends the captured audio and video data to the emotion engine. The emotion engine analyzes this data to identify the user's emotional state (e.g., concentration, joy, stress).

[0726] Step 3:

[0727] The server receives emotion data from the emotion engine and integrates it with the user's current learning progress data. Based on this integrated data, it considers customizing the learning content and feedback.

[0728] Step 4:

[0729] The device provides the user with appropriate feedback and tailored learning content based on instructions received from the server. For example, if the user is detected as being in a state of agitation, a message prompting them to take a break will be displayed.

[0730] Step 5:

[0731] As the user continues to learn, emotion and progress data are continuously collected and sent to the server by the device. A real-time feedback loop is formed.

[0732] Step 6:

[0733] The server analyzes long-term sentiment data to generate insights for improving the quality of educational content. This information is used to revise teaching materials and review educational policies.

[0734] Step 7:

[0735] Throughout the entire system, the terminal and server continuously work together to enable flexible responses tailored to the user's emotions and learning progress. This provides the optimal educational environment for learners.

[0736] (Example 2)

[0737] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0738] Traditional educational support systems often fail to adequately address the individual needs of learners because they provide standardized learning content and progress management without considering the learner's emotional state. In particular, there is a lack of feedback that responds to the stress and changes in understanding that learners experience, making it difficult to maximize learning efficiency.

[0739] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0740] In this invention, the server includes means for collecting and analyzing video and audio data to identify the user's emotional state, means for adjusting the difficulty level of learning materials or presenting supplementary information based on the analyzed emotional state, and means for combining the emotional data with the learning progress to generate individualized feedback for the learner. This enables the provision of an educational experience tailored to the individual emotional state of the learner, thereby improving learning efficiency.

[0741] "User emotional state" refers to the psychological state exhibited by users during learning, such as joy, impatience, and stress, and is identified through video and audio data.

[0742] "Video and audio data" refers to digital data including the user's captured facial expressions and recorded voice, which is used to analyze their emotional state.

[0743] "Means of analysis" refers to technical methods and devices used to identify and evaluate emotional states from collected video and audio data.

[0744] "Adjusting the difficulty level of learning materials" refers to the process of changing the content and difficulty level of learning materials according to the user's emotional state.

[0745] "Supplementary information" refers to additional learning materials and support information provided to assist users in their learning, with the aim of improving learning efficiency.

[0746] "Individualized feedback" refers to specialized advice and instructions provided based on each learner's emotional state and learning progress.

[0747] "Learner progress" refers to a standard used to measure the degree of progress a learner has achieved during the educational process and to evaluate their growth based on that progress.

[0748] "Educational experience" is a collective term for the series of learning processes that learners acquire through an educational system, and the emotional and intellectual responses that accompany them.

[0749] This invention provides an educational support system that recognizes the emotional state of learners and adjusts the educational experience based on that state. This system is realized through the cooperation of a learner terminal, a server, and an emotion engine.

[0750] Terminal role

[0751] The device collects video and audio data from learners through sensors. Specifically, it records the learner's facial expressions with a camera and acquires audio with a microphone. This data is preprocessed using image processing libraries and speech analysis tools (e.g., OpenCV or Google Cloud Speech-to-Text).

[0752] Emotion recognition and analysis

[0753] The emotion engine analyzes data received from the device and uses machine learning models (e.g., TensorFlow or PyTorch-based models) to identify the learner's emotional state. This process makes it possible to recognize the learner's emotions in real time.

[0754] Server Role

[0755] Based on analyzed sentiment data, the server adjusts the difficulty level of learning materials to provide personalized educational content for each learner. It also provides supplementary materials as needed and generates individualized feedback. Furthermore, the server aggregates sentiment data from multiple users to gain insights that guide improvements to the overall learning environment.

[0756] Examples of specific cases and prompt statements

[0757] For example, if a user is feeling anxious about using a new programming environment, the device will detect their anxiety and the server will provide a simple user guide or instructional video. Also, if a user is stressed about a particular topic, the server will suggest relaxing content.

[0758] An example of a prompt for a generative AI model is: "Design a system that provides appropriate learning materials and feedback based on emotions, so that students can learn the fundamentals of programming effectively."

[0759] This system is designed to ensure that all learners have the optimal learning experience, and it provides personalized learning support using emotion recognition technology.

[0760] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0761] Step 1:

[0762] The device collects video and audio data from learners using sensors. It takes facial expressions captured by the camera and audio recorded by the microphone as input, and preprocesses this data using image processing libraries and audio analysis tools. This provides foundational data for analyzing the learners' emotional states.

[0763] Step 2:

[0764] The device sends pre-processed data to the emotion engine, which analyzes the learner's emotional state. The input data is analyzed using a machine learning model, and the emotional state is determined as output. Specifically, the model assigns emotional labels such as joy, anxiety, and stress.

[0765] Step 3:

[0766] The server receives the analyzed emotional state data and adjusts the learning content. Based on the input emotional data, it decides whether to adjust the difficulty level of the learning materials or provide supplementary information. For example, it might generate materials with a lower difficulty level or video links to aid understanding.

[0767] Step 4:

[0768] The server integrates emotional data and learning progress data to generate personalized feedback. Based on the input data, it calculates what kind of feedback is most effective and presents the user with specialized advice and suggestions for breaks as output.

[0769] Step 5:

[0770] The server aggregates emotional data collected from multiple learners to gain insights into the overall educational environment. Based on the accumulated data, suggestions for improving educational policies and materials are made, and reports and plans for optimizing the educational environment are presented as output.

[0771] (Application Example 2)

[0772] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0773] In today's learning environment, a challenge exists in that learners often struggle to receive education that is tailored to their individual emotional states. Traditional education systems respond to learning progress but rarely directly consider learners' emotions, thus limiting effective individualized instruction. Furthermore, when students experience stress or anxiety during learning, they often do not receive appropriate support, which leads to decreased learning efficiency.

[0774] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0775] In this invention, the server includes means for connecting to an educational information repository to collect the latest information and generating teaching materials based on predetermined standards; means for distributing the generated teaching materials to student terminals and tracking and recording learning progress; and means for identifying the emotions of students and providing learning support in accordance with those emotions. This makes it possible to provide immediate individualized instruction that takes into account the emotional state of students, thereby improving the effectiveness of learning.

[0776] The "Educational Information Library" is a data storage system that stores and makes accessible the latest data on learning materials, subject matter materials, and educational guidance provided to students.

[0777] "Student devices" refer to electronic devices used by learners to access educational content and progress in their studies, and include tablets, personal computers, and smartphones.

[0778] "Means for identifying emotions" refers to functions that include technical elements for analyzing the participant's facial expressions, tone of voice, etc., to determine the participant's emotional state.

[0779] "Means of providing learning support" refers to functions that provide adjustments to learning content, advice, and encouragement according to the learner's progress and emotions.

[0780] "Means for tracking and recording learning progress" refers to systems and functions for monitoring how far a student has progressed in their ongoing learning and recording that data.

[0781] The system that realizes this invention includes an educational information repository, student terminals, and an emotion recognition engine, each of which functions in cooperation with one another.

[0782] The server connects to the educational information repository and collects and updates the latest learning content to provide to students. This content is automatically generated based on pre-configured educational standards and is designed to meet diverse learning needs.

[0783] The learner's device collects the learner's facial expressions and voice in real time using a camera and microphone, and transmits this data to an emotion recognition engine. The emotion recognition engine incorporates commercially available technological elements, which analyze the learner's emotional state. If the learner shows signs of stress or anxiety, the device can automatically adjust the difficulty level of the content or display encouraging messages.

[0784] Furthermore, the server combines learning progress and sentiment data to generate feedback tailored to each learner's individual situation. As a result, learners can continue learning at a pace that is optimal for them. The data in the educational information repository is regularly updated, providing highly accurate feedback that accommodates various learning patterns.

[0785] As a concrete example, suppose a student is working on a new mathematical proposition and a confused expression is detected. In this case, the system analyzes the expression and immediately displays links to relevant video tutorials and practice problems to aid understanding.

[0786] An example of a prompt would be: "Please tell me how to adjust the learning content based on the emotions the learner is experiencing while working on a particular problem. Currently, the learner is not concentrating."

[0787] This system can significantly improve the quality of education by combining emotion recognition with personalized learning support.

[0788] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0789] Step 1:

[0790] The device captures the participant's facial expressions and voice using a camera and microphone. Real-time video and audio data are obtained as input. This data is sent to an emotion recognition engine, where it is converted into emotional states using facial recognition algorithms and voice analysis. The output is digital data indicating the participant's emotional state.

[0791] Step 2:

[0792] The server receives emotional state data and combines it with the learner's learning progress data. In this step, an adaptive learning algorithm is used to determine which learning content is appropriate based on the learner's current learning status and emotional state. The input is numerical data of learning progress and emotional state data, and the output is a list of adjusted learning content.

[0793] Step 3:

[0794] The device presents the learner with customized learning content. Specifically, video tutorials, supplementary materials, or new practice exercises are displayed on the learner's device. The input is a list of learning content received from the server, and the output is the learning material displayed on the learner's screen.

[0795] Step 4:

[0796] The user (learner) engages in learning activities based on the information presented. If a new question arises during this process, the device analyzes the question and presents additional relevant learning materials. The input is the user's question, and the output is a list of relevant learning materials. A question analysis algorithm uses a generative AI model to create prompts to identify appropriate learning materials.

[0797] Step 5:

[0798] The server accumulates learning history and sentiment data, and generates statistical data for future system improvements. This data serves as a starting point for developing new algorithms and contributes to the continuous optimization of educational content. The input is past learning session data, and the output is statistical data and improvement suggestions.

[0799] This will enable the entire system to provide a flexible learning experience tailored to the specific needs of each student.

[0800] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0801] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0802] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0803] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0804] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0805] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0806] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0807] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0808] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0809] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0810] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0811] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0812] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0813] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0814] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0815] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0816] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0817] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0818] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0819] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0820] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0821] The following is further disclosed regarding the embodiments described above.

[0822] (Claim 1)

[0823] A means of connecting to an educational database to collect the latest information and generating teaching materials based on predetermined standards,

[0824] A means of distributing generated learning materials to learner devices and tracking and recording learning progress,

[0825] A means of analyzing questions from learners and providing relevant learning materials,

[0826] A means of analyzing learning data and updating the content of the learning materials,

[0827] A system that includes this.

[0828] (Claim 2)

[0829] The system according to claim 1, characterized by having means for generating individually tailored feedback based on the learner's progress.

[0830] (Claim 3)

[0831] The system according to claim 1, characterized by comprising means for executing an algorithm that optimizes educational content based on collected learning data.

[0832] "Example 1"

[0833] (Claim 1)

[0834] A means of generating learning materials based on educational information,

[0835] A means of distributing generated learning materials to a device and recording learning progress,

[0836] A means of analyzing inquiries from learners and providing relevant learning materials,

[0837] A means of improving learning materials by analyzing collected learning data,

[0838] A means of tracking progress in real time through an interface,

[0839] A means of analyzing natural language to respond accurately to questions,

[0840] A method for dynamically updating educational materials using data-generated AI models,

[0841] A system that includes this.

[0842] (Claim 2)

[0843] The system according to claim 1, characterized in that it generates individualized responses based on the learner's progress.

[0844] (Claim 3)

[0845] The system according to claim 1, characterized by implementing an algorithm that efficiently improves educational content based on accumulated learning information.

[0846] "Application Example 1"

[0847] (Claim 1)

[0848] A means of connecting to educational information sources to collect the latest data and generating educational materials based on predetermined standards,

[0849] A means of distributing generated educational materials to learners' devices and tracking and recording educational progress,

[0850] A means of analyzing learners' questions and providing relevant educational materials,

[0851] A means of analyzing educational data and updating the content of educational materials,

[0852] A means of providing real-time educational feedback through eye-tracking or voice input,

[0853] A system that includes this.

[0854] (Claim 2)

[0855] The system according to claim 1, characterized by having means for presenting individually tailored educational responses based on the progress of the educator.

[0856] (Claim 3)

[0857] The system according to claim 1, characterized by comprising means for performing a procedure to optimize educational materials based on collected educational data.

[0858] "Example 2 of combining an emotion engine"

[0859] (Claim 1)

[0860] A means for collecting and analyzing video and audio data in order to identify the emotional state of the user,

[0861] A means of adjusting the difficulty level of learning materials or presenting supplementary information based on the analyzed emotional state,

[0862] A means of combining emotional data and learning progress to generate individualized feedback for learners,

[0863] A means of aggregating emotional data from multiple users to provide insights into improving the overall educational environment,

[0864] A system that includes this.

[0865] (Claim 2)

[0866] The system according to claim 1, which uses an improved emotion engine to identify a learner's stress level and suggests content that can adjust the learning time or help them relax.

[0867] (Claim 3)

[0868] The system according to claim 1, comprising means for executing an algorithm that optimizes information provision based on collected sentiment data and learning progress.

[0869] "Application example 2 when combining with an emotional engine"

[0870] (Claim 1)

[0871] A means of connecting to an educational information repository to collect the latest information and generating teaching materials based on predetermined standards,

[0872] A means of distributing generated learning materials to student devices and tracking and recording learning progress,

[0873] A means of analyzing questions from participants and providing relevant materials,

[0874] A means of analyzing learning data and updating the content of the learning materials,

[0875] A means of identifying the emotions of participants and providing learning support that corresponds to those emotions,

[0876] A system that includes this.

[0877] (Claim 2)

[0878] The system according to claim 1, characterized by having means for generating individually tailored feedback based on progress and emotional information.

[0879] (Claim 3)

[0880] The system according to claim 1, characterized by comprising means for executing an algorithm that optimizes educational content based on collected learning data and sentiment data. [Explanation of Symbols]

[0881] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. A means of connecting to an educational database to collect the latest information and generating teaching materials based on predetermined standards, A means of distributing generated learning materials to learner devices and tracking and recording learning progress, A means of analyzing questions from learners and providing relevant learning materials, A means of analyzing learning data and updating the content of the learning materials, A system that includes this.

2. The system according to claim 1, characterized by having means for generating individually adjusted feedback based on the learner's progress.

3. The system according to claim 1, characterized by comprising means for executing an algorithm that optimizes educational content based on collected learning data.

Images