system

Abstract

Provide a system. A method for receiving authentication information from a student; means for authenticating the student based on the authentication information; means for retrieving selected educational materials from a database; means for transmitting the teaching materials to the student's terminal; means for collecting learning activity data of the students from the terminals; means for analyzing the learning activity data and evaluating the student's level of understanding; means for generating feedback based on the evaluation result and transmitting the feedback to the terminal; A system including:

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In today's education system, educational disparities based on economic background and region are a major problem. Furthermore, teachers are overwhelmed with their workload, making it difficult for them to provide consistent, high-quality instruction to each student. This makes it difficult to provide appropriate instruction tailored to each student's learning progress and level of understanding, posing a risk of a decline in the overall quality of education. Therefore, there is a need for a system that can provide high-quality education to students worldwide, regardless of economic background, reduce the burden on teachers, and enable individualized instruction tailored to each student's learning progress. [Means for solving the problem]

[0005] The present invention provides a system that solves the above problems by the following means. Specifically, the system includes a means for receiving authentication information from a student and authenticating the student based on this information. The system also includes a means for retrieving selected learning materials from a database and transmitting them to the student's terminal. The system also includes a means for collecting student learning activity data from the terminal and analyzing the data to evaluate the student's level of understanding. The system also includes a means for generating feedback based on the evaluation results and transmitting it to the terminal, thereby providing an optimized learning experience for each student. In addition, the system includes a means for suggesting the next learning step based on the evaluation results and a means for individually generating feedback and suggesting customized next learning materials, thereby realizing high-quality education while reducing the burden on teachers.

[0006] "Student" refers to a person who belongs to an educational institution and is engaged in learning activities, and is a user of this system.

[0007] "Authentication Information" refers to the ID, password, or other authentication means used by a user to log in to a system.

[0008] A "database" is an electronic information storage device for systematically storing and managing teaching materials, user information, learning activity data, etc.

[0009] "Instructional Materials" refers to educational content, including videos, text, interactive quizzes, etc., used to convey learning content.

[0010] "Devices" are electronic devices such as computers, tablets, and smartphones that students and teachers use to access the system, view learning materials, and conduct learning activities.

[0011] "Learning activity data" refers to information about the learning activities students carry out through the system, and mainly refers to viewing time, quiz answers, etc.

[0012] "Analysis" is the process of evaluating students' learning progress and understanding based on collected data, and is a means of generating feedback based on the results.

[0013] "Evaluation results" are information about students' learning progress and level of understanding obtained based on the analysis.

[0014] "Feedback" is information to support students' learning, such as advice generated based on the evaluation results and suggestions for the next learning step.

[0015] "Next learning step suggestions" are suggestions for next learning content and learning materials recommended to students based on the evaluation results.

[0016] "Customized next learning materials" refers to next learning content and materials that are individually adjusted based on the student's learning progress and level of understanding. [Brief explanation of the drawings]

[0017] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8]FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram illustrating a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0018] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0019] First, the terms used in the following description will be explained.

[0020] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0021] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0022] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0023] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0024] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0025] [First embodiment]

[0026] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0027] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0028] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0029] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0030] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0031] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0032] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0033] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0034] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0035] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0036] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0037] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0038] This invention is a system that provides high-quality education to students around the world, reduces the burden on teachers, and enables individualized instruction according to learning progress. This system is realized through cooperation between a server, terminals, and users.

[0039] System Configuration

[0040] server

[0041] 1. User Authentication

[0042] The server authenticates the user (student) by checking the information in the database based on the authentication information (ID and password) received from the device. If authentication is successful, the server sends the user's profile data to the device.

[0043] 2. Management and distribution of teaching materials

[0044] The server manages various educational materials (videos, texts, interactive quizzes, etc.) stored in a database, retrieves the educational materials requested by the user from the device, and sends them to the device.

[0045] 3. Collection and analysis of training data

[0046] The server collects learning activity data sent from the device, including viewing time, quiz answer results, etc. The server analyzes this data and evaluates the user's level of understanding.

[0047] 4. Generate feedback

[0048] Based on the analysis results, the server generates appropriate feedback, such as suggestions for next learning steps or the provision of individually customized supplementary learning materials, and sends the generated feedback to the device.

[0049] Terminal

[0050] 1. User authentication and profile management

[0051] The user enters authentication information on the login screen from their device and sends it to the server. After authentication, the learning content begins to be displayed based on the profile data received from the server.

[0052] 2. Display of teaching materials

[0053] The device displays the educational material sent from the server to the user, for example, playing video lectures, displaying text materials, and providing interactive quizzes.

[0054] 3. Tracking learning activities

[0055] The device transmits the user's learning activity data (viewing time, quiz correct answer rate, etc.) to the server in real time.

[0056] 4. Viewing Feedback

[0057] Feedback sent from the server is displayed to the user, suggesting next steps and supplementary learning materials to help them continue their learning.

[0058] User

[0059] 1. Login and Authentication

[0060] Users log in to the system by entering their authentication information on a terminal.

[0061] 2. Learning Progression

[0062] Users can view and learn from the learning materials presented on their devices, including videos, textbooks, and quizzes.

[0063] 3. Receive feedback and take next steps

[0064] The user reviews the feedback displayed on the device and takes the suggested next learning steps.

[0065] Specific examples

[0066] For example, consider the case where a junior high school student user begins learning "quadratic equations."

[0067] 1. Login and Authentication

[0068] The user logs in by entering their ID and password on their device. The server performs authentication, and if authentication is successful, profile data is obtained.

[0069] 2. Acquiring and displaying educational materials

[0070] The user selects the learning material for "quadratic equations." The server retrieves the relevant learning material (video lectures, text materials, quizzes) from the database and sends it to the device. The device displays it, and the user begins learning.

[0071] 3. Learning Progress and Data Collection

[0072] Users watch videos, read text materials, and answer quizzes, and their devices send learning activity data (such as viewing time and quiz answers) to a server.

[0073] 4. Data analysis and feedback generation

[0074] The server analyzes the collected learning activity data, evaluates the user's level of understanding, and generates feedback (such as suggestions for the next learning step or additional practice questions) based on the analysis results and sends it to the device.

[0075] 5. View feedback and suggest next steps

[0076] The device displays feedback to the user, who can see what to work on next and complete suggested additional exercises.

[0077] In this way, the system can provide optimal education according to students' learning progress, reduce the burden on teachers, and improve the quality of education.

[0078] The processing flow will be explained below.

[0079] Step 1:

[0080] The user accesses the login screen from the terminal and enters their ID and password.

[0081] Step 2:

[0082] The terminal transmits the input authentication information to the server.

[0083] Step 3:

[0084] The server compares the received authentication information with information in a database and authenticates the user.

[0085] Step 4:

[0086] If the authentication is successful, the server sends the user's profile data to the terminal.

[0087] Step 5:

[0088] The terminal prepares to display the learning content based on the profile data received from the server.

[0089] Step 6:

[0090] The user selects the learning material they want to study on the screen of their device, for example, "quadratic equations."

[0091] Step 7:

[0092] The terminal requests the selected teaching material information from the server.

[0093] Step 8:

[0094] The server retrieves the relevant teaching materials (video lectures, text materials, quizzes) from the database and sends them to the terminal.

[0095] Step 9:

[0096] The terminal displays the educational materials sent from the server to the user, for example, playing lecture videos on a video player and displaying text materials.

[0097] Step 10:

[0098] Users watch videos, read text materials, and answer interactive quizzes on their devices.

[0099] Step 11:

[0100] The terminal collects the user's learning activity data (viewing time, quiz answers) and transmits it to the server in real time.

[0101] Step 12:

[0102] The server analyzes the learning activity data sent from the terminal and evaluates the user's level of understanding.

[0103] Step 13:

[0104] The server generates feedback based on the analysis results, such as "Your understanding of quadratic equations is insufficient; we recommend you do some additional practice problems."

[0105] Step 14:

[0106] The server transmits the generated feedback to the terminal.

[0107] Step 15:

[0108] The terminal displays the feedback received from the server to the user.

[0109] Step 16:

[0110] The user selects the next learning step based on the feedback and continues learning.

[0111] Through this series of steps, users are provided with the most appropriate content based on their learning progress. The server, devices, and users work together to provide high-quality education while reducing the burden on teachers.

[0112] Example 1

[0113] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0114] In conventional educational systems, it was difficult to provide optimal learning materials and feedback based on each student's individual learning progress. This also placed a heavy burden on teachers, making it difficult to provide high-quality education to all students. In particular, there was a lack of technological means to analyze students' learning data in real time and suggest personalized learning steps based on the results.

[0115] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0116] In this invention, the server includes means for receiving authentication information from the student, means for authenticating the student based on the authentication information, means for retrieving selected learning materials from a database, means for transmitting the learning materials to the student's terminal, means for collecting the student's learning activity data from the terminal, means for analyzing the learning activity data and evaluating the student's level of understanding, means for transmitting generated feedback to the terminal, means for evaluating the student's level of understanding using a generative AI model based on the student's learning progress data, and means for customizing suggestions for the next learning step based on the evaluation and presenting appropriate learning content using prompt sentences. This makes it possible to provide optimal learning materials and feedback according to each student's learning progress, thereby reducing the burden on teachers and providing high-quality education.

[0117] "Authentication information" refers to information used to log in to the system, such as a student ID and password.

[0118] A "database" is a system that centrally manages and stores information such as teaching materials, learning activity data, and user profiles.

[0119] "Devices" are devices used by students, such as computers, tablets, and smartphones.

[0120] "Learning activity data" refers to data related to the learning process, such as students' viewing time, quiz answer results, and text material viewing history.

[0121] "Comprehension" is an assessment of how well a student understands a particular learning material or topic.

[0122] "Feedback" refers to guidance and advice provided based on learning activity data and comprehension analysis results.

[0123] A "generative AI model" is a model that uses machine learning algorithms to extract patterns and trends from learning data and assess students' understanding.

[0124] A "prompt" is text that expresses instructions or questions that are input to a generative AI model.

[0125] "Personalization" means providing individually optimized educational content and feedback based on each student's learning progress and level of understanding.

[0126] A "learning step" is a specific learning activity or progress that a student should undertake next.

[0127] This invention provides an educational system that provides individualized instruction according to students' learning progress and reduces the burden on teachers. The system is realized through cooperation between a server, a terminal, and a user. The details are described below.

[0128] Server configuration and operation

[0129] The server performs the following main functions:

[0130] User authentication: The server authenticates the user (student) by comparing the authentication information (ID and password) received from the device with information in the database. If authentication is successful, the server sends the user's profile data to the device. At this time, a JWT (JSON Web Token) is used as the authentication token.

[0131] Management and distribution of learning materials: The server manages learning materials (videos, texts, quizzes, etc.) stored in a database. It retrieves learning materials requested by users from their devices and sends them to the devices. For example, learning materials are stored in a MySQL (registered trademark) database.

[0132] Collection and analysis of learning data: The server collects learning activity data sent from the device. This data includes viewing time, quiz answer results, etc. The server uses the Python scikit-learn library to train a generative AI model and analyze the data.

[0133] Feedback Generation: Based on the analysis results, the server generates appropriate feedback, including suggesting next learning steps and providing personalized learning materials.

[0134] Terminal configuration and operation

[0135] The terminal performs the following main functions:

[0136] User authentication and profile management: The user enters authentication information on the login screen from the device and sends it to the server. After authentication, the learning content begins to be displayed based on the profile data received from the server.

[0137] Display of educational materials: The terminal displays the educational materials sent from the server to the user. For example, a YouTube (registered trademark) video player can be embedded to play video lectures.

[0138] Learning activity tracking: The device sends the user's learning activity data (viewing time, quiz correct answer rate, etc.) to the server in real time. This data is accumulated on the server and used for analysis.

[0139] Show feedback: Show feedback sent by the server to the user, for example, "It seems you don't understand the basics of quadratic equations. Please study the supplementary material below."

[0140] User Actions

[0141] The user performs the following main actions:

[0142] Login and authentication: The user logs in by entering their ID and password on the terminal. For example, they enter "student123" and "password!" and press the login button.

[0143] Learning Progress: Users can view and learn the materials presented on their devices. They can watch videos, read text materials, and answer quizzes.

[0144] Receive feedback and take next steps: Users review the feedback displayed on their device and take the suggested next learning step. For example, they can click on "Additional learning materials for quadratic equations" to begin additional learning.

[0145] Specific examples

[0146] For example, consider a case where a junior high school student user begins studying "quadratic equations." The user logs in by entering their ID and password on their device. If authentication is successful, profile data is retrieved from the server. Next, the user selects learning materials for "quadratic equations," and the server retrieves the relevant materials from the database and sends them to the device. The user watches videos, reads text materials, and answers quizzes. The device sends this learning activity data to the server. The server analyzes the collected data and evaluates the level of understanding using a generative AI model. Feedback is generated and the next learning step is suggested. The user continues studying according to this feedback.

[0147] Prompt Sentence Examples

[0148] "How can we use generative AI models to tailor optimal feedback to users based on their learning progress?"

[0149] In this way, the system of the present invention provides optimal education according to the student's learning progress, enabling the student to progress in learning efficiently.

[0150] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0151] Step 1: Enter and submit your credentials

[0152] User: The user enters their ID and password on their device and clicks the login button. For example, they enter "student123" and "password!"

[0153] Input: User ID and password.

[0154] Terminal: The terminal sends the entered authentication information to the server.

[0155] Output: Sending authentication information to the server.

[0156] Step 2: User authentication

[0157] Server: The server checks the received authentication information against a database and authenticates the user.

[0158] Input: Authentication information sent from the device (user ID and password).

[0159] Data manipulation: Using an SQL query, the authentication information is matched against the user table in the database.

[0160] Output: Authentication result (success or failure) and profile data.

[0161] Specific operation: Connect to the MySQL database and search the user table. If authentication is successful, obtain profile data and send it to the device. If authentication fails, send an error message.

[0162] Step 3: View your profile data

[0163] Terminal: The terminal saves the profile data received from the server and transitions to the learning screen.

[0164] Input: Profile data from the server.

[0165] Output: Transition to the learning screen after logging in.

[0166] Specific operation: Upon successful authentication, a dashboard showing the user's specific learning progress and history will be displayed.

[0167] Step 4: Request and submit materials

[0168] User: Select the material you want to learn on the learning screen. For example, click "Math > Quadratic Equations."

[0169] Input: The ID of the selected material.

[0170] Terminal: Sends a request including the learning material ID to the server.

[0171] Output: Sending a learning material request to the server.

[0172] Step 5: Acquire and distribute materials

[0173] Server: Receives the learning material request and retrieves the corresponding learning material data from the database.

[0174] Input: Teaching material request from the terminal (teaching material ID).

[0175] Data processing: Retrieve teaching material data from the database using SQL queries.

[0176] Output: Sending teaching material data to the terminal.

[0177] Specific operation: The relevant video URL, text material, quiz, and other data are read from the database and sent to the device.

[0178] Step 6: Displaying the materials

[0179] Terminal: The terminal receives the teaching material data sent from the server and displays it to the user.

[0180] Input: Teaching material data from the server (video URL, text materials, quizzes).

[0181] Output: Display of teaching materials.

[0182] What it does: Embeds a video player using YouTube's API, plays videos, displays text material, and starts an interactive quiz.

[0183] Step 7: Tracking learning activities

[0184] Terminal: Sends user learning activity data (viewing time, quiz answer results, etc.) to the server in real time.

[0185] Input: User learning activity data.

[0186] Data processing: Collect various activity data (e.g., start time, end time, correct answer rate) and convert them into an appropriate format.

[0187] Output: Sending learning activity data to the server.

[0188] Specific operation: Generate a log of learning activities and periodically send it to the server.

[0189] Step 8: Collect and analyze training data

[0190] Server: Analyzes the collected data and evaluates comprehension using a generative AI model.

[0191] Input: Learning activity data sent from the device.

[0192] Data processing: Preprocess the data using Python's scikit-learn library and analyze it using a generative AI model.

[0193] Output: Generate comprehension assessment results and feedback.

[0194] Specific operation: Learning activity data is fed into an AI model to generate performance metrics (e.g., grades, comprehension scores).

[0195] Step 9: Generate feedback

[0196] Server: Generates feedback based on the analysis results, including suggesting next learning steps and providing personalized learning materials.

[0197] Input: Comprehension assessment results and output of the generative AI model.

[0198] Data processing: Generating feedback messages and selecting the next teaching materials.

[0199] Output: Sending feedback information to a terminal.

[0200] What it does: Based on a specific comprehension score, generate links to additional practice questions or supplemental materials and send them to your device.

[0201] Step 10: View feedback and take next steps

[0202] Terminal: The terminal displays the feedback received from the server to the user.

[0203] Input: Feedback information from the server.

[0204] Output: Displaying feedback to the user and suggesting next learning steps.

[0205] What it does: A dialog box appears on the screen, showing the next learning step and links to additional learning materials. It tells the user to click on "Additional learning materials for quadratic equations."

[0206] Example prompt

[0207] "How can we use generative AI models to tailor optimal feedback to users based on their learning progress?"

[0208] Through the above process, this system can provide optimal education according to students' learning progress, realizing an efficient and personalized learning experience.

[0209] (Application example 1)

[0210] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0211] Modern factories require efficient and effective training for new workers and existing workers learning new processes. However, conventional methods have difficulty providing education and training tailored to each worker's individual level of understanding, making it difficult to contribute to improving overall production efficiency and safety. The present invention aims to solve these problems by providing a system that collects and analyzes workers' learning activity data in real time and provides training materials and feedback based on each worker's individual level of understanding.

[0212] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0213] In this invention, the server includes means for receiving authentication information from a student, means for authenticating the student based on the authentication information, means for retrieving selected learning materials from a database, means for transmitting the learning materials to the student's terminal, means for collecting learning activity data of the student from the terminal, means for analyzing the learning activity data and evaluating the student's level of understanding, means for generating feedback based on the evaluation results and transmitting the feedback to the terminal, means for supporting the learning progress of factory workers based on the feedback, and means for providing the factory workers with training materials for learning new processes and machine operations. This enables factory workers to receive effective training tailored to their individual levels of understanding.

[0214] "Students" refers to factory workers who are users, and includes beginners who need to learn new processes or machine operations, as well as existing experts.

[0215] "Authentication Information" refers to information such as ID and password required for a user to access the system.

[0216] "Database" refers to the centralized management system where training materials, user profiles, learning activity data, etc. are stored.

[0217] "Instructional Materials" means educational content provided to factory workers, including video lectures, text materials, interactive quizzes, etc.

[0218] "Devices" refers to devices used by factory workers to view learning materials and carry out learning activities. Examples include computers and tablets.

[0219] "Learning activity data" is data related to the worker's learning activities, including viewing time and quiz answer results.

[0220] "Feedback" refers to information generated based on the analysis of learning activity data, suggesting the next learning step or additional training content.

[0221] "Factory workers" refers to people who operate machines and manage processes at production sites and receive training.

[0222] "Training materials" refers to educational materials provided to factory workers to learn new processes or machine operations.

[0223] The present invention is a system for efficiently providing training to factory workers and tracking and analyzing their progress in real time. An embodiment of this system will be described in detail below.

[0224] System Configuration

[0225] server

[0226] The server has multiple functions and plays a central role in managing and implementing training for factory workers. Specifically, it implements the following functions:

[0227] 1. User Authentication

[0228] The server authenticates the factory worker by checking the information in the database based on the authentication information (ID and password) of the factory worker received from the terminal. If authentication is successful, the server sends the worker's profile data to the terminal.

[0229] 2. Management and distribution of teaching materials

[0230] The server manages the training materials (videos, text, interactive quizzes, etc.) used by factory workers in a database, retrieves the materials requested by the workers from their terminals, and sends them to the terminals.

[0231] 3. Collection and analysis of training data

[0232] The server collects learning activity data of factory workers in real time, including viewing time, quiz answer results, etc. The server analyzes this data and evaluates the worker's level of understanding.

[0233] 4. Generate feedback

[0234] Based on the analysis results, the server generates appropriate feedback, including suggestions for next learning steps and the provision of additional training materials, and sends the generated feedback to the device.

[0235] Terminal

[0236] The terminal is a device that is directly operated by factory workers and provides an interface for training. Specifically, it implements the following functions:

[0237] 1. User authentication and profile management

[0238] The worker enters authentication information on the login screen from their terminal and sends it to the server. After authentication, the learning content begins to be displayed based on the profile data received from the server.

[0239] 2. Display of teaching materials

[0240] The terminals display educational materials sent from the server to the workers, for example, playing video lectures on machine operation, displaying text materials on operating procedures, and providing interactive quizzes.

[0241] 3. Tracking learning activities

[0242] The terminal collects data on the worker's learning activities (such as viewing time and quiz correct answer rate) in real time and sends it to the server.

[0243] 4. Viewing Feedback

[0244] Feedback sent from the server is displayed to the worker, suggesting next learning steps and additional training content to help them continue learning.

[0245] user

[0246] Factory workers can be efficiently trained using this system, specifically through the following process:

[0247] 1. Login and Authentication

[0248] Workers log in to the system by entering their authentication information on a terminal.

[0249] 2. Learning Progression

[0250] Workers can access the learning materials provided on their devices and progress through their studies, watching videos, reading textbooks, and answering quizzes.

[0251] 3. Receive feedback and take next steps

[0252] The worker reviews the feedback displayed on the terminal and executes the suggested next learning steps.

[0253] Specific examples

[0254] For example, when a new worker begins training to operate a new machine, the steps would be as follows:

[0255] 1. Login and Authentication

[0256] The worker logs in by entering their ID and password on the terminal. The server performs authentication and, if successful, obtains their profile data.

[0257] 2. Acquiring and displaying educational materials

[0258] A worker selects a new machine's operating manual. The server retrieves the relevant learning materials (video lectures, text materials, interactive quizzes) and sends them to the terminal. The terminal displays them, and the worker begins learning.

[0259] 3. Learning Progress and Data Collection

[0260] Workers watch videos, read textual materials on operating procedures, and answer quizzes, and their devices send this learning activity data to a server.

[0261] 4. Data analysis and feedback generation

[0262] The server analyzes the collected learning activity data, evaluates the worker's level of understanding, and generates feedback (such as suggestions for the next learning step or additional practice questions) based on the analysis results and sends it to the terminal.

[0263] 5. View feedback and suggest next steps

[0264] The device displays feedback to the worker, who can see what to do next and implement any additional techniques suggested.

[0265] Example prompts for generative AI models

[0266] "Please explain how to implement a factory worker operation training system. Please explain in detail the specific steps, including user authentication, learning material management, learning data collection and analysis, and feedback generation."

[0267] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0268] Step 1:

[0269] A factory worker (hereafter referred to as the user) enters authentication information (ID and password) into a terminal. The terminal receives this information and sends it to the server. The server compares it with user data in a database, and if authentication is successful, it returns profile data to the terminal. The input is the user's ID and password, and the output is the authentication result and profile data.

[0270] Step 2:

[0271] The terminal displays the user's learning dashboard based on the profile data received from the server. This dashboard contains a list of available training materials. The user selects the training material they want to learn. The input is the profile data and the user's material selection, and the output is a display of the selected material.

[0272] Step 3:

[0273] Based on the user's selection, the terminal sends a learning material request to the server. The server retrieves the requested learning material from the database and sends it to the terminal. The input is the learning material request, and the output is the learning material data.

[0274] Step 4:

[0275] The device displays the learning material data (videos, text, quizzes) received from the server to the user. The user progresses through learning by watching or reading the data and answering quizzes. The input is the learning material data, and the output is the user's learning activity data.

[0276] Step 5:

[0277] The device collects the user's learning activity data in real time and sends it to the server. The learning activity data includes viewing time, answer results, correct answer rate, etc. The server receives this data and stores it in a database. The input is the learning activity data, and the output is the stored learning activity data.

[0278] Step 6:

[0279] The server analyzes the saved learning activity data and evaluates the user's level of understanding. Here, the analysis is based on factors such as viewing time and quiz correct answer rate. The input is the learning activity data, and the output is the comprehension evaluation result.

[0280] Step 7:

[0281] The server generates individually customized feedback based on the comprehension assessment results. This feedback may include suggestions for the next learning step or additional practice questions. The generated feedback is sent to the device. The input is the comprehension assessment results, and the output is the feedback.

[0282] Step 8:

[0283] The terminal displays the feedback received from the server to the user, and the user performs the next learning step based on the feedback. The input is the feedback, and the output is the user's continued learning.

[0284] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0285] This invention is a system that provides high-quality education to students around the world, reduces the burden on teachers, and enables individualized instruction according to learning progress. By combining this invention with an emotion engine that recognizes students' emotions, it is possible to provide more precise feedback and adjust learning. This system works in cooperation between the server, the terminal, and the user, and realizes learning support that takes into account the user's emotions through the emotion engine.

[0286] System Configuration

[0287] server

[0288] 1. User Authentication

[0289] The server authenticates the user (student) by checking the information in the database based on the authentication information (ID and password) received from the device. If authentication is successful, the server sends the user's profile data to the device.

[0290] 2. Management and distribution of teaching materials

[0291] The server manages various educational materials (videos, text, interactive quizzes, etc.) stored in a database. It retrieves educational materials requested by users from their devices and sends them to the devices.

[0292] 3. Collection and analysis of training data

[0293] The server collects learning activity data sent from the device, including viewing time, quiz answer results, etc. The server analyzes this data and evaluates the user's level of understanding.

[0294] 4. Generate feedback

[0295] Based on the analysis results, the server generates appropriate feedback, such as suggestions for next learning steps or the provision of individually customized supplementary learning materials, and sends the generated feedback to the device.

[0296] 5. Emotion engine integration and analysis

[0297] The emotion engine analyzes the user's facial expressions, tone of voice, input data, etc. sent from the device during learning, and recognizes the user's emotions. The recognition results are sent to the server, and the feedback content is adjusted based on this.

[0298] Terminal

[0299] 1. User authentication and profile management

[0300] The user enters authentication information on the login screen from the terminal and sends it to the server. After authentication, the system prepares to display the learning content based on the profile data received from the server.

[0301] 2. Display of teaching materials

[0302] The terminal displays the educational materials sent from the server to the user, for example, playing lecture videos in a video player, displaying text materials, and providing interactive quizzes.

[0303] 3. Tracking learning activities

[0304] The terminal collects the user's learning activity data (viewing time, quiz correct answer rate, etc.) and transmits it to the server in real time.

[0305] 4. Viewing Feedback

[0306] Feedback sent from the server is displayed to the user, suggesting next steps and supplementary learning materials to help them continue their learning.

[0307] 5. Collecting and transmitting emotional data

[0308] The device sends the user's facial expressions, tone of voice, input data, etc. to the emotion engine to recognize emotions. The recognition results are then sent to the server.

[0309] User

[0310] 1. Login and Authentication

[0311] The user logs in to the system by entering authentication information from the terminal.

[0312] 2. Learning Progression

[0313] Users can view and learn from the learning materials presented on their devices, including videos, textbooks, and quizzes.

[0314] 3. Receive feedback and take next steps

[0315] The user reviews the feedback displayed on the device and performs the suggested next learning step.

[0316] Specific examples

[0317] For example, consider the case where a junior high school student user begins learning "quadratic equations."

[0318] 1. Login and Authentication

[0319] The user logs in by entering their ID and password on their device. The server performs authentication, and if authentication is successful, the profile data is retrieved.

[0320] 2. Acquiring and displaying educational materials

[0321] The user selects the learning material for "quadratic equations." The server retrieves the relevant learning material (video lectures, text materials, quizzes) from the database and sends it to the terminal. The terminal displays it, and the user begins learning.

[0322] 3. Learning Progress and Data Collection

[0323] Users watch videos, read text materials, and answer quizzes. The device sends learning activity data (such as viewing time and quiz answers) to the server.

[0324] 4. Data analysis and feedback generation

[0325] The server analyzes the collected learning activity data, evaluates the user's level of understanding, and generates feedback (such as suggestions for the next learning step or additional practice questions) based on the analysis results and sends it to the device.

[0326] 5. Emotion recognition and feedback regulation

[0327] The device sends the user's facial expressions and tone of voice to the emotion engine to recognize emotions. For example, if a student is feeling stressed, the emotion engine reports this to the server, which then generates feedback suggesting a break to relax.

[0328] 6. View feedback and suggest next steps

[0329] The device displays feedback to the user, who can then review the next learning step or suggestions for relaxation and continue learning.

[0330] In this way, the system not only provides optimal content according to students' learning progress, but also provides more effective learning support by taking emotions into consideration. The server, devices, and users work together to provide high-quality education through the emotion engine, while reducing the burden on teachers.

[0331] The processing flow will be explained below.

[0332] Step 1:

[0333] The user accesses the login screen from the terminal and enters their ID and password.

[0334] Step 2:

[0335] The terminal transmits the input authentication information to the server.

[0336] Step 3:

[0337] The server compares the received authentication information with information in a database and authenticates the user.

[0338] Step 4:

[0339] If the authentication is successful, the server sends the user's profile data to the terminal.

[0340] Step 5:

[0341] The terminal prepares to display the learning content based on the profile data received from the server.

[0342] Step 6:

[0343] The user selects the learning material they want to study on the screen of their device, for example, "quadratic equations."

[0344] Step 7:

[0345] The terminal requests the selected teaching material information from the server.

[0346] Step 8:

[0347] The server retrieves the relevant teaching materials (video lectures, text materials, quizzes) from the database and sends them to the terminal.

[0348] Step 9:

[0349] The terminal displays the educational materials sent from the server to the user, for example, playing lecture videos on a video player and displaying text materials.

[0350] Step 10:

[0351] Users watch videos, read text materials, and answer interactive quizzes on their devices.

[0352] Step 11:

[0353] The terminal collects the user's learning activity data (viewing time, quiz answers) and transmits it to the server in real time.

[0354] Step 12:

[0355] The device sends the user's facial expressions, tone of voice, input data, etc. that are being learned to the emotion engine to recognize emotions.

[0356] Step 13:

[0357] The emotion engine analyzes the user's facial expressions and tone of voice to understand the user's emotions.

[0358] Step 14:

[0359] The terminal transmits the recognition result of the emotion engine to the server.

[0360] Step 15:

[0361] The server analyzes the learning activity data sent from the device and the recognition results of the emotion engine to evaluate the user's level of understanding and emotional state.

[0362] Step 16:

[0363] The server generates feedback based on the analysis results, for example, if the user is feeling stressed, it generates feedback suggesting a break.

[0364] Step 17:

[0365] The server transmits the generated feedback to the terminal.

[0366] Step 18:

[0367] The terminal displays the feedback received from the server to the user.

[0368] Step 19:

[0369] The user reviews the feedback and takes suggested next learning steps or breaks.

[0370] Through this series of steps, the system provides optimal educational support that takes into account the user's learning progress and emotional state. The server, device, and emotion engine work together to deliver a high-quality learning experience and reduce the burden on teachers.

[0371] Example 2

[0372] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0373] In the current educational system, it is difficult to assess each student's individual learning progress and level of understanding in real time and provide appropriate feedback. Furthermore, there is a lack of learning support that takes students' emotions into consideration, which often leads to a decline in students' motivation to learn. This leads to a decline in the quality of education and an increase in the burden on teachers. A system that efficiently and effectively addresses these challenges is needed.

[0374] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0375] In this invention, the server includes means for receiving authentication information from a student, means for authenticating the student based on the authentication information, means for retrieving selected learning materials from a database, means for transmitting the learning materials to the student's terminal, means for collecting learning activity data of the student from the terminal, means for analyzing the learning activity data and evaluating the student's level of understanding, means for generating feedback based on the evaluation results and transmitting the feedback to the terminal, means for transmitting the student's facial expressions and tone of voice to an emotion engine for analysis, and means for adjusting the feedback content based on the emotion analysis results. This makes it possible to provide feedback according to the student's learning progress and support that takes emotions into consideration.

[0376] "Authentication information" refers to information that uniquely identifies a student, such as the ID and password that a student enters when logging in to the system.

[0377] "Study materials" refers to data such as videos, texts, and quizzes that are provided as learning materials necessary for students to advance their studies.

[0378] "Learning activity data" is a record of the activities that students performed while studying, including information such as viewing time and quiz correct answer rate.

[0379] "Feedback" is information that provides students with advice on their studies and suggests next steps based on the learning activity data analyzed by the server.

[0380] The "emotion engine" is computer software that analyzes students' facial expressions and tone of voice to recognize their emotions at that time.

[0381] "Emotion analysis results" are information about a student's emotional state obtained by the emotion engine by analyzing data such as facial expressions and tone of voice.

[0382] "Devices" are electronic devices used by students, such as computers, tablets, and smartphones.

[0383] The "server" is a central control computer that manages authentication information, distributes learning materials, analyzes learning activity data, generates feedback, and so on.

[0384] "Database" means a data storage system that the server accesses to store learning materials and student information.

[0385] This invention is an educational support system that provides high-quality education to students around the world, reduces the burden on teachers, and enables individualized instruction according to learning progress. This system is composed of three main elements: a server, a terminal, and a user, and realizes learning support that takes into account students' emotions through an emotion engine.

[0386] System Configuration

[0387] server

[0388] The server acts as a central control computer and is responsible for the following major processes:

[0389] 1. User Authentication

[0390] The server receives the authentication information (ID and password) sent by the student and authenticates the student by comparing it with the information in the database. If authentication is successful, the server sends the profile data to the device.

[0391] 2. Management and distribution of teaching materials

[0392] The server retrieves learning materials (videos, texts, quizzes, etc.) from the database and sends the materials selected by the student to the terminal.

[0393] 3. Collection and analysis of training data

[0394] The server collects learning activity data sent from the device, analyzes it to evaluate the level of understanding, and generates feedback based on the analysis results and sends it to the device.

[0395] 4. Emotion engine integration and analysis

[0396] The emotion engine analyzes facial expressions and tone of voice to recognize the student's emotions and sends the results to the server, which then adjusts the feedback content based on the emotion analysis results.

[0397] Terminal

[0398] Devices are electronic devices used by students, such as computers, tablets, and smartphones, and have the following functions:

[0399] 1. User authentication and profile management

[0400] The terminal sends the authentication information entered by the user to the server, and when the authentication result is returned, it displays the learning content based on the profile data.

[0401] 2. Display of teaching materials

[0402] The device displays the learning materials sent from the server and supports learning by, for example, playing lecture videos in a video player, displaying materials in a text viewer, and providing interactive quizzes.

[0403] 3. Tracking learning activities

[0404] The terminal collects students' learning activity data (viewing time, quiz correct answer rate, etc.) in real time and sends it to the server.

[0405] 4. Viewing Feedback

[0406] The device displays the feedback sent from the server and suggests next learning steps and supplementary learning materials.

[0407] 5. Collecting and transmitting emotional data

[0408] The device uses a built-in camera and microphone to transmit the student's facial expressions and tone of voice to the emotion engine, which then recognizes their emotions. The recognition results are then sent to the server.

[0409] User

[0410] The users are students who operate the system. They use the system as follows:

[0411] 1. Login and Authentication

[0412] The user logs in by entering their ID and password into the terminal. The server performs authentication, and if successful, the profile data is displayed on the terminal.

[0413] 2. Learning Progression

[0414] Users can view and learn from the learning materials presented on their devices, including videos, textbooks, and quizzes.

[0415] 3. Receive feedback and take next steps

[0416] The user reviews the feedback displayed on the terminal and performs the suggested next learning step.

[0417] Specific examples

[0418] For example, consider the case where a junior high school student user begins learning "quadratic equations."

[0419] 1. Login and Authentication

[0420] The user logs in by entering their ID and password on the terminal. The server performs authentication, and if authentication is successful, the profile data is acquired.

[0421] 2. Acquiring and displaying educational materials

[0422] The user selects the learning material for "quadratic equations." The server retrieves the corresponding learning material from the database and sends it to the terminal. The terminal displays it and begins learning.

[0423] 3. Learning Progress and Data Collection

[0424] Users watch videos, read text materials, and answer quizzes, and their devices send this learning activity data to a server.

[0425] 4. Data analysis and feedback generation

[0426] The server analyzes the collected learning activity data, evaluates the user's level of understanding, and generates feedback based on the analysis results and sends it to the device.

[0427] 5. Emotion recognition and feedback regulation

[0428] The device sends the user's facial expressions and tone of voice to the emotion engine to recognize emotions. For example, if a student is feeling stressed, the emotion engine reports this to the server and generates feedback suggesting a break to relax.

[0429] 6. View feedback and suggest next steps

[0430] The device displays feedback to the user, who can then review the next learning step or suggestions for relaxation and continue learning.

[0431] Example prompt

[0432] Use the following prompt to provide input to the generative AI model:

[0433] "Based on the following conditions, please explain the process of displaying teaching materials and generating feedback for students learning 'quadratic equations.'"

[0434] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0435] Step 1:

[0436] User Login and Authentication

[0437] The user starts up the device and enters their ID and password on the login screen. The entered authentication information is sent from the device to the server.

[0438] The server compares the received authentication information (ID and password) with the information in the database. If the comparison is successful, the server obtains the profile data and sends it to the device.

[0439] Specifically, the server queries the database to see if the matching ID and password combination exists.

[0440] Input: User ID and password

[0441] Output: Authentication result (if authentication is successful, profile data is sent to the device)

[0442] Step 2:

[0443] Requesting and receiving materials

[0444] The user selects the material they wish to study using the device interface, and this selection is sent as a request from the device to the server.

[0445] The server retrieves the requested learning materials from the database and sends them to the terminal.

[0446] Specifically, the server searches the database for learning materials (videos, texts, quizzes, etc.) that correspond to the specified learning material ID and retrieves them.

[0447] Input: ID of selected teaching material

[0448] Output: Send learning materials (videos, text, quizzes) to your device

[0449] Step 3:

[0450] Learning progression and data collection

[0451] The terminal displays the received learning materials to the user, who then uses them to progress with their learning.

[0452] All user activities (watching videos, reading texts, answering quizzes, etc.) are tracked by the device and sent to the server as learning activity data.

[0453] Specifically, the device collects the start and end times of the study session, quiz scores, and so on.

[0454] Input: User learning activity (video playback, quiz answers, etc.)

[0455] Output: Send learning activity data to the server

[0456] Step 4:

[0457] Analyzing training data and generating feedback

[0458] The server analyzes the collected learning activity data and evaluates the user's level of understanding, using criteria such as viewing time and quiz accuracy.

[0459] Based on the evaluation results, the server generates feedback including suggestions for next learning steps and additional practice questions and sends this to the device.

[0460] Specifically, the server uses an analysis algorithm to assess comprehension and determine appropriate feedback content.

[0461] Input: Learning activity data

[0462] Output: Send generated feedback to the terminal

[0463] Step 5:

[0464] Emotion data collection and analysis

[0465] The device sends the user's facial expressions and tone of voice to the emotion engine during training to recognize emotions, and the recognition results are sent to the server.

[0466] Specifically, the device captures facial expressions with its built-in camera and collects voice tone with its microphone.

[0467] The emotion engine analyzes this data and recognizes the user's emotions, and sends the results to the server.

[0468] Input: User's facial expression data, voice tone data

[0469] Output: Send the sentiment analysis results to the server

[0470] Step 6:

[0471] Adjusting and viewing feedback

[0472] The server adjusts the feedback based on the results of the emotion analysis. For example, if the user is feeling stressed, it generates feedback suggesting a break to relax.

[0473] The device displays the adjusted feedback to the user.

[0474] Specifically, the server dynamically changes the content of the feedback depending on the emotion analysis results.

[0475] Input: Sentiment analysis results

[0476] Output: Sends adjusted feedback to the device

[0477] In this way, the system optimizes the user's learning experience based on a flow that clearly indicates the specific processing and input / output at each step.

[0478] (Application example 2)

[0479] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0480] With the spread of online education, there is a need to provide high-quality education to students around the world. However, it is difficult to respond appropriately to the different levels of understanding and learning progress of each student. Furthermore, there is no support that takes into account emotional changes during learning. Therefore, there is a need for a system that can maximize students' learning effectiveness and reduce the burden on teachers.

[0481] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0482] In this invention, the server includes means for receiving authentication information from the student, means for authenticating the student based on the authentication information, means for retrieving selected learning materials from a database, means for transmitting the learning materials to the student's terminal, means for collecting the student's learning activity data from the terminal, means for analyzing the learning activity data and evaluating the student's level of understanding, means for generating feedback based on the evaluation results and transmitting the feedback to the terminal, means for analyzing the student's facial expressions and tone of voice to recognize emotions, and means for adjusting the feedback content in accordance with the recognized emotions. This enables personalized feedback that takes into account the student's level of understanding and emotional state, thereby improving learning effectiveness.

[0483] "Student" refers to a user of the learning content.

[0484] "Authentication information" refers to information such as ID and password that a user uses to log in to a system.

[0485] A "database" is an information management system for storing educational materials and learning data.

[0486] "Instructional Materials" refers to educational resources such as videos, textbooks, and interactive quizzes provided for learning purposes.

[0487] "Device" refers to a device (smartphone, tablet, PC, etc.) that students use to display and operate learning content.

[0488] "Learning activity data" refers to information about students' activities, such as the amount of time they spend viewing learning materials and the results of quizzes.

[0489] A "server" is a central computing unit that verifies authentication information, manages and distributes learning materials, analyzes learning data, and so on.

[0490] "Analysis" refers to the process of evaluating collected learning activity data and determining students' level of understanding.

[0491] "Feedback" refers to information provided to students based on the assessment results, such as suggestions for next learning steps or the provision of supplementary learning materials.

[0492] "Facial expressions" refers to data that infers emotions based on students' facial movements and expressions.

[0493] "Tone of voice" refers to data that infers emotions based on the way a student speaks, the pitch and intensity of their voice.

[0494] "Emotions" refers to the psychological state of students that can be inferred from their facial expressions and tone of voice.

[0495] An "emotion engine" is software or hardware that analyzes students' facial expressions and tone of voice to recognize their emotions.

[0496] "Personalization" means providing support that is individually tailored to each student's learning progress and emotional state.

[0497] System Overview

[0498] The system that realizes this application example provides high-quality education by linking the server, terminals, and users, and collecting, analyzing, and providing feedback on students' learning activity and emotional data. In particular, by combining it with an emotion engine, it is possible to recognize students' emotional states in real time and provide appropriate feedback according to their learning situation.

[0499] Hardware and Software Configuration

[0500] Server: A central computing unit that manages databases, authenticates users, analyzes data, and generates feedback. For example, a cloud server such as AWS (registered trademark) or Google (registered trademark) Cloud Platform can be used.

[0501] Device: A device on which students view and interact with learning content. Examples include smartphones, tablets, and PCs. This device is equipped with a camera and microphone, which are used to collect emotional data.

[0502] Emotion engine: Software that analyzes students' facial expressions and tone of voice to recognize their emotions. For example, libraries such as OpenCV and Google Cloud Vision AI can be used.

[0503] Processing steps

[0504] 1. User authentication: The user enters authentication information (ID and password) on the device and sends it to the server. The server verifies the authentication information, authenticates the user, and sends profile data to the device.

[0505] 2. Acquisition of learning materials: The server acquires the requested learning materials from the database and sends them to the user's terminal.

[0506] 3. Collection of learning activity data: The terminal collects learning activity data such as the user's viewing time and quiz correct answer rate, and transmits it to the server in real time.

[0507] 4. Data analysis: The server analyzes the collected learning activity data and evaluates the user's level of understanding.

[0508] 5. Feedback generation: Based on the evaluation results, feedback is generated suggesting next learning steps and supplementary learning materials, and this is sent to the device.

[0509] 6. Emotion data collection and analysis: The device sends the user's facial expressions and tone of voice during training via a camera and microphone to the emotion engine to recognize emotions. The server receives the recognition results and adjusts the feedback content.

[0510] Specific examples

[0511] For example, below is a specific scenario in which a junior high school student is learning about quadratic equations.

[0512] 1. The user logs into the app on their smartphone and selects the quadratic equations study material.

[0513] 2. The server sends the teaching materials (video lectures, text materials, quizzes, etc.) to the terminal.

[0514] 3. The user watches videos and answers quizzes. The device collects this data and sends it to the server.

[0515] 4. The server analyzes the learning activity data and evaluates the user's level of understanding.

[0516] 5. If the emotion engine recognizes "confusion" from the user's facial expression while watching a movie, the server generates feedback including supplementary explanations and sends it to the device.

[0517] 6. Furthermore, if the user is feeling "stressed," suggestions such as "take a five-minute break" will be made.

[0518] Example prompt sentence:

[0519] "My students are stressed. What are some ways to help them relax?"

[0520] "A student is confused. Please provide additional explanation for quadratic equations."

[0521] This allows for personalized feedback that takes into account the student's level of understanding and emotional state, maximizing learning outcomes while providing high-quality education.

[0522] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0523] Step 1: User authentication

[0524] The user enters authentication information (ID and password) from the terminal and sends it to the server, which collates this authentication information with information in a database and authenticates the user.

[0525] Input: ID, password

[0526] Output: Authentication result (profile data if successful)

[0527] Operation: The ID and password entered by the user are sent to the server, which then authenticates them by checking them against the authentication information in the database. If authentication is successful, the user's profile data is sent to the device.

[0528] Step 2: Acquire the study materials

[0529] The server retrieves the educational material requested by the user from the database and transmits it to the terminal.

[0530] Input: User request (e.g., teaching material ID)

[0531] Output: Teaching material data (videos, text, quizzes, etc.)

[0532] Operation: When a user sends a requested educational material ID from a terminal to the server, the server retrieves the corresponding educational material from the database and sends it to the terminal.

[0533] Step 3: Collect learning activity data

[0534] The terminal collects the user's viewing time, quiz correct answer rate, etc. in real time and transmits them to the server.

[0535] Input: Learning activity data (viewing time, quiz answer results, etc.)

[0536] Output: Sending learning activity data to the server

[0537] Operation: When a user watches or listens to educational materials or answers quizzes, their activity data is collected on the device and sent to the server in real time.

[0538] Step 4: Data analysis

[0539] The server analyzes the collected learning activity data and evaluates the user's level of understanding.

[0540] Input: Learning activity data

[0541] Output: Comprehension assessment results

[0542] Operation: The server analyzes learning activity data and performs data calculations using statistical analysis and machine learning algorithms to evaluate the user's level of understanding.

[0543] Step 5: Generate feedback

[0544] Based on the evaluation results, the server generates feedback suggesting the next learning step or supplementary learning materials and sends this to the terminal.

[0545] Input: Comprehension assessment results

[0546] Output: Feedback (suggestions for next learning steps, supplementary materials, etc.)

[0547] Operation: Based on the comprehension assessment results, the server uses a generative AI model to generate feedback suggesting next steps and supplementary learning materials. The generated feedback is then sent to the device.

[0548] Step 6: Collect and analyze emotion data

[0549] The device sends the user's facial expressions and tone of voice during training via a camera and microphone to the emotion engine to recognize emotions.

[0550] Input: facial expression data, tone of voice

[0551] Output: Emotion recognition result

[0552] How it works: Facial expression data and voice tone captured by the device's camera and microphone are sent to the emotion engine, which then analyzes the data and recognizes emotions. The recognized emotion results are then sent to the server.

[0553] Step 7: Adjusting Emotional Feedback

[0554] The server adjusts the feedback content based on the emotion recognition results and sends it to the terminal.

[0555] Input: Emotion recognition results

[0556] Output: Regulated Feedback

[0557] How it works: The server receives the emotion recognition results and adjusts the feedback content according to the recognized emotion. For example, if the student is tired, it suggests taking a break. The adjusted feedback is then sent to the device.

[0558] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0559] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0560] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0561] [Second embodiment]

[0562] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0563] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0564] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0565] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0566] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0567] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0568] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0569] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0570] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0571] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0572] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0573] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0574] This invention is a system that provides high-quality education to students around the world, reduces the burden on teachers, and enables individualized instruction according to learning progress. This system is realized through cooperation between a server, terminals, and users.

[0575] System Configuration

[0576] server

[0577] 1. User Authentication

[0578] The server authenticates the user (student) by checking the information in the database based on the authentication information (ID and password) received from the device. If authentication is successful, the server sends the user's profile data to the device.

[0579] 2. Management and distribution of teaching materials

[0580] The server manages various educational materials (videos, texts, interactive quizzes, etc.) stored in a database, retrieves the educational materials requested by the user from the device, and sends them to the device.

[0581] 3. Collection and analysis of training data

[0582] The server collects learning activity data sent from the device, including viewing time, quiz answer results, etc. The server analyzes this data and evaluates the user's level of understanding.

[0583] 4. Generate feedback

[0584] Based on the analysis results, the server generates appropriate feedback, such as suggestions for next learning steps or the provision of individually customized supplementary learning materials, and sends the generated feedback to the device.

[0585] Terminal

[0586] 1. User authentication and profile management

[0587] The user enters authentication information on the login screen from their device and sends it to the server. After authentication, the learning content begins to be displayed based on the profile data received from the server.

[0588] 2. Display of teaching materials

[0589] The device displays the educational material sent from the server to the user, for example, playing video lectures, displaying text materials, and providing interactive quizzes.

[0590] 3. Tracking learning activities

[0591] The device transmits the user's learning activity data (viewing time, quiz correct answer rate, etc.) to the server in real time.

[0592] 4. Viewing Feedback

[0593] Feedback sent from the server is displayed to the user, suggesting next steps and supplementary learning materials to help them continue their learning.

[0594] User

[0595] 1. Login and Authentication

[0596] Users log in to the system by entering their authentication information on a terminal.

[0597] 2. Learning Progression

[0598] Users can view and learn from the learning materials presented on their devices, including videos, textbooks, and quizzes.

[0599] 3. Receive feedback and take next steps

[0600] The user reviews the feedback displayed on the device and takes the suggested next learning steps.

[0601] Specific examples

[0602] For example, consider the case where a junior high school student user begins learning "quadratic equations."

[0603] 1. Login and Authentication

[0604] The user logs in by entering their ID and password on their device. The server performs authentication, and if authentication is successful, profile data is obtained.

[0605] 2. Acquiring and displaying educational materials

[0606] The user selects the learning material for "quadratic equations." The server retrieves the relevant learning material (video lectures, text materials, quizzes) from the database and sends it to the device. The device displays it, and the user begins learning.

[0607] 3. Learning Progress and Data Collection

[0608] Users watch videos, read text materials, and answer quizzes, and their devices send learning activity data (such as viewing time and quiz answers) to a server.

[0609] 4. Data analysis and feedback generation

[0610] The server analyzes the collected learning activity data, evaluates the user's level of understanding, and generates feedback (such as suggestions for the next learning step or additional practice questions) based on the analysis results and sends it to the device.

[0611] 5. View feedback and suggest next steps

[0612] The device displays feedback to the user, who can see what to work on next and complete suggested additional exercises.

[0613] In this way, the system can provide optimal education according to students' learning progress, reduce the burden on teachers, and improve the quality of education.

[0614] The processing flow will be explained below.

[0615] Step 1:

[0616] The user accesses the login screen from the terminal and enters their ID and password.

[0617] Step 2:

[0618] The terminal transmits the input authentication information to the server.

[0619] Step 3:

[0620] The server compares the received authentication information with information in a database and authenticates the user.

[0621] Step 4:

[0622] If the authentication is successful, the server sends the user's profile data to the terminal.

[0623] Step 5:

[0624] The terminal prepares to display the learning content based on the profile data received from the server.

[0625] Step 6:

[0626] The user selects the learning material they want to study on the screen of their device, for example, "quadratic equations."

[0627] Step 7:

[0628] The terminal requests the selected teaching material information from the server.

[0629] Step 8:

[0630] The server retrieves the relevant teaching materials (video lectures, text materials, quizzes) from the database and sends them to the terminal.

[0631] Step 9:

[0632] The terminal displays the educational materials sent from the server to the user, for example, playing lecture videos on a video player and displaying text materials.

[0633] Step 10:

[0634] Users watch videos, read text materials, and answer interactive quizzes on their devices.

[0635] Step 11:

[0636] The terminal collects the user's learning activity data (viewing time, quiz answers) and transmits it to the server in real time.

[0637] Step 12:

[0638] The server analyzes the learning activity data sent from the terminal and evaluates the user's level of understanding.

[0639] Step 13:

[0640] The server generates feedback based on the analysis results, such as "Your understanding of quadratic equations is insufficient; we recommend you do some additional practice problems."

[0641] Step 14:

[0642] The server transmits the generated feedback to the terminal.

[0643] Step 15:

[0644] The terminal displays the feedback received from the server to the user.

[0645] Step 16:

[0646] The user selects the next learning step based on the feedback and continues learning.

[0647] Through this series of steps, users are provided with the most appropriate content based on their learning progress. The server, devices, and users work together to provide high-quality education while reducing the burden on teachers.

[0648] Example 1

[0649] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0650] In conventional educational systems, it was difficult to provide optimal learning materials and feedback based on each student's individual learning progress. This also placed a heavy burden on teachers, making it difficult to provide high-quality education to all students. In particular, there was a lack of technological means to analyze students' learning data in real time and suggest personalized learning steps based on the results.

[0651] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0652] In this invention, the server includes means for receiving authentication information from the student, means for authenticating the student based on the authentication information, means for retrieving selected learning materials from a database, means for transmitting the learning materials to the student's terminal, means for collecting the student's learning activity data from the terminal, means for analyzing the learning activity data and evaluating the student's level of understanding, means for transmitting generated feedback to the terminal, means for evaluating the student's level of understanding using a generative AI model based on the student's learning progress data, and means for customizing suggestions for the next learning step based on the evaluation and presenting appropriate learning content using prompt sentences. This makes it possible to provide optimal learning materials and feedback according to each student's learning progress, thereby reducing the burden on teachers and providing high-quality education.

[0653] "Authentication information" refers to information used to log in to the system, such as a student ID and password.

[0654] A "database" is a system that centrally manages and stores information such as teaching materials, learning activity data, and user profiles.

[0655] "Devices" are devices used by students, such as computers, tablets, and smartphones.

[0656] "Learning activity data" refers to data related to the learning process, such as students' viewing time, quiz answer results, and text material viewing history.

[0657] "Comprehension" is an assessment of how well a student understands a particular learning material or topic.

[0658] "Feedback" refers to guidance and advice provided based on learning activity data and comprehension analysis results.

[0659] A "generative AI model" is a model that uses machine learning algorithms to extract patterns and trends from learning data and assess students' understanding.

[0660] A "prompt" is text that expresses instructions or questions that are input to a generative AI model.

[0661] "Personalization" means providing individually optimized educational content and feedback based on each student's learning progress and level of understanding.

[0662] A "learning step" is a specific learning activity or progress that a student should undertake next.

[0663] This invention provides an educational system that provides individualized instruction according to students' learning progress and reduces the burden on teachers. The system is realized through cooperation between a server, a terminal, and a user. The details are described below.

[0664] Server configuration and operation

[0665] The server performs the following main functions:

[0666] User authentication: The server authenticates the user (student) by comparing the authentication information (ID and password) received from the device with information in the database. If authentication is successful, the server sends the user's profile data to the device. At this time, a JWT (JSON Web Token) is used as the authentication token.

[0667] Management and distribution of learning materials: The server manages learning materials (videos, texts, quizzes, etc.) stored in a database. It retrieves learning materials requested by users from their devices and sends them to the devices. For example, learning materials are stored in a MySQL database.

[0668] Collection and analysis of learning data: The server collects learning activity data sent from the device. This data includes viewing time, quiz answer results, etc. The server uses the Python scikit-learn library to train a generative AI model and analyze the data.

[0669] Feedback Generation: Based on the analysis results, the server generates appropriate feedback, including suggesting next learning steps and providing personalized learning materials.

[0670] Terminal configuration and operation

[0671] The terminal performs the following main functions:

[0672] User authentication and profile management: The user enters authentication information on the login screen from the device and sends it to the server. After authentication, the learning content begins to be displayed based on the profile data received from the server.

[0673] Displaying educational materials: The device displays educational materials sent from the server to the user. For example, a YouTube video player can be embedded to play video lectures.

[0674] Learning activity tracking: The device sends the user's learning activity data (viewing time, quiz correct answer rate, etc.) to the server in real time. This data is accumulated on the server and used for analysis.

[0675] Show feedback: Show feedback sent by the server to the user, for example, "It seems you don't understand the basics of quadratic equations. Please study the supplementary material below."

[0676] User Actions

[0677] The user performs the following main actions:

[0678] Login and authentication: The user logs in by entering their ID and password on the terminal. For example, they enter "student123" and "password!" and press the login button.

[0679] Learning Progress: Users can view and learn the materials presented on their devices. They can watch videos, read text materials, and answer quizzes.

[0680] Receive feedback and take next steps: Users review the feedback displayed on their device and take the suggested next learning step. For example, they can click on "Additional learning materials for quadratic equations" to begin additional learning.

[0681] Specific examples

[0682] For example, consider a case where a junior high school student user begins studying "quadratic equations." The user logs in by entering their ID and password on their device. If authentication is successful, profile data is retrieved from the server. Next, the user selects learning materials for "quadratic equations," and the server retrieves the relevant materials from the database and sends them to the device. The user watches videos, reads text materials, and answers quizzes. The device sends this learning activity data to the server. The server analyzes the collected data and evaluates the level of understanding using a generative AI model. Feedback is generated and the next learning step is suggested. The user continues studying according to this feedback.

[0683] Prompt Sentence Examples

[0684] "How can we use generative AI models to tailor optimal feedback to users based on their learning progress?"

[0685] In this way, the system of the present invention provides optimal education according to the student's learning progress, enabling the student to progress in learning efficiently.

[0686] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0687] Step 1: Enter and submit your credentials

[0688] User: The user enters their ID and password on their device and clicks the login button. For example, they enter "student123" and "password!"

[0689] Input: User ID and password.

[0690] Terminal: The terminal sends the entered authentication information to the server.

[0691] Output: Sending authentication information to the server.

[0692] Step 2: User authentication

[0693] Server: The server checks the received authentication information against a database and authenticates the user.

[0694] Input: Authentication information sent from the device (user ID and password).

[0695] Data manipulation: Using an SQL query, the authentication information is matched against the user table in the database.

[0696] Output: Authentication result (success or failure) and profile data.

[0697] Specific operation: Connect to the MySQL database and search the user table. If authentication is successful, obtain profile data and send it to the device. If authentication fails, send an error message.

[0698] Step 3: View your profile data

[0699] Terminal: The terminal saves the profile data received from the server and transitions to the learning screen.

[0700] Input: Profile data from the server.

[0701] Output: Transition to the learning screen after logging in.

[0702] Specific operation: Upon successful authentication, a dashboard showing the user's specific learning progress and history will be displayed.

[0703] Step 4: Request and submit materials

[0704] User: Select the material you want to learn on the learning screen. For example, click "Math > Quadratic Equations."

[0705] Input: The ID of the selected material.

[0706] Terminal: Sends a request including the learning material ID to the server.

[0707] Output: Sending a learning material request to the server.

[0708] Step 5: Acquire and distribute materials

[0709] Server: Receives the learning material request and retrieves the corresponding learning material data from the database.

[0710] Input: Teaching material request from the terminal (teaching material ID).

[0711] Data processing: Retrieve teaching material data from the database using SQL queries.

[0712] Output: Sending teaching material data to the terminal.

[0713] Specific operation: The relevant video URL, text material, quiz, and other data are read from the database and sent to the device.

[0714] Step 6: Displaying the materials

[0715] Terminal: The terminal receives the teaching material data sent from the server and displays it to the user.

[0716] Input: Teaching material data from the server (video URL, text materials, quizzes).

[0717] Output: Display of teaching materials.

[0718] What it does: Embeds a video player using YouTube's API, plays videos, displays text material, and starts an interactive quiz.

[0719] Step 7: Tracking learning activities

[0720] Terminal: Sends user learning activity data (viewing time, quiz answer results, etc.) to the server in real time.

[0721] Input: User learning activity data.

[0722] Data processing: Collect various activity data (e.g., start time, end time, correct answer rate) and convert them into an appropriate format.

[0723] Output: Sending learning activity data to the server.

[0724] Specific operation: Generate a log of learning activities and periodically send it to the server.

[0725] Step 8: Collect and analyze training data

[0726] Server: Analyzes the collected data and evaluates comprehension using a generative AI model.

[0727] Input: Learning activity data sent from the device.

[0728] Data processing: Preprocess the data using Python's scikit-learn library and analyze it using a generative AI model.

[0729] Output: Generate comprehension assessment results and feedback.

[0730] Specific operation: Learning activity data is fed into an AI model to generate performance metrics (e.g., grades, comprehension scores).

[0731] Step 9: Generate feedback

[0732] Server: Generates feedback based on the analysis results, including suggesting next learning steps and providing personalized learning materials.

[0733] Input: Comprehension assessment results and output of the generative AI model.

[0734] Data processing: Generating feedback messages and selecting the next teaching materials.

[0735] Output: Sending feedback information to a terminal.

[0736] What it does: Based on a specific comprehension score, generate links to additional practice questions or supplemental materials and send them to your device.

[0737] Step 10: View feedback and take next steps

[0738] Terminal: The terminal displays the feedback received from the server to the user.

[0739] Input: Feedback information from the server.

[0740] Output: Displaying feedback to the user and suggesting next learning steps.

[0741] What it does: A dialog box appears on the screen, showing the next learning step and links to additional learning materials. It tells the user to click on "Additional learning materials for quadratic equations."

[0742] Example prompt

[0743] "How can we use generative AI models to tailor optimal feedback to users based on their learning progress?"

[0744] Through the above process, this system can provide optimal education according to students' learning progress, realizing an efficient and personalized learning experience.

[0745] (Application example 1)

[0746] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0747] Modern factories require efficient and effective training for new workers and existing workers learning new processes. However, conventional methods have difficulty providing education and training tailored to each worker's individual level of understanding, making it difficult to contribute to improving overall production efficiency and safety. The present invention aims to solve these problems by providing a system that collects and analyzes workers' learning activity data in real time and provides training materials and feedback based on each worker's individual level of understanding.

[0748] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0749] In this invention, the server includes means for receiving authentication information from a student, means for authenticating the student based on the authentication information, means for retrieving selected learning materials from a database, means for transmitting the learning materials to the student's terminal, means for collecting learning activity data of the student from the terminal, means for analyzing the learning activity data and evaluating the student's level of understanding, means for generating feedback based on the evaluation results and transmitting the feedback to the terminal, means for supporting the learning progress of factory workers based on the feedback, and means for providing the factory workers with training materials for learning new processes and machine operations. This enables factory workers to receive effective training tailored to their individual levels of understanding.

[0750] "Students" refers to factory workers who are users, and includes beginners who need to learn new processes or machine operations, as well as existing experts.

[0751] "Authentication Information" refers to information such as ID and password required for a user to access the system.

[0752] "Database" refers to the centralized management system where training materials, user profiles, learning activity data, etc. are stored.

[0753] "Instructional Materials" means educational content provided to factory workers, including video lectures, text materials, interactive quizzes, etc.

[0754] "Devices" refers to devices used by factory workers to view learning materials and carry out learning activities. Examples include computers and tablets.

[0755] "Learning activity data" is data related to the worker's learning activities, including viewing time and quiz answer results.

[0756] "Feedback" refers to information generated based on the analysis of learning activity data, suggesting the next learning step or additional training content.

[0757] "Factory workers" refers to people who operate machines and manage processes at production sites and receive training.

[0758] "Training materials" refers to educational materials provided to factory workers to learn new processes or machine operations.

[0759] The present invention is a system for efficiently providing training to factory workers and tracking and analyzing their progress in real time. An embodiment of this system will be described in detail below.

[0760] System Configuration

[0761] server

[0762] The server has multiple functions and plays a central role in managing and implementing training for factory workers. Specifically, it implements the following functions:

[0763] 1. User Authentication

[0764] The server authenticates the factory worker by checking the information in the database based on the authentication information (ID and password) of the factory worker received from the terminal. If authentication is successful, the server sends the worker's profile data to the terminal.

[0765] 2. Management and distribution of teaching materials

[0766] The server manages the training materials (videos, text, interactive quizzes, etc.) used by factory workers in a database, retrieves the materials requested by the workers from their terminals, and sends them to the terminals.

[0767] 3. Collection and analysis of training data

[0768] The server collects learning activity data of factory workers in real time, including viewing time, quiz answer results, etc. The server analyzes this data and evaluates the worker's level of understanding.

[0769] 4. Generate feedback

[0770] Based on the analysis results, the server generates appropriate feedback, including suggestions for next learning steps and the provision of additional training materials, and sends the generated feedback to the device.

[0771] Terminal

[0772] The terminal is a device that is directly operated by factory workers and provides an interface for training. Specifically, it implements the following functions:

[0773] 1. User authentication and profile management

[0774] The worker enters authentication information on the login screen from their terminal and sends it to the server. After authentication, the learning content begins to be displayed based on the profile data received from the server.

[0775] 2. Display of teaching materials

[0776] The terminals display educational materials sent from the server to the workers, for example, playing video lectures on machine operation, displaying text materials on operating procedures, and providing interactive quizzes.

[0777] 3. Tracking learning activities

[0778] The terminal collects data on the worker's learning activities (such as viewing time and quiz correct answer rate) in real time and sends it to the server.

[0779] 4. Viewing Feedback

[0780] Feedback sent from the server is displayed to the worker, suggesting next learning steps and additional training content to help them continue learning.

[0781] user

[0782] Factory workers can be efficiently trained using this system, specifically through the following process:

[0783] 1. Login and Authentication

[0784] Workers log in to the system by entering their authentication information on a terminal.

[0785] 2. Learning Progression

[0786] Workers can access the learning materials provided on their devices and progress through their studies, watching videos, reading textbooks, and answering quizzes.

[0787] 3. Receive feedback and take next steps

[0788] The worker reviews the feedback displayed on the terminal and executes the suggested next learning steps.

[0789] Specific examples

[0790] For example, when a new worker begins training to operate a new machine, the steps would be as follows:

[0791] 1. Login and Authentication

[0792] The worker logs in by entering their ID and password on the terminal. The server performs authentication and, if successful, obtains their profile data.

[0793] 2. Acquiring and displaying educational materials

[0794] A worker selects a new machine's operating manual. The server retrieves the relevant learning materials (video lectures, text materials, interactive quizzes) and sends them to the terminal. The terminal displays them, and the worker begins learning.

[0795] 3. Learning Progress and Data Collection

[0796] Workers watch videos, read textual materials on operating procedures, and answer quizzes, and their devices send this learning activity data to a server.

[0797] 4. Data analysis and feedback generation

[0798] The server analyzes the collected learning activity data, evaluates the worker's level of understanding, and generates feedback (such as suggestions for the next learning step or additional practice questions) based on the analysis results and sends it to the terminal.

[0799] 5. View feedback and suggest next steps

[0800] The device displays feedback to the worker, who can see what to do next and implement any additional techniques suggested.

[0801] Example prompts for generative AI models

[0802] "Please explain how to implement a factory worker operation training system. Please explain in detail the specific steps, including user authentication, learning material management, learning data collection and analysis, and feedback generation."

[0803] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0804] Step 1:

[0805] A factory worker (hereafter referred to as the user) enters authentication information (ID and password) into a terminal. The terminal receives this information and sends it to the server. The server compares it with user data in a database, and if authentication is successful, it returns profile data to the terminal. The input is the user's ID and password, and the output is the authentication result and profile data.

[0806] Step 2:

[0807] The terminal displays the user's learning dashboard based on the profile data received from the server. This dashboard contains a list of available training materials. The user selects the training material they want to learn. The input is the profile data and the user's material selection, and the output is a display of the selected material.

[0808] Step 3:

[0809] Based on the user's selection, the terminal sends a learning material request to the server. The server retrieves the requested learning material from the database and sends it to the terminal. The input is the learning material request, and the output is the learning material data.

[0810] Step 4:

[0811] The device displays the learning material data (videos, text, quizzes) received from the server to the user. The user progresses through learning by watching or reading the data and answering quizzes. The input is the learning material data, and the output is the user's learning activity data.

[0812] Step 5:

[0813] The device collects the user's learning activity data in real time and sends it to the server. The learning activity data includes viewing time, answer results, correct answer rate, etc. The server receives this data and stores it in a database. The input is the learning activity data, and the output is the stored learning activity data.

[0814] Step 6:

[0815] The server analyzes the saved learning activity data and evaluates the user's level of understanding. Here, the analysis is based on factors such as viewing time and quiz correct answer rate. The input is the learning activity data, and the output is the comprehension evaluation result.

[0816] Step 7:

[0817] The server generates individually customized feedback based on the comprehension assessment results. This feedback may include suggestions for the next learning step or additional practice questions. The generated feedback is sent to the device. The input is the comprehension assessment results, and the output is the feedback.

[0818] Step 8:

[0819] The terminal displays the feedback received from the server to the user, and the user performs the next learning step based on the feedback. The input is the feedback, and the output is the user's continued learning.

[0820] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0821] This invention is a system that provides high-quality education to students around the world, reduces the burden on teachers, and enables individualized instruction according to learning progress. By combining this invention with an emotion engine that recognizes students' emotions, it is possible to provide more precise feedback and adjust learning. This system works in cooperation between the server, the terminal, and the user, and realizes learning support that takes into account the user's emotions through the emotion engine.

[0822] System Configuration

[0823] server

[0824] 1. User Authentication

[0825] The server authenticates the user (student) by checking the information in the database based on the authentication information (ID and password) received from the device. If authentication is successful, the server sends the user's profile data to the device.

[0826] 2. Management and distribution of teaching materials

[0827] The server manages various educational materials (videos, text, interactive quizzes, etc.) stored in a database. It retrieves educational materials requested by users from their devices and sends them to the devices.

[0828] 3. Collection and analysis of training data

[0829] The server collects learning activity data sent from the device, including viewing time, quiz answer results, etc. The server analyzes this data and evaluates the user's level of understanding.

[0830] 4. Generate feedback

[0831] Based on the analysis results, the server generates appropriate feedback, such as suggestions for next learning steps or the provision of individually customized supplementary learning materials, and sends the generated feedback to the device.

[0832] 5. Emotion engine integration and analysis

[0833] The emotion engine analyzes the user's facial expressions, tone of voice, input data, etc. sent from the device during learning, and recognizes the user's emotions. The recognition results are sent to the server, and the feedback content is adjusted based on this.

[0834] Terminal

[0835] 1. User authentication and profile management

[0836] The user enters authentication information on the login screen from the terminal and sends it to the server. After authentication, the system prepares to display the learning content based on the profile data received from the server.

[0837] 2. Display of teaching materials

[0838] The terminal displays the educational materials sent from the server to the user, for example, playing lecture videos in a video player, displaying text materials, and providing interactive quizzes.

[0839] 3. Tracking learning activities

[0840] The terminal collects the user's learning activity data (viewing time, quiz correct answer rate, etc.) and transmits it to the server in real time.

[0841] 4. Viewing Feedback

[0842] Feedback sent from the server is displayed to the user, suggesting next steps and supplementary learning materials to help them continue their learning.

[0843] 5. Collecting and transmitting emotional data

[0844] The device sends the user's facial expressions, tone of voice, input data, etc. to the emotion engine to recognize emotions. The recognition results are then sent to the server.

[0845] User

[0846] 1. Login and Authentication

[0847] The user logs in to the system by entering authentication information from the terminal.

[0848] 2. Learning Progression

[0849] Users can view and learn from the learning materials presented on their devices, including videos, textbooks, and quizzes.

[0850] 3. Receive feedback and take next steps

[0851] The user reviews the feedback displayed on the device and performs the suggested next learning step.

[0852] Specific examples

[0853] For example, consider the case where a junior high school student user begins learning "quadratic equations."

[0854] 1. Login and Authentication

[0855] The user logs in by entering their ID and password on their device. The server performs authentication, and if authentication is successful, the profile data is retrieved.

[0856] 2. Acquiring and displaying educational materials

[0857] The user selects the learning material for "quadratic equations." The server retrieves the relevant learning material (video lectures, text materials, quizzes) from the database and sends it to the terminal. The terminal displays it, and the user begins learning.

[0858] 3. Learning Progress and Data Collection

[0859] Users watch videos, read text materials, and answer quizzes. The device sends learning activity data (such as viewing time and quiz answers) to the server.

[0860] 4. Data analysis and feedback generation

[0861] The server analyzes the collected learning activity data, evaluates the user's level of understanding, and generates feedback (such as suggestions for the next learning step or additional practice questions) based on the analysis results and sends it to the device.

[0862] 5. Emotion recognition and feedback regulation

[0863] The device sends the user's facial expressions and tone of voice to the emotion engine to recognize emotions. For example, if a student is feeling stressed, the emotion engine reports this to the server, which then generates feedback suggesting a break to relax.

[0864] 6. View feedback and suggest next steps

[0865] The device displays feedback to the user, who can then review the next learning step or suggestions for relaxation and continue learning.

[0866] In this way, the system not only provides optimal content according to students' learning progress, but also provides more effective learning support by taking emotions into consideration. The server, devices, and users work together to provide high-quality education through the emotion engine, while reducing the burden on teachers.

[0867] The processing flow will be explained below.

[0868] Step 1:

[0869] The user accesses the login screen from the terminal and enters their ID and password.

[0870] Step 2:

[0871] The terminal transmits the input authentication information to the server.

[0872] Step 3:

[0873] The server compares the received authentication information with information in a database and authenticates the user.

[0874] Step 4:

[0875] If the authentication is successful, the server sends the user's profile data to the terminal.

[0876] Step 5:

[0877] The terminal prepares to display the learning content based on the profile data received from the server.

[0878] Step 6:

[0879] The user selects the learning material they want to study on the screen of their device, for example, "quadratic equations."

[0880] Step 7:

[0881] The terminal requests the selected teaching material information from the server.

[0882] Step 8:

[0883] The server retrieves the relevant teaching materials (video lectures, text materials, quizzes) from the database and sends them to the terminal.

[0884] Step 9:

[0885] The terminal displays the educational materials sent from the server to the user, for example, playing lecture videos on a video player and displaying text materials.

[0886] Step 10:

[0887] Users watch videos, read text materials, and answer interactive quizzes on their devices.

[0888] Step 11:

[0889] The terminal collects the user's learning activity data (viewing time, quiz answers) and transmits it to the server in real time.

[0890] Step 12:

[0891] The device sends the user's facial expressions, tone of voice, input data, etc. that are being learned to the emotion engine to recognize emotions.

[0892] Step 13:

[0893] The emotion engine analyzes the user's facial expressions and tone of voice to understand the user's emotions.

[0894] Step 14:

[0895] The terminal transmits the recognition result of the emotion engine to the server.

[0896] Step 15:

[0897] The server analyzes the learning activity data sent from the device and the recognition results of the emotion engine to evaluate the user's level of understanding and emotional state.

[0898] Step 16:

[0899] The server generates feedback based on the analysis results, for example, if the user is feeling stressed, it generates feedback suggesting a break.

[0900] Step 17:

[0901] The server transmits the generated feedback to the terminal.

[0902] Step 18:

[0903] The terminal displays the feedback received from the server to the user.

[0904] Step 19:

[0905] The user reviews the feedback and takes suggested next learning steps or breaks.

[0906] Through this series of steps, the system provides optimal educational support that takes into account the user's learning progress and emotional state. The server, device, and emotion engine work together to deliver a high-quality learning experience and reduce the burden on teachers.

[0907] Example 2

[0908] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0909] In the current educational system, it is difficult to assess each student's individual learning progress and level of understanding in real time and provide appropriate feedback. Furthermore, there is a lack of learning support that takes students' emotions into consideration, which often leads to a decline in students' motivation to learn. This leads to a decline in the quality of education and an increase in the burden on teachers. A system that efficiently and effectively addresses these challenges is needed.

[0910] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0911] In this invention, the server includes means for receiving authentication information from a student, means for authenticating the student based on the authentication information, means for retrieving selected learning materials from a database, means for transmitting the learning materials to the student's terminal, means for collecting learning activity data of the student from the terminal, means for analyzing the learning activity data and evaluating the student's level of understanding, means for generating feedback based on the evaluation results and transmitting the feedback to the terminal, means for transmitting the student's facial expressions and tone of voice to an emotion engine for analysis, and means for adjusting the feedback content based on the emotion analysis results. This makes it possible to provide feedback according to the student's learning progress and support that takes emotions into consideration.

[0912] "Authentication information" refers to information that uniquely identifies a student, such as the ID and password that a student enters when logging in to the system.

[0913] "Study materials" refers to data such as videos, texts, and quizzes that are provided as learning materials necessary for students to advance their studies.

[0914] "Learning activity data" is a record of the activities that students performed while studying, including information such as viewing time and quiz correct answer rate.

[0915] "Feedback" is information that provides students with advice on their studies and suggests next steps based on the learning activity data analyzed by the server.

[0916] The "emotion engine" is computer software that analyzes students' facial expressions and tone of voice to recognize their emotions at that time.

[0917] "Emotion analysis results" are information about a student's emotional state obtained by the emotion engine by analyzing data such as facial expressions and tone of voice.

[0918] "Devices" are electronic devices used by students, such as computers, tablets, and smartphones.

[0919] The "server" is a central control computer that manages authentication information, distributes learning materials, analyzes learning activity data, generates feedback, and so on.

[0920] "Database" means a data storage system that the server accesses to store learning materials and student information.

[0921] This invention is an educational support system that provides high-quality education to students around the world, reduces the burden on teachers, and enables individualized instruction according to learning progress. This system is composed of three main elements: a server, a terminal, and a user, and realizes learning support that takes into account students' emotions through an emotion engine.

[0922] System Configuration

[0923] server

[0924] The server acts as a central control computer and is responsible for the following major processes:

[0925] 1. User Authentication

[0926] The server receives the authentication information (ID and password) sent by the student and authenticates the student by comparing it with the information in the database. If authentication is successful, the server sends the profile data to the device.

[0927] 2. Management and distribution of teaching materials

[0928] The server retrieves learning materials (videos, texts, quizzes, etc.) from the database and sends the materials selected by the student to the terminal.

[0929] 3. Collection and analysis of training data

[0930] The server collects learning activity data sent from the device, analyzes it to evaluate the level of understanding, and generates feedback based on the analysis results and sends it to the device.

[0931] 4. Emotion engine integration and analysis

[0932] The emotion engine analyzes facial expressions and tone of voice to recognize the student's emotions and sends the results to the server, which then adjusts the feedback content based on the emotion analysis results.

[0933] Terminal

[0934] Devices are electronic devices used by students, such as computers, tablets, and smartphones, and have the following functions:

[0935] 1. User authentication and profile management

[0936] The terminal sends the authentication information entered by the user to the server, and when the authentication result is returned, it displays the learning content based on the profile data.

[0937] 2. Display of teaching materials

[0938] The device displays the learning materials sent from the server and supports learning by, for example, playing lecture videos in a video player, displaying materials in a text viewer, and providing interactive quizzes.

[0939] 3. Tracking learning activities

[0940] The terminal collects students' learning activity data (viewing time, quiz correct answer rate, etc.) in real time and sends it to the server.

[0941] 4. Viewing Feedback

[0942] The device displays the feedback sent from the server and suggests next learning steps and supplementary learning materials.

[0943] 5. Collecting and transmitting emotional data

[0944] The device uses a built-in camera and microphone to transmit the student's facial expressions and tone of voice to the emotion engine, which then recognizes their emotions. The recognition results are then sent to the server.

[0945] User

[0946] The users are students who operate the system. They use the system as follows:

[0947] 1. Login and Authentication

[0948] The user logs in by entering their ID and password into the terminal. The server performs authentication, and if successful, the profile data is displayed on the terminal.

[0949] 2. Learning Progression

[0950] Users can view and learn from the learning materials presented on their devices, including videos, textbooks, and quizzes.

[0951] 3. Receive feedback and take next steps

[0952] The user reviews the feedback displayed on the terminal and performs the suggested next learning step.

[0953] Specific examples

[0954] For example, consider the case where a junior high school student user begins learning "quadratic equations."

[0955] 1. Login and Authentication

[0956] The user logs in by entering their ID and password on the terminal. The server performs authentication, and if authentication is successful, the profile data is acquired.

[0957] 2. Acquiring and displaying educational materials

[0958] The user selects the learning material for "quadratic equations." The server retrieves the corresponding learning material from the database and sends it to the terminal. The terminal displays it and begins learning.

[0959] 3. Learning Progress and Data Collection

[0960] Users watch videos, read text materials, and answer quizzes, and their devices send this learning activity data to a server.

[0961] 4. Data analysis and feedback generation

[0962] The server analyzes the collected learning activity data, evaluates the user's level of understanding, and generates feedback based on the analysis results and sends it to the device.

[0963] 5. Emotion recognition and feedback regulation

[0964] The device sends the user's facial expressions and tone of voice to the emotion engine to recognize emotions. For example, if a student is feeling stressed, the emotion engine reports this to the server and generates feedback suggesting a break to relax.

[0965] 6. View feedback and suggest next steps

[0966] The device displays feedback to the user, who can then review the next learning step or suggestions for relaxation and continue learning.

[0967] Example prompt

[0968] Use the following prompt to provide input to the generative AI model:

[0969] "Based on the following conditions, please explain the process of displaying teaching materials and generating feedback for students learning 'quadratic equations.'"

[0970] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0971] Step 1:

[0972] User Login and Authentication

[0973] The user starts up the device and enters their ID and password on the login screen. The entered authentication information is sent from the device to the server.

[0974] The server compares the received authentication information (ID and password) with the information in the database. If the comparison is successful, the server obtains the profile data and sends it to the device.

[0975] Specifically, the server queries the database to see if the matching ID and password combination exists.

[0976] Input: User ID and password

[0977] Output: Authentication result (if authentication is successful, profile data is sent to the device)

[0978] Step 2:

[0979] Requesting and receiving materials

[0980] The user selects the material they wish to study using the device interface, and this selection is sent as a request from the device to the server.

[0981] The server retrieves the requested learning materials from the database and sends them to the terminal.

[0982] Specifically, the server searches the database for learning materials (videos, texts, quizzes, etc.) that correspond to the specified learning material ID and retrieves them.

[0983] Input: ID of selected teaching material

[0984] Output: Send learning materials (videos, text, quizzes) to your device

[0985] Step 3:

[0986] Learning progression and data collection

[0987] The terminal displays the received learning materials to the user, who then uses them to progress with their learning.

[0988] All user activities (watching videos, reading texts, answering quizzes, etc.) are tracked by the device and sent to the server as learning activity data.

[0989] Specifically, the device collects the start and end times of the study session, quiz scores, and so on.

[0990] Input: User learning activity (video playback, quiz answers, etc.)

[0991] Output: Send learning activity data to the server

[0992] Step 4:

[0993] Analyzing training data and generating feedback

[0994] The server analyzes the collected learning activity data and evaluates the user's level of understanding, using criteria such as viewing time and quiz accuracy.

[0995] Based on the evaluation results, the server generates feedback including suggestions for next learning steps and additional practice questions and sends this to the device.

[0996] Specifically, the server uses an analysis algorithm to assess comprehension and determine appropriate feedback content.

[0997] Input: Learning activity data

[0998] Output: Send generated feedback to the terminal

[0999] Step 5:

[1000] Emotion data collection and analysis

[1001] The device sends the user's facial expressions and tone of voice to the emotion engine during training to recognize emotions, and the recognition results are sent to the server.

[1002] Specifically, the device captures facial expressions with its built-in camera and collects voice tone with its microphone.

[1003] The emotion engine analyzes this data and recognizes the user's emotions, and sends the results to the server.

[1004] Input: User's facial expression data, voice tone data

[1005] Output: Send the sentiment analysis results to the server

[1006] Step 6:

[1007] Adjusting and viewing feedback

[1008] The server adjusts the feedback based on the results of the emotion analysis. For example, if the user is feeling stressed, it generates feedback suggesting a break to relax.

[1009] The device displays the adjusted feedback to the user.

[1010] Specifically, the server dynamically changes the content of the feedback depending on the emotion analysis results.

[1011] Input: Sentiment analysis results

[1012] Output: Sends adjusted feedback to the device

[1013] In this way, the system optimizes the user's learning experience based on a flow that clearly indicates the specific processing and input / output at each step.

[1014] (Application example 2)

[1015] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1016] With the spread of online education, there is a need to provide high-quality education to students around the world. However, it is difficult to respond appropriately to the different levels of understanding and learning progress of each student. Furthermore, there is no support that takes into account emotional changes during learning. Therefore, there is a need for a system that can maximize students' learning effectiveness and reduce the burden on teachers.

[1017] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1018] In this invention, the server includes means for receiving authentication information from the student, means for authenticating the student based on the authentication information, means for retrieving selected learning materials from a database, means for transmitting the learning materials to the student's terminal, means for collecting the student's learning activity data from the terminal, means for analyzing the learning activity data and evaluating the student's level of understanding, means for generating feedback based on the evaluation results and transmitting the feedback to the terminal, means for analyzing the student's facial expressions and tone of voice to recognize emotions, and means for adjusting the feedback content in accordance with the recognized emotions. This enables personalized feedback that takes into account the student's level of understanding and emotional state, thereby improving learning effectiveness.

[1019] "Student" refers to a user of the learning content.

[1020] "Authentication information" refers to information such as ID and password that a user uses to log in to a system.

[1021] A "database" is an information management system for storing educational materials and learning data.

[1022] "Instructional Materials" refers to educational resources such as videos, textbooks, and interactive quizzes provided for learning purposes.

[1023] "Device" refers to a device (smartphone, tablet, PC, etc.) that students use to display and operate learning content.

[1024] "Learning activity data" refers to information about students' activities, such as the amount of time they spend viewing learning materials and the results of quizzes.

[1025] A "server" is a central computing unit that verifies authentication information, manages and distributes learning materials, analyzes learning data, and so on.

[1026] "Analysis" refers to the process of evaluating collected learning activity data and determining students' level of understanding.

[1027] "Feedback" refers to information provided to students based on the assessment results, such as suggestions for next learning steps or the provision of supplementary learning materials.

[1028] "Facial expressions" refers to data that infers emotions based on students' facial movements and expressions.

[1029] "Tone of voice" refers to data that infers emotions based on the way a student speaks, the pitch and intensity of their voice.

[1030] "Emotions" refers to the psychological state of students that can be inferred from their facial expressions and tone of voice.

[1031] An "emotion engine" is software or hardware that analyzes students' facial expressions and tone of voice to recognize their emotions.

[1032] "Personalization" means providing support that is individually tailored to each student's learning progress and emotional state.

[1033] System Overview

[1034] The system that realizes this application example provides high-quality education by linking the server, terminals, and users, and collecting, analyzing, and providing feedback on students' learning activity and emotional data. In particular, by combining it with an emotion engine, it is possible to recognize students' emotional states in real time and provide appropriate feedback according to their learning situation.

[1035] Hardware and Software Configuration

[1036] Server: A central computing unit that manages databases, authenticates users, analyzes data, and generates feedback. For example, a cloud server such as AWS or Google Cloud Platform can be used.

[1037] Device: A device on which students view and interact with learning content. Examples include smartphones, tablets, and PCs. This device is equipped with a camera and microphone, which are used to collect emotional data.

[1038] Emotion engine: Software that analyzes students' facial expressions and tone of voice to recognize their emotions. For example, libraries such as OpenCV and Google Cloud Vision AI can be used.

[1039] Processing steps

[1040] 1. User authentication: The user enters authentication information (ID and password) on the device and sends it to the server. The server verifies the authentication information, authenticates the user, and sends profile data to the device.

[1041] 2. Acquisition of learning materials: The server acquires the requested learning materials from the database and sends them to the user's terminal.

[1042] 3. Collection of learning activity data: The terminal collects learning activity data such as the user's viewing time and quiz correct answer rate, and transmits it to the server in real time.

[1043] 4. Data analysis: The server analyzes the collected learning activity data and evaluates the user's level of understanding.

[1044] 5. Feedback generation: Based on the evaluation results, feedback is generated suggesting next learning steps and supplementary learning materials, and this is sent to the device.

[1045] 6. Emotion data collection and analysis: The device sends the user's facial expressions and tone of voice during training via a camera and microphone to the emotion engine to recognize emotions. The server receives the recognition results and adjusts the feedback content.

[1046] Specific examples

[1047] For example, below is a specific scenario in which a junior high school student is learning about quadratic equations.

[1048] 1. The user logs into the app on their smartphone and selects the quadratic equations study material.

[1049] 2. The server sends the teaching materials (video lectures, text materials, quizzes, etc.) to the terminal.

[1050] 3. The user watches videos and answers quizzes. The device collects this data and sends it to the server.

[1051] 4. The server analyzes the learning activity data and evaluates the user's level of understanding.

[1052] 5. If the emotion engine recognizes "confusion" from the user's facial expression while watching a movie, the server generates feedback including supplementary explanations and sends it to the device.

[1053] 6. Furthermore, if the user is feeling "stressed," suggestions such as "take a five-minute break" will be made.

[1054] Example prompt sentence:

[1055] "My students are stressed. What are some ways to help them relax?"

[1056] "A student is confused. Please provide additional explanation for quadratic equations."

[1057] This allows for personalized feedback that takes into account the student's level of understanding and emotional state, maximizing learning outcomes while providing high-quality education.

[1058] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1059] Step 1: User authentication

[1060] The user enters authentication information (ID and password) from the terminal and sends it to the server, which collates this authentication information with information in a database and authenticates the user.

[1061] Input: ID, password

[1062] Output: Authentication result (profile data if successful)

[1063] Operation: The ID and password entered by the user are sent to the server, which then authenticates them by checking them against the authentication information in the database. If authentication is successful, the user's profile data is sent to the device.

[1064] Step 2: Acquire the study materials

[1065] The server retrieves the educational material requested by the user from the database and transmits it to the terminal.

[1066] Input: User request (e.g., teaching material ID)

[1067] Output: Teaching material data (videos, text, quizzes, etc.)

[1068] Operation: When a user sends a requested educational material ID from a terminal to the server, the server retrieves the corresponding educational material from the database and sends it to the terminal.

[1069] Step 3: Collect learning activity data

[1070] The terminal collects the user's viewing time, quiz correct answer rate, etc. in real time and transmits them to the server.

[1071] Input: Learning activity data (viewing time, quiz answer results, etc.)

[1072] Output: Sending learning activity data to the server

[1073] Operation: When a user watches or listens to educational materials or answers quizzes, their activity data is collected on the device and sent to the server in real time.

[1074] Step 4: Data analysis

[1075] The server analyzes the collected learning activity data and evaluates the user's level of understanding.

[1076] Input: Learning activity data

[1077] Output: Comprehension assessment results

[1078] Operation: The server analyzes learning activity data and performs data calculations using statistical analysis and machine learning algorithms to evaluate the user's level of understanding.

[1079] Step 5: Generate feedback

[1080] Based on the evaluation results, the server generates feedback suggesting the next learning step or supplementary learning materials and sends this to the terminal.

[1081] Input: Comprehension assessment results

[1082] Output: Feedback (suggestions for next learning steps, supplementary materials, etc.)

[1083] Operation: Based on the comprehension assessment results, the server uses a generative AI model to generate feedback suggesting next steps and supplementary learning materials. The generated feedback is then sent to the device.

[1084] Step 6: Collect and analyze emotion data

[1085] The device sends the user's facial expressions and tone of voice during training via a camera and microphone to the emotion engine to recognize emotions.

[1086] Input: facial expression data, tone of voice

[1087] Output: Emotion recognition result

[1088] How it works: Facial expression data and voice tone captured by the device's camera and microphone are sent to the emotion engine, which then analyzes the data and recognizes emotions. The recognized emotion results are then sent to the server.

[1089] Step 7: Adjusting Emotional Feedback

[1090] The server adjusts the feedback content based on the emotion recognition results and sends it to the terminal.

[1091] Input: Emotion recognition results

[1092] Output: Regulated Feedback

[1093] How it works: The server receives the emotion recognition results and adjusts the feedback content according to the recognized emotion. For example, if the student is tired, it suggests taking a break. The adjusted feedback is then sent to the device.

[1094] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1095] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1096] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1097] [Third embodiment]

[1098] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1099] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1100] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1101] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1102] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1103] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1104] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1105] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1106] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1107] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1108] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1109] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1110] This invention is a system that provides high-quality education to students around the world, reduces the burden on teachers, and enables individualized instruction according to learning progress. This system is realized through cooperation between a server, terminals, and users.

[1111] System Configuration

[1112] server

[1113] 1. User Authentication

[1114] The server authenticates the user (student) by checking the information in the database based on the authentication information (ID and password) received from the device. If authentication is successful, the server sends the user's profile data to the device.

[1115] 2. Management and distribution of teaching materials

[1116] The server manages various educational materials (videos, texts, interactive quizzes, etc.) stored in a database, retrieves the educational materials requested by the user from the device, and sends them to the device.

[1117] 3. Collection and analysis of training data

[1118] The server collects learning activity data sent from the device, including viewing time, quiz answer results, etc. The server analyzes this data and evaluates the user's level of understanding.

[1119] 4. Generate feedback

[1120] Based on the analysis results, the server generates appropriate feedback, such as suggestions for next learning steps or the provision of individually customized supplementary learning materials, and sends the generated feedback to the device.

[1121] Terminal

[1122] 1. User authentication and profile management

[1123] The user enters authentication information on the login screen from their device and sends it to the server. After authentication, the learning content begins to be displayed based on the profile data received from the server.

[1124] 2. Display of teaching materials

[1125] The device displays the educational material sent from the server to the user, for example, playing video lectures, displaying text materials, and providing interactive quizzes.

[1126] 3. Tracking learning activities

[1127] The device transmits the user's learning activity data (viewing time, quiz correct answer rate, etc.) to the server in real time.

[1128] 4. Viewing Feedback

[1129] Feedback sent from the server is displayed to the user, suggesting next steps and supplementary learning materials to help them continue their learning.

[1130] User

[1131] 1. Login and Authentication

[1132] Users log in to the system by entering their authentication information on a terminal.

[1133] 2. Learning Progression

[1134] Users can view and learn from the learning materials presented on their devices, including videos, textbooks, and quizzes.

[1135] 3. Receive feedback and take next steps

[1136] The user reviews the feedback displayed on the device and takes the suggested next learning steps.

[1137] Specific examples

[1138] For example, consider the case where a junior high school student user begins learning "quadratic equations."

[1139] 1. Login and Authentication

[1140] The user logs in by entering their ID and password on their device. The server performs authentication, and if authentication is successful, profile data is obtained.

[1141] 2. Acquiring and displaying educational materials

[1142] The user selects the learning material for "quadratic equations." The server retrieves the relevant learning material (video lectures, text materials, quizzes) from the database and sends it to the device. The device displays it, and the user begins learning.

[1143] 3. Learning Progress and Data Collection

[1144] Users watch videos, read text materials, and answer quizzes, and their devices send learning activity data (such as viewing time and quiz answers) to a server.

[1145] 4. Data analysis and feedback generation

[1146] The server analyzes the collected learning activity data, evaluates the user's level of understanding, and generates feedback (such as suggestions for the next learning step or additional practice questions) based on the analysis results and sends it to the device.

[1147] 5. View feedback and suggest next steps

[1148] The device displays feedback to the user, who can see what to work on next and complete suggested additional exercises.

[1149] In this way, the system can provide optimal education according to students' learning progress, reduce the burden on teachers, and improve the quality of education.

[1150] The processing flow will be explained below.

[1151] Step 1:

[1152] The user accesses the login screen from the terminal and enters their ID and password.

[1153] Step 2:

[1154] The terminal transmits the input authentication information to the server.

[1155] Step 3:

[1156] The server compares the received authentication information with information in a database and authenticates the user.

[1157] Step 4:

[1158] If the authentication is successful, the server sends the user's profile data to the terminal.

[1159] Step 5:

[1160] The terminal prepares to display the learning content based on the profile data received from the server.

[1161] Step 6:

[1162] The user selects the learning material they want to study on the screen of their device, for example, "quadratic equations."

[1163] Step 7:

[1164] The terminal requests the selected teaching material information from the server.

[1165] Step 8:

[1166] The server retrieves the relevant teaching materials (video lectures, text materials, quizzes) from the database and sends them to the terminal.

[1167] Step 9:

[1168] The terminal displays the educational materials sent from the server to the user, for example, playing lecture videos on a video player and displaying text materials.

[1169] Step 10:

[1170] Users watch videos, read text materials, and answer interactive quizzes on their devices.

[1171] Step 11:

[1172] The terminal collects the user's learning activity data (viewing time, quiz answers) and transmits it to the server in real time.

[1173] Step 12:

[1174] The server analyzes the learning activity data sent from the terminal and evaluates the user's level of understanding.

[1175] Step 13:

[1176] The server generates feedback based on the analysis results, such as "Your understanding of quadratic equations is insufficient; we recommend you do some additional practice problems."

[1177] Step 14:

[1178] The server transmits the generated feedback to the terminal.

[1179] Step 15:

[1180] The terminal displays the feedback received from the server to the user.

[1181] Step 16:

[1182] The user selects the next learning step based on the feedback and continues learning.

[1183] Through this series of steps, users are provided with the most appropriate content based on their learning progress. The server, devices, and users work together to provide high-quality education while reducing the burden on teachers.

[1184] Example 1

[1185] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1186] In conventional educational systems, it was difficult to provide optimal learning materials and feedback based on each student's individual learning progress. This also placed a heavy burden on teachers, making it difficult to provide high-quality education to all students. In particular, there was a lack of technological means to analyze students' learning data in real time and suggest personalized learning steps based on the results.

[1187] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1188] In this invention, the server includes means for receiving authentication information from the student, means for authenticating the student based on the authentication information, means for retrieving selected learning materials from a database, means for transmitting the learning materials to the student's terminal, means for collecting the student's learning activity data from the terminal, means for analyzing the learning activity data and evaluating the student's level of understanding, means for transmitting generated feedback to the terminal, means for evaluating the student's level of understanding using a generative AI model based on the student's learning progress data, and means for customizing suggestions for the next learning step based on the evaluation and presenting appropriate learning content using prompt sentences. This makes it possible to provide optimal learning materials and feedback according to each student's learning progress, thereby reducing the burden on teachers and providing high-quality education.

[1189] "Authentication information" refers to information used to log in to the system, such as a student ID and password.

[1190] A "database" is a system that centrally manages and stores information such as teaching materials, learning activity data, and user profiles.

[1191] "Devices" are devices used by students, such as computers, tablets, and smartphones.

[1192] "Learning activity data" refers to data related to the learning process, such as students' viewing time, quiz answer results, and text material viewing history.

[1193] "Comprehension" is an assessment of how well a student understands a particular learning material or topic.

[1194] "Feedback" refers to guidance and advice provided based on learning activity data and comprehension analysis results.

[1195] A "generative AI model" is a model that uses machine learning algorithms to extract patterns and trends from learning data and assess students' understanding.

[1196] A "prompt" is text that expresses instructions or questions that are input to a generative AI model.

[1197] "Personalization" means providing individually optimized educational content and feedback based on each student's learning progress and level of understanding.

[1198] A "learning step" is a specific learning activity or progress that a student should undertake next.

[1199] This invention provides an educational system that provides individualized instruction according to students' learning progress and reduces the burden on teachers. The system is realized through cooperation between a server, a terminal, and a user. The details are described below.

[1200] Server configuration and operation

[1201] The server performs the following main functions:

[1202] User authentication: The server authenticates the user (student) by comparing the authentication information (ID and password) received from the device with information in the database. If authentication is successful, the server sends the user's profile data to the device. At this time, a JWT (JSON Web Token) is used as the authentication token.

[1203] Management and distribution of learning materials: The server manages learning materials (videos, texts, quizzes, etc.) stored in a database. It retrieves learning materials requested by users from their devices and sends them to the devices. For example, learning materials are stored in a MySQL database.

[1204] Collection and analysis of learning data: The server collects learning activity data sent from the device. This data includes viewing time, quiz answer results, etc. The server uses the Python scikit-learn library to train a generative AI model and analyze the data.

[1205] Feedback Generation: Based on the analysis results, the server generates appropriate feedback, including suggesting next learning steps and providing personalized learning materials.

[1206] Terminal configuration and operation

[1207] The terminal performs the following main functions:

[1208] User authentication and profile management: The user enters authentication information on the login screen from the device and sends it to the server. After authentication, the learning content begins to be displayed based on the profile data received from the server.

[1209] Displaying educational materials: The device displays educational materials sent from the server to the user. For example, a YouTube video player can be embedded to play video lectures.

[1210] Learning activity tracking: The device sends the user's learning activity data (viewing time, quiz correct answer rate, etc.) to the server in real time. This data is accumulated on the server and used for analysis.

[1211] Show feedback: Show feedback sent by the server to the user, for example, "It seems you don't understand the basics of quadratic equations. Please study the supplementary material below."

[1212] User Actions

[1213] The user performs the following main actions:

[1214] Login and authentication: The user logs in by entering their ID and password on the terminal. For example, they enter "student123" and "password!" and press the login button.

[1215] Learning Progress: Users can view and learn the materials presented on their devices. They can watch videos, read text materials, and answer quizzes.

[1216] Receive feedback and take next steps: Users review the feedback displayed on their device and take the suggested next learning step. For example, they can click on "Additional learning materials for quadratic equations" to begin additional learning.

[1217] Specific examples

[1218] For example, consider a case where a junior high school student user begins studying "quadratic equations." The user logs in by entering their ID and password on their device. If authentication is successful, profile data is retrieved from the server. Next, the user selects learning materials for "quadratic equations," and the server retrieves the relevant materials from the database and sends them to the device. The user watches videos, reads text materials, and answers quizzes. The device sends this learning activity data to the server. The server analyzes the collected data and evaluates the level of understanding using a generative AI model. Feedback is generated and the next learning step is suggested. The user continues studying according to this feedback.

[1219] Prompt Sentence Examples

[1220] "How can we use generative AI models to tailor optimal feedback to users based on their learning progress?"

[1221] In this way, the system of the present invention provides optimal education according to the student's learning progress, enabling the student to progress in learning efficiently.

[1222] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1223] Step 1: Enter and submit your credentials

[1224] User: The user enters their ID and password on their device and clicks the login button. For example, they enter "student123" and "password!"

[1225] Input: User ID and password.

[1226] Terminal: The terminal sends the entered authentication information to the server.

[1227] Output: Sending authentication information to the server.

[1228] Step 2: User authentication

[1229] Server: The server checks the received authentication information against a database and authenticates the user.

[1230] Input: Authentication information sent from the device (user ID and password).

[1231] Data manipulation: Using an SQL query, the authentication information is matched against the user table in the database.

[1232] Output: Authentication result (success or failure) and profile data.

[1233] Specific operation: Connect to the MySQL database and search the user table. If authentication is successful, obtain profile data and send it to the device. If authentication fails, send an error message.

[1234] Step 3: View your profile data

[1235] Terminal: The terminal saves the profile data received from the server and transitions to the learning screen.

[1236] Input: Profile data from the server.

[1237] Output: Transition to the learning screen after logging in.

[1238] Specific operation: Upon successful authentication, a dashboard showing the user's specific learning progress and history will be displayed.

[1239] Step 4: Request and submit materials

[1240] User: Select the material you want to learn on the learning screen. For example, click "Math > Quadratic Equations."

[1241] Input: The ID of the selected material.

[1242] Terminal: Sends a request including the learning material ID to the server.

[1243] Output: Sending a learning material request to the server.

[1244] Step 5: Acquire and distribute materials

[1245] Server: Receives the learning material request and retrieves the corresponding learning material data from the database.

[1246] Input: Teaching material request from the terminal (teaching material ID).

[1247] Data processing: Retrieve teaching material data from the database using SQL queries.

[1248] Output: Sending teaching material data to the terminal.

[1249] Specific operation: The relevant video URL, text material, quiz, and other data are read from the database and sent to the device.

[1250] Step 6: Displaying the materials

[1251] Terminal: The terminal receives the teaching material data sent from the server and displays it to the user.

[1252] Input: Teaching material data from the server (video URL, text materials, quizzes).

[1253] Output: Display of teaching materials.

[1254] What it does: Embeds a video player using YouTube's API, plays videos, displays text material, and starts an interactive quiz.

[1255] Step 7: Tracking learning activities

[1256] Terminal: Sends user learning activity data (viewing time, quiz answer results, etc.) to the server in real time.

[1257] Input: User learning activity data.

[1258] Data processing: Collect various activity data (e.g., start time, end time, correct answer rate) and convert them into an appropriate format.

[1259] Output: Sending learning activity data to the server.

[1260] Specific operation: Generate a log of learning activities and periodically send it to the server.

[1261] Step 8: Collect and analyze training data

[1262] Server: Analyzes the collected data and evaluates comprehension using a generative AI model.

[1263] Input: Learning activity data sent from the device.

[1264] Data processing: Preprocess the data using Python's scikit-learn library and analyze it using a generative AI model.

[1265] Output: Generate comprehension assessment results and feedback.

[1266] Specific operation: Learning activity data is fed into an AI model to generate performance metrics (e.g., grades, comprehension scores).

[1267] Step 9: Generate feedback

[1268] Server: Generates feedback based on the analysis results, including suggesting next learning steps and providing personalized learning materials.

[1269] Input: Comprehension assessment results and output of the generative AI model.

[1270] Data processing: Generating feedback messages and selecting the next teaching materials.

[1271] Output: Sending feedback information to a terminal.

[1272] What it does: Based on a specific comprehension score, generate links to additional practice questions or supplemental materials and send them to your device.

[1273] Step 10: View feedback and take next steps

[1274] Terminal: The terminal displays the feedback received from the server to the user.

[1275] Input: Feedback information from the server.

[1276] Output: Displaying feedback to the user and suggesting next learning steps.

[1277] What it does: A dialog box appears on the screen, showing the next learning step and links to additional learning materials. It tells the user to click on "Additional learning materials for quadratic equations."

[1278] Example prompt

[1279] "How can we use generative AI models to tailor optimal feedback to users based on their learning progress?"

[1280] Through the above process, this system can provide optimal education according to students' learning progress, realizing an efficient and personalized learning experience.

[1281] (Application example 1)

[1282] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1283] Modern factories require efficient and effective training for new workers and existing workers learning new processes. However, conventional methods have difficulty providing education and training tailored to each worker's individual level of understanding, making it difficult to contribute to improving overall production efficiency and safety. The present invention aims to solve these problems by providing a system that collects and analyzes workers' learning activity data in real time and provides training materials and feedback based on each worker's individual level of understanding.

[1284] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1285] In this invention, the server includes means for receiving authentication information from a student, means for authenticating the student based on the authentication information, means for retrieving selected learning materials from a database, means for transmitting the learning materials to the student's terminal, means for collecting learning activity data of the student from the terminal, means for analyzing the learning activity data and evaluating the student's level of understanding, means for generating feedback based on the evaluation results and transmitting the feedback to the terminal, means for supporting the learning progress of factory workers based on the feedback, and means for providing the factory workers with training materials for learning new processes and machine operations. This enables factory workers to receive effective training tailored to their individual levels of understanding.

[1286] "Students" refers to factory workers who are users, and includes beginners who need to learn new processes or machine operations, as well as existing experts.

[1287] "Authentication Information" refers to information such as ID and password required for a user to access the system.

[1288] "Database" refers to the centralized management system where training materials, user profiles, learning activity data, etc. are stored.

[1289] "Instructional Materials" means educational content provided to factory workers, including video lectures, text materials, interactive quizzes, etc.

[1290] "Devices" refers to devices used by factory workers to view learning materials and carry out learning activities. Examples include computers and tablets.

[1291] "Learning activity data" is data related to the worker's learning activities, including viewing time and quiz answer results.

[1292] "Feedback" refers to information generated based on the analysis of learning activity data, suggesting the next learning step or additional training content.

[1293] "Factory workers" refers to people who operate machines and manage processes at production sites and receive training.

[1294] "Training materials" refers to educational materials provided to factory workers to learn new processes or machine operations.

[1295] The present invention is a system for efficiently providing training to factory workers and tracking and analyzing their progress in real time. An embodiment of this system will be described in detail below.

[1296] System Configuration

[1297] server

[1298] The server has multiple functions and plays a central role in managing and implementing training for factory workers. Specifically, it implements the following functions:

[1299] 1. User Authentication

[1300] The server authenticates the factory worker by checking the information in the database based on the authentication information (ID and password) of the factory worker received from the terminal. If authentication is successful, the server sends the worker's profile data to the terminal.

[1301] 2. Management and distribution of teaching materials

[1302] The server manages the training materials (videos, text, interactive quizzes, etc.) used by factory workers in a database, retrieves the materials requested by the workers from their terminals, and sends them to the terminals.

[1303] 3. Collection and analysis of training data

[1304] The server collects learning activity data of factory workers in real time, including viewing time, quiz answer results, etc. The server analyzes this data and evaluates the worker's level of understanding.

[1305] 4. Generate feedback

[1306] Based on the analysis results, the server generates appropriate feedback, including suggestions for next learning steps and the provision of additional training materials, and sends the generated feedback to the device.

[1307] Terminal

[1308] The terminal is a device that is directly operated by factory workers and provides an interface for training. Specifically, it implements the following functions:

[1309] 1. User authentication and profile management

[1310] The worker enters authentication information on the login screen from their terminal and sends it to the server. After authentication, the learning content begins to be displayed based on the profile data received from the server.

[1311] 2. Display of teaching materials

[1312] The terminals display educational materials sent from the server to the workers, for example, playing video lectures on machine operation, displaying text materials on operating procedures, and providing interactive quizzes.

[1313] 3. Tracking learning activities

[1314] The terminal collects data on the worker's learning activities (such as viewing time and quiz correct answer rate) in real time and sends it to the server.

[1315] 4. Viewing Feedback

[1316] Feedback sent from the server is displayed to the worker, suggesting next learning steps and additional training content to help them continue learning.

[1317] user

[1318] Factory workers can be efficiently trained using this system, specifically through the following process:

[1319] 1. Login and Authentication

[1320] Workers log in to the system by entering their authentication information on a terminal.

[1321] 2. Learning Progression

[1322] Workers can access the learning materials provided on their devices and progress through their studies, watching videos, reading textbooks, and answering quizzes.

[1323] 3. Receive feedback and take next steps

[1324] The worker reviews the feedback displayed on the terminal and executes the suggested next learning steps.

[1325] Specific examples

[1326] For example, when a new worker begins training to operate a new machine, the steps would be as follows:

[1327] 1. Login and Authentication

[1328] The worker logs in by entering their ID and password on the terminal. The server performs authentication and, if successful, obtains their profile data.

[1329] 2. Acquiring and displaying educational materials

[1330] A worker selects a new machine's operating manual. The server retrieves the relevant learning materials (video lectures, text materials, interactive quizzes) and sends them to the terminal. The terminal displays them, and the worker begins learning.

[1331] 3. Learning Progress and Data Collection

[1332] Workers watch videos, read textual materials on operating procedures, and answer quizzes, and their devices send this learning activity data to a server.

[1333] 4. Data analysis and feedback generation

[1334] The server analyzes the collected learning activity data, evaluates the worker's level of understanding, and generates feedback (such as suggestions for the next learning step or additional practice questions) based on the analysis results and sends it to the terminal.

[1335] 5. View feedback and suggest next steps

[1336] The device displays feedback to the worker, who can see what to do next and implement any additional techniques suggested.

[1337] Example prompts for generative AI models

[1338] "Please explain how to implement a factory worker operation training system. Please explain in detail the specific steps, including user authentication, learning material management, learning data collection and analysis, and feedback generation."

[1339] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1340] Step 1:

[1341] A factory worker (hereafter referred to as the user) enters authentication information (ID and password) into a terminal. The terminal receives this information and sends it to the server. The server compares it with user data in a database, and if authentication is successful, it returns profile data to the terminal. The input is the user's ID and password, and the output is the authentication result and profile data.

[1342] Step 2:

[1343] The terminal displays the user's learning dashboard based on the profile data received from the server. This dashboard contains a list of available training materials. The user selects the training material they want to learn. The input is the profile data and the user's material selection, and the output is a display of the selected material.

[1344] Step 3:

[1345] Based on the user's selection, the terminal sends a learning material request to the server. The server retrieves the requested learning material from the database and sends it to the terminal. The input is the learning material request, and the output is the learning material data.

[1346] Step 4:

[1347] The device displays the learning material data (videos, text, quizzes) received from the server to the user. The user progresses through learning by watching or reading the data and answering quizzes. The input is the learning material data, and the output is the user's learning activity data.

[1348] Step 5:

[1349] The device collects the user's learning activity data in real time and sends it to the server. The learning activity data includes viewing time, answer results, correct answer rate, etc. The server receives this data and stores it in a database. The input is the learning activity data, and the output is the stored learning activity data.

[1350] Step 6:

[1351] The server analyzes the saved learning activity data and evaluates the user's level of understanding. Here, the analysis is based on factors such as viewing time and quiz correct answer rate. The input is the learning activity data, and the output is the comprehension evaluation result.

[1352] Step 7:

[1353] The server generates individually customized feedback based on the comprehension assessment results. This feedback may include suggestions for the next learning step or additional practice questions. The generated feedback is sent to the device. The input is the comprehension assessment results, and the output is the feedback.

[1354] Step 8:

[1355] The terminal displays the feedback received from the server to the user, and the user performs the next learning step based on the feedback. The input is the feedback, and the output is the user's continued learning.

[1356] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1357] This invention is a system that provides high-quality education to students around the world, reduces the burden on teachers, and enables individualized instruction according to learning progress. By combining this invention with an emotion engine that recognizes students' emotions, it is possible to provide more precise feedback and adjust learning. This system works in cooperation between the server, the terminal, and the user, and realizes learning support that takes into account the user's emotions through the emotion engine.

[1358] System Configuration

[1359] server

[1360] 1. User Authentication

[1361] The server authenticates the user (student) by checking the information in the database based on the authentication information (ID and password) received from the device. If authentication is successful, the server sends the user's profile data to the device.

[1362] 2. Management and distribution of teaching materials

[1363] The server manages various educational materials (videos, text, interactive quizzes, etc.) stored in a database. It retrieves educational materials requested by users from their devices and sends them to the devices.

[1364] 3. Collection and analysis of training data

[1365] The server collects learning activity data sent from the device, including viewing time, quiz answer results, etc. The server analyzes this data and evaluates the user's level of understanding.

[1366] 4. Generate feedback

[1367] Based on the analysis results, the server generates appropriate feedback, such as suggestions for next learning steps or the provision of individually customized supplementary learning materials, and sends the generated feedback to the device.

[1368] 5. Emotion engine integration and analysis

[1369] The emotion engine analyzes the user's facial expressions, tone of voice, input data, etc. sent from the device during learning, and recognizes the user's emotions. The recognition results are sent to the server, and the feedback content is adjusted based on this.

[1370] Terminal

[1371] 1. User authentication and profile management

[1372] The user enters authentication information on the login screen from the terminal and sends it to the server. After authentication, the system prepares to display the learning content based on the profile data received from the server.

[1373] 2. Display of teaching materials

[1374] The terminal displays the educational materials sent from the server to the user, for example, playing lecture videos in a video player, displaying text materials, and providing interactive quizzes.

[1375] 3. Tracking learning activities

[1376] The terminal collects the user's learning activity data (viewing time, quiz correct answer rate, etc.) and transmits it to the server in real time.

[1377] 4. Viewing Feedback

[1378] Feedback sent from the server is displayed to the user, suggesting next steps and supplementary learning materials to help them continue their learning.

[1379] 5. Collecting and transmitting emotional data

[1380] The device sends the user's facial expressions, tone of voice, input data, etc. to the emotion engine to recognize emotions. The recognition results are then sent to the server.

[1381] User

[1382] 1. Login and Authentication

[1383] The user logs in to the system by entering authentication information from the terminal.

[1384] 2. Learning Progression

[1385] Users can view and learn from the learning materials presented on their devices, including videos, textbooks, and quizzes.

[1386] 3. Receive feedback and take next steps

[1387] The user reviews the feedback displayed on the device and performs the suggested next learning step.

[1388] Specific examples

[1389] For example, consider the case where a junior high school student user begins learning "quadratic equations."

[1390] 1. Login and Authentication

[1391] The user logs in by entering their ID and password on their device. The server performs authentication, and if authentication is successful, the profile data is retrieved.

[1392] 2. Acquiring and displaying educational materials

[1393] The user selects the learning material for "quadratic equations." The server retrieves the relevant learning material (video lectures, text materials, quizzes) from the database and sends it to the terminal. The terminal displays it, and the user begins learning.

[1394] 3. Learning Progress and Data Collection

[1395] Users watch videos, read text materials, and answer quizzes. The device sends learning activity data (such as viewing time and quiz answers) to the server.

[1396] 4. Data analysis and feedback generation

[1397] The server analyzes the collected learning activity data, evaluates the user's level of understanding, and generates feedback (such as suggestions for the next learning step or additional practice questions) based on the analysis results and sends it to the device.

[1398] 5. Emotion recognition and feedback regulation

[1399] The device sends the user's facial expressions and tone of voice to the emotion engine to recognize emotions. For example, if a student is feeling stressed, the emotion engine reports this to the server, which then generates feedback suggesting a break to relax.

[1400] 6. View feedback and suggest next steps

[1401] The device displays feedback to the user, who can then review the next learning step or suggestions for relaxation and continue learning.

[1402] In this way, the system not only provides optimal content according to students' learning progress, but also provides more effective learning support by taking emotions into consideration. The server, devices, and users work together to provide high-quality education through the emotion engine, while reducing the burden on teachers.

[1403] The processing flow will be explained below.

[1404] Step 1:

[1405] The user accesses the login screen from the terminal and enters their ID and password.

[1406] Step 2:

[1407] The terminal transmits the input authentication information to the server.

[1408] Step 3:

[1409] The server compares the received authentication information with information in a database and authenticates the user.

[1410] Step 4:

[1411] If the authentication is successful, the server sends the user's profile data to the terminal.

[1412] Step 5:

[1413] The terminal prepares to display the learning content based on the profile data received from the server.

[1414] Step 6:

[1415] The user selects the learning material they want to study on the screen of their device, for example, "quadratic equations."

[1416] Step 7:

[1417] The terminal requests the selected teaching material information from the server.

[1418] Step 8:

[1419] The server retrieves the relevant teaching materials (video lectures, text materials, quizzes) from the database and sends them to the terminal.

[1420] Step 9:

[1421] The terminal displays the educational materials sent from the server to the user, for example, playing lecture videos on a video player and displaying text materials.

[1422] Step 10:

[1423] Users watch videos, read text materials, and answer interactive quizzes on their devices.

[1424] Step 11:

[1425] The terminal collects the user's learning activity data (viewing time, quiz answers) and transmits it to the server in real time.

[1426] Step 12:

[1427] The device sends the user's facial expressions, tone of voice, input data, etc. that are being learned to the emotion engine to recognize emotions.

[1428] Step 13:

[1429] The emotion engine analyzes the user's facial expressions and tone of voice to understand the user's emotions.

[1430] Step 14:

[1431] The terminal transmits the recognition result of the emotion engine to the server.

[1432] Step 15:

[1433] The server analyzes the learning activity data sent from the device and the recognition results of the emotion engine to evaluate the user's level of understanding and emotional state.

[1434] Step 16:

[1435] The server generates feedback based on the analysis results, for example, if the user is feeling stressed, it generates feedback suggesting a break.

[1436] Step 17:

[1437] The server transmits the generated feedback to the terminal.

[1438] Step 18:

[1439] The terminal displays the feedback received from the server to the user.

[1440] Step 19:

[1441] The user reviews the feedback and takes suggested next learning steps or breaks.

[1442] Through this series of steps, the system provides optimal educational support that takes into account the user's learning progress and emotional state. The server, device, and emotion engine work together to deliver a high-quality learning experience and reduce the burden on teachers.

[1443] Example 2

[1444] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1445] In the current educational system, it is difficult to assess each student's individual learning progress and level of understanding in real time and provide appropriate feedback. Furthermore, there is a lack of learning support that takes students' emotions into consideration, which often leads to a decline in students' motivation to learn. This leads to a decline in the quality of education and an increase in the burden on teachers. A system that efficiently and effectively addresses these challenges is needed.

[1446] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1447] In this invention, the server includes means for receiving authentication information from a student, means for authenticating the student based on the authentication information, means for retrieving selected learning materials from a database, means for transmitting the learning materials to the student's terminal, means for collecting learning activity data of the student from the terminal, means for analyzing the learning activity data and evaluating the student's level of understanding, means for generating feedback based on the evaluation results and transmitting the feedback to the terminal, means for transmitting the student's facial expressions and tone of voice to an emotion engine for analysis, and means for adjusting the feedback content based on the emotion analysis results. This makes it possible to provide feedback according to the student's learning progress and support that takes emotions into consideration.

[1448] "Authentication information" refers to information that uniquely identifies a student, such as the ID and password that a student enters when logging in to the system.

[1449] "Study materials" refers to data such as videos, texts, and quizzes that are provided as learning materials necessary for students to advance their studies.

[1450] "Learning activity data" is a record of the activities that students performed while studying, including information such as viewing time and quiz correct answer rate.

[1451] "Feedback" is information that provides students with advice on their studies and suggests next steps based on the learning activity data analyzed by the server.

[1452] The "emotion engine" is computer software that analyzes students' facial expressions and tone of voice to recognize their emotions at that time.

[1453] "Emotion analysis results" are information about a student's emotional state obtained by the emotion engine by analyzing data such as facial expressions and tone of voice.

[1454] "Devices" are electronic devices used by students, such as computers, tablets, and smartphones.

[1455] The "server" is a central control computer that manages authentication information, distributes learning materials, analyzes learning activity data, generates feedback, and so on.

[1456] "Database" means a data storage system that the server accesses to store learning materials and student information.

[1457] This invention is an educational support system that provides high-quality education to students around the world, reduces the burden on teachers, and enables individualized instruction according to learning progress. This system is composed of three main elements: a server, a terminal, and a user, and realizes learning support that takes into account students' emotions through an emotion engine.

[1458] System Configuration

[1459] server

[1460] The server acts as a central control computer and is responsible for the following major processes:

[1461] 1. User Authentication

[1462] The server receives the authentication information (ID and password) sent by the student and authenticates the student by comparing it with the information in the database. If authentication is successful, the server sends the profile data to the device.

[1463] 2. Management and distribution of teaching materials

[1464] The server retrieves learning materials (videos, texts, quizzes, etc.) from the database and sends the materials selected by the student to the terminal.

[1465] 3. Collection and analysis of training data

[1466] The server collects learning activity data sent from the device, analyzes it to evaluate the level of understanding, and generates feedback based on the analysis results and sends it to the device.

[1467] 4. Emotion engine integration and analysis

[1468] The emotion engine analyzes facial expressions and tone of voice to recognize the student's emotions and sends the results to the server, which then adjusts the feedback content based on the emotion analysis results.

[1469] Terminal

[1470] Devices are electronic devices used by students, such as computers, tablets, and smartphones, and have the following functions:

[1471] 1. User authentication and profile management

[1472] The terminal sends the authentication information entered by the user to the server, and when the authentication result is returned, it displays the learning content based on the profile data.

[1473] 2. Display of teaching materials

[1474] The device displays the learning materials sent from the server and supports learning by, for example, playing lecture videos in a video player, displaying materials in a text viewer, and providing interactive quizzes.

[1475] 3. Tracking learning activities

[1476] The terminal collects students' learning activity data (viewing time, quiz correct answer rate, etc.) in real time and sends it to the server.

[1477] 4. Viewing Feedback

[1478] The device displays the feedback sent from the server and suggests next learning steps and supplementary learning materials.

[1479] 5. Collecting and transmitting emotional data

[1480] The device uses a built-in camera and microphone to transmit the student's facial expressions and tone of voice to the emotion engine, which then recognizes their emotions. The recognition results are then sent to the server.

[1481] User

[1482] The users are students who operate the system. They use the system as follows:

[1483] 1. Login and Authentication

[1484] The user logs in by entering their ID and password into the terminal. The server performs authentication, and if successful, the profile data is displayed on the terminal.

[1485] 2. Learning Progression

[1486] Users can view and learn from the learning materials presented on their devices, including videos, textbooks, and quizzes.

[1487] 3. Receive feedback and take next steps

[1488] The user reviews the feedback displayed on the terminal and performs the suggested next learning step.

[1489] Specific examples

[1490] For example, consider the case where a junior high school student user begins learning "quadratic equations."

[1491] 1. Login and Authentication

[1492] The user logs in by entering their ID and password on the terminal. The server performs authentication, and if authentication is successful, the profile data is acquired.

[1493] 2. Acquiring and displaying educational materials

[1494] The user selects the learning material for "quadratic equations." The server retrieves the corresponding learning material from the database and sends it to the terminal. The terminal displays it and begins learning.

[1495] 3. Learning Progress and Data Collection

[1496] Users watch videos, read text materials, and answer quizzes, and their devices send this learning activity data to a server.

[1497] 4. Data analysis and feedback generation

[1498] The server analyzes the collected learning activity data, evaluates the user's level of understanding, and generates feedback based on the analysis results and sends it to the device.

[1499] 5. Emotion recognition and feedback regulation

[1500] The device sends the user's facial expressions and tone of voice to the emotion engine to recognize emotions. For example, if a student is feeling stressed, the emotion engine reports this to the server and generates feedback suggesting a break to relax.

[1501] 6. View feedback and suggest next steps

[1502] The device displays feedback to the user, who can then review the next learning step or suggestions for relaxation and continue learning.

[1503] Example prompt

[1504] Use the following prompt to provide input to the generative AI model:

[1505] "Based on the following conditions, please explain the process of displaying teaching materials and generating feedback for students learning 'quadratic equations.'"

[1506] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1507] Step 1:

[1508] User Login and Authentication

[1509] The user starts up the device and enters their ID and password on the login screen. The entered authentication information is sent from the device to the server.

[1510] The server compares the received authentication information (ID and password) with the information in the database. If the comparison is successful, the server obtains the profile data and sends it to the device.

[1511] Specifically, the server queries the database to see if the matching ID and password combination exists.

[1512] Input: User ID and password

[1513] Output: Authentication result (if authentication is successful, profile data is sent to the device)

[1514] Step 2:

[1515] Requesting and receiving materials

[1516] The user selects the material they wish to study using the device interface, and this selection is sent as a request from the device to the server.

[1517] The server retrieves the requested learning materials from the database and sends them to the terminal.

[1518] Specifically, the server searches the database for learning materials (videos, texts, quizzes, etc.) that correspond to the specified learning material ID and retrieves them.

[1519] Input: ID of selected teaching material

[1520] Output: Send learning materials (videos, text, quizzes) to your device

[1521] Step 3:

[1522] Learning progression and data collection

[1523] The terminal displays the received learning materials to the user, who then uses them to progress with their learning.

[1524] All user activities (watching videos, reading texts, answering quizzes, etc.) are tracked by the device and sent to the server as learning activity data.

[1525] Specifically, the device collects the start and end times of the study session, quiz scores, and so on.

[1526] Input: User learning activity (video playback, quiz answers, etc.)

[1527] Output: Send learning activity data to the server

[1528] Step 4:

[1529] Analyzing training data and generating feedback

[1530] The server analyzes the collected learning activity data and evaluates the user's level of understanding, using criteria such as viewing time and quiz accuracy.

[1531] Based on the evaluation results, the server generates feedback including suggestions for next learning steps and additional practice questions and sends this to the device.

[1532] Specifically, the server uses an analysis algorithm to assess comprehension and determine appropriate feedback content.

[1533] Input: Learning activity data

[1534] Output: Send generated feedback to the terminal

[1535] Step 5:

[1536] Emotion data collection and analysis

[1537] The device sends the user's facial expressions and tone of voice to the emotion engine during training to recognize emotions, and the recognition results are sent to the server.

[1538] Specifically, the device captures facial expressions with its built-in camera and collects voice tone with its microphone.

[1539] The emotion engine analyzes this data and recognizes the user's emotions, and sends the results to the server.

[1540] Input: User's facial expression data, voice tone data

[1541] Output: Send the sentiment analysis results to the server

[1542] Step 6:

[1543] Adjusting and viewing feedback

[1544] The server adjusts the feedback based on the results of the emotion analysis. For example, if the user is feeling stressed, it generates feedback suggesting a break to relax.

[1545] The device displays the adjusted feedback to the user.

[1546] Specifically, the server dynamically changes the content of the feedback depending on the emotion analysis results.

[1547] Input: Sentiment analysis results

[1548] Output: Sends adjusted feedback to the device

[1549] In this way, the system optimizes the user's learning experience based on a flow that clearly indicates the specific processing and input / output at each step.

[1550] (Application example 2)

[1551] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1552] With the spread of online education, there is a need to provide high-quality education to students around the world. However, it is difficult to respond appropriately to the different levels of understanding and learning progress of each student. Furthermore, there is no support that takes into account emotional changes during learning. Therefore, there is a need for a system that can maximize students' learning effectiveness and reduce the burden on teachers.

[1553] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1554] In this invention, the server includes means for receiving authentication information from the student, means for authenticating the student based on the authentication information, means for retrieving selected learning materials from a database, means for transmitting the learning materials to the student's terminal, means for collecting the student's learning activity data from the terminal, means for analyzing the learning activity data and evaluating the student's level of understanding, means for generating feedback based on the evaluation results and transmitting the feedback to the terminal, means for analyzing the student's facial expressions and tone of voice to recognize emotions, and means for adjusting the feedback content in accordance with the recognized emotions. This enables personalized feedback that takes into account the student's level of understanding and emotional state, thereby improving learning effectiveness.

[1555] "Student" refers to a user of the learning content.

[1556] "Authentication information" refers to information such as ID and password that a user uses to log in to a system.

[1557] A "database" is an information management system for storing educational materials and learning data.

[1558] "Instructional Materials" refers to educational resources such as videos, textbooks, and interactive quizzes provided for learning purposes.

[1559] "Device" refers to a device (smartphone, tablet, PC, etc.) that students use to display and operate learning content.

[1560] "Learning activity data" refers to information about students' activities, such as the amount of time they spend viewing learning materials and the results of quizzes.

[1561] A "server" is a central computing unit that verifies authentication information, manages and distributes learning materials, analyzes learning data, and so on.

[1562] "Analysis" refers to the process of evaluating collected learning activity data and determining students' level of understanding.

[1563] "Feedback" refers to information provided to students based on the assessment results, such as suggestions for next learning steps or the provision of supplementary learning materials.

[1564] "Facial expressions" refers to data that infers emotions based on students' facial movements and expressions.

[1565] "Tone of voice" refers to data that infers emotions based on the way a student speaks, the pitch and intensity of their voice.

[1566] "Emotions" refers to the psychological state of students that can be inferred from their facial expressions and tone of voice.

[1567] An "emotion engine" is software or hardware that analyzes students' facial expressions and tone of voice to recognize their emotions.

[1568] "Personalization" means providing support that is individually tailored to each student's learning progress and emotional state.

[1569] System Overview

[1570] The system that realizes this application example provides high-quality education by linking the server, terminals, and users, and collecting, analyzing, and providing feedback on students' learning activity and emotional data. In particular, by combining it with an emotion engine, it is possible to recognize students' emotional states in real time and provide appropriate feedback according to their learning situation.

[1571] Hardware and Software Configuration

[1572] Server: A central computing unit that manages databases, authenticates users, analyzes data, and generates feedback. For example, a cloud server such as AWS or Google Cloud Platform can be used.

[1573] Device: A device on which students view and interact with learning content. Examples include smartphones, tablets, and PCs. This device is equipped with a camera and microphone, which are used to collect emotional data.

[1574] Emotion engine: Software that analyzes students' facial expressions and tone of voice to recognize their emotions. For example, libraries such as OpenCV and Google Cloud Vision AI can be used.

[1575] Processing steps

[1576] 1. User authentication: The user enters authentication information (ID and password) on the device and sends it to the server. The server verifies the authentication information, authenticates the user, and sends profile data to the device.

[1577] 2. Acquisition of learning materials: The server acquires the requested learning materials from the database and sends them to the user's terminal.

[1578] 3. Collection of learning activity data: The terminal collects learning activity data such as the user's viewing time and quiz correct answer rate, and transmits it to the server in real time.

[1579] 4. Data analysis: The server analyzes the collected learning activity data and evaluates the user's level of understanding.

[1580] 5. Feedback generation: Based on the evaluation results, feedback is generated suggesting next learning steps and supplementary learning materials, and this is sent to the device.

[1581] 6. Emotion data collection and analysis: The device sends the user's facial expressions and tone of voice during training via a camera and microphone to the emotion engine to recognize emotions. The server receives the recognition results and adjusts the feedback content.

[1582] Specific examples

[1583] For example, below is a specific scenario in which a junior high school student is learning about quadratic equations.

[1584] 1. The user logs into the app on their smartphone and selects the quadratic equations study material.

[1585] 2. The server sends the teaching materials (video lectures, text materials, quizzes, etc.) to the terminal.

[1586] 3. The user watches videos and answers quizzes. The device collects this data and sends it to the server.

[1587] 4. The server analyzes the learning activity data and evaluates the user's level of understanding.

[1588] 5. If the emotion engine recognizes "confusion" from the user's facial expression while watching a movie, the server generates feedback including supplementary explanations and sends it to the device.

[1589] 6. Furthermore, if the user is feeling "stressed," suggestions such as "take a five-minute break" will be made.

[1590] Example prompt sentence:

[1591] "My students are stressed. What are some ways to help them relax?"

[1592] "A student is confused. Please provide additional explanation for quadratic equations."

[1593] This allows for personalized feedback that takes into account the student's level of understanding and emotional state, maximizing learning outcomes while providing high-quality education.

[1594] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1595] Step 1: User authentication

[1596] The user enters authentication information (ID and password) from the terminal and sends it to the server, which collates this authentication information with information in a database and authenticates the user.

[1597] Input: ID, password

[1598] Output: Authentication result (profile data if successful)

[1599] Operation: The ID and password entered by the user are sent to the server, which then authenticates them by checking them against the authentication information in the database. If authentication is successful, the user's profile data is sent to the device.

[1600] Step 2: Acquire the study materials

[1601] The server retrieves the educational material requested by the user from the database and transmits it to the terminal.

[1602] Input: User request (e.g., teaching material ID)

[1603] Output: Teaching material data (videos, text, quizzes, etc.)

[1604] Operation: When a user sends a requested educational material ID from a terminal to the server, the server retrieves the corresponding educational material from the database and sends it to the terminal.

[1605] Step 3: Collect learning activity data

[1606] The terminal collects the user's viewing time, quiz correct answer rate, etc. in real time and transmits them to the server.

[1607] Input: Learning activity data (viewing time, quiz answer results, etc.)

[1608] Output: Sending learning activity data to the server

[1609] Operation: When a user watches or listens to educational materials or answers quizzes, their activity data is collected on the device and sent to the server in real time.

[1610] Step 4: Data analysis

[1611] The server analyzes the collected learning activity data and evaluates the user's level of understanding.

[1612] Input: Learning activity data

[1613] Output: Comprehension assessment results

[1614] Operation: The server analyzes learning activity data and performs data calculations using statistical analysis and machine learning algorithms to evaluate the user's level of understanding.

[1615] Step 5: Generate feedback

[1616] Based on the evaluation results, the server generates feedback suggesting the next learning step or supplementary learning materials and sends this to the terminal.

[1617] Input: Comprehension assessment results

[1618] Output: Feedback (suggestions for next learning steps, supplementary materials, etc.)

[1619] Operation: Based on the comprehension assessment results, the server uses a generative AI model to generate feedback suggesting next steps and supplementary learning materials. The generated feedback is then sent to the device.

[1620] Step 6: Collect and analyze emotion data

[1621] The device sends the user's facial expressions and tone of voice during training via a camera and microphone to the emotion engine to recognize emotions.

[1622] Input: facial expression data, tone of voice

[1623] Output: Emotion recognition result

[1624] How it works: Facial expression data and voice tone captured by the device's camera and microphone are sent to the emotion engine, which then analyzes the data and recognizes emotions. The recognized emotion results are then sent to the server.

[1625] Step 7: Adjusting Emotional Feedback

[1626] The server adjusts the feedback content based on the emotion recognition results and sends it to the terminal.

[1627] Input: Emotion recognition results

[1628] Output: Regulated Feedback

[1629] How it works: The server receives the emotion recognition results and adjusts the feedback content according to the recognized emotion. For example, if the student is tired, it suggests taking a break. The adjusted feedback is then sent to the device.

[1630] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1631] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1632] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1633] [Fourth embodiment]

[1634] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1635] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1636] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1637] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1638] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1639] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1640] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1641] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1642] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1643] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1644] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1645] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1646] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1647] This invention is a system that provides high-quality education to students around the world, reduces the burden on teachers, and enables individualized instruction according to learning progress. This system is realized through cooperation between a server, terminals, and users.

[1648] System Configuration

[1649] server

[1650] 1. User Authentication

[1651] The server authenticates the user (student) by checking the information in the database based on the authentication information (ID and password) received from the device. If authentication is successful, the server sends the user's profile data to the device.

[1652] 2. Management and distribution of teaching materials

[1653] The server manages various educational materials (videos, texts, interactive quizzes, etc.) stored in a database, retrieves the educational materials requested by the user from the device, and sends them to the device.

[1654] 3. Collection and analysis of training data

[1655] The server collects learning activity data sent from the device, including viewing time, quiz answer results, etc. The server analyzes this data and evaluates the user's level of understanding.

[1656] 4. Generate feedback

[1657] Based on the analysis results, the server generates appropriate feedback, such as suggestions for next learning steps or the provision of individually customized supplementary learning materials, and sends the generated feedback to the device.

[1658] Terminal

[1659] 1. User authentication and profile management

[1660] The user enters authentication information on the login screen from their device and sends it to the server. After authentication, the learning content begins to be displayed based on the profile data received from the server.

[1661] 2. Display of teaching materials

[1662] The device displays the educational material sent from the server to the user, for example, playing video lectures, displaying text materials, and providing interactive quizzes.

[1663] 3. Tracking learning activities

[1664] The device transmits the user's learning activity data (viewing time, quiz correct answer rate, etc.) to the server in real time.

[1665] 4. Viewing Feedback

[1666] Feedback sent from the server is displayed to the user, suggesting next steps and supplementary learning materials to help them continue their learning.

[1667] User

[1668] 1. Login and Authentication

[1669] Users log in to the system by entering their authentication information on a terminal.

[1670] 2. Learning Progression

[1671] Users can view and learn from the learning materials presented on their devices, including videos, textbooks, and quizzes.

[1672] 3. Receive feedback and take next steps

[1673] The user reviews the feedback displayed on the device and takes the suggested next learning steps.

[1674] Specific examples

[1675] For example, consider the case where a junior high school student user begins learning "quadratic equations."

[1676] 1. Login and Authentication

[1677] The user logs in by entering their ID and password on their device. The server performs authentication, and if authentication is successful, profile data is obtained.

[1678] 2. Acquiring and displaying educational materials

[1679] The user selects the learning material for "quadratic equations." The server retrieves the relevant learning material (video lectures, text materials, quizzes) from the database and sends it to the device. The device displays it, and the user begins learning.

[1680] 3. Learning Progress and Data Collection

[1681] Users watch videos, read text materials, and answer quizzes, and their devices send learning activity data (such as viewing time and quiz answers) to a server.

[1682] 4. Data analysis and feedback generation

[1683] The server analyzes the collected learning activity data, evaluates the user's level of understanding, and generates feedback (such as suggestions for the next learning step or additional practice questions) based on the analysis results and sends it to the device.

[1684] 5. View feedback and suggest next steps

[1685] The device displays feedback to the user, who can see what to work on next and complete suggested additional exercises.

[1686] In this way, the system can provide optimal education according to students' learning progress, reduce the burden on teachers, and improve the quality of education.

[1687] The processing flow will be explained below.

[1688] Step 1:

[1689] The user accesses the login screen from the terminal and enters their ID and password.

[1690] Step 2:

[1691] The terminal transmits the input authentication information to the server.

[1692] Step 3:

[1693] The server compares the received authentication information with information in a database and authenticates the user.

[1694] Step 4:

[1695] If the authentication is successful, the server sends the user's profile data to the terminal.

[1696] Step 5:

[1697] The terminal prepares to display the learning content based on the profile data received from the server.

[1698] Step 6:

[1699] The user selects the learning material they want to study on the screen of their device, for example, "quadratic equations."

[1700] Step 7:

[1701] The terminal requests the selected teaching material information from the server.

[1702] Step 8:

[1703] The server retrieves the relevant teaching materials (video lectures, text materials, quizzes) from the database and sends them to the terminal.

[1704] Step 9:

[1705] The terminal displays the educational materials sent from the server to the user, for example, playing lecture videos on a video player and displaying text materials.

[1706] Step 10:

[1707] Users watch videos, read text materials, and answer interactive quizzes on their devices.

[1708] Step 11:

[1709] The terminal collects the user's learning activity data (viewing time, quiz answers) and transmits it to the server in real time.

[1710] Step 12:

[1711] The server analyzes the learning activity data sent from the terminal and evaluates the user's level of understanding.

[1712] Step 13:

[1713] The server generates feedback based on the analysis results, such as "Your understanding of quadratic equations is insufficient; we recommend you do some additional practice problems."

[1714] Step 14:

[1715] The server transmits the generated feedback to the terminal.

[1716] Step 15:

[1717] The terminal displays the feedback received from the server to the user.

[1718] Step 16:

[1719] The user selects the next learning step based on the feedback and continues learning.

[1720] Through this series of steps, users are provided with the most appropriate content based on their learning progress. The server, devices, and users work together to provide high-quality education while reducing the burden on teachers.

[1721] Example 1

[1722] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1723] In conventional educational systems, it was difficult to provide optimal learning materials and feedback based on each student's individual learning progress. This also placed a heavy burden on teachers, making it difficult to provide high-quality education to all students. In particular, there was a lack of technological means to analyze students' learning data in real time and suggest personalized learning steps based on the results.

[1724] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1725] In this invention, the server includes means for receiving authentication information from the student, means for authenticating the student based on the authentication information, means for retrieving selected learning materials from a database, means for transmitting the learning materials to the student's terminal, means for collecting the student's learning activity data from the terminal, means for analyzing the learning activity data and evaluating the student's level of understanding, means for transmitting generated feedback to the terminal, means for evaluating the student's level of understanding using a generative AI model based on the student's learning progress data, and means for customizing suggestions for the next learning step based on the evaluation and presenting appropriate learning content using prompt sentences. This makes it possible to provide optimal learning materials and feedback according to each student's learning progress, thereby reducing the burden on teachers and providing high-quality education.

[1726] "Authentication information" refers to information used to log in to the system, such as a student ID and password.

[1727] A "database" is a system that centrally manages and stores information such as teaching materials, learning activity data, and user profiles.

[1728] "Devices" are devices used by students, such as computers, tablets, and smartphones.

[1729] "Learning activity data" refers to data related to the learning process, such as students' viewing time, quiz answer results, and text material viewing history.

[1730] "Comprehension" is an assessment of how well a student understands a particular learning material or topic.

[1731] "Feedback" refers to guidance and advice provided based on learning activity data and comprehension analysis results.

[1732] A "generative AI model" is a model that uses machine learning algorithms to extract patterns and trends from learning data and assess students' understanding.

[1733] A "prompt" is text that expresses instructions or questions that are input to a generative AI model.

[1734] "Personalization" means providing individually optimized educational content and feedback based on each student's learning progress and level of understanding.

[1735] A "learning step" is a specific learning activity or progress that a student should undertake next.

[1736] This invention provides an educational system that provides individualized instruction according to students' learning progress and reduces the burden on teachers. The system is realized through cooperation between a server, a terminal, and a user. The details are described below.

[1737] Server configuration and operation

[1738] The server performs the following main functions:

[1739] User authentication: The server authenticates the user (student) by comparing the authentication information (ID and password) received from the device with information in the database. If authentication is successful, the server sends the user's profile data to the device. At this time, a JWT (JSON Web Token) is used as the authentication token.

[1740] Management and distribution of learning materials: The server manages learning materials (videos, texts, quizzes, etc.) stored in a database. It retrieves learning materials requested by users from their devices and sends them to the devices. For example, learning materials are stored in a MySQL database.

[1741] Collection and analysis of learning data: The server collects learning activity data sent from the device. This data includes viewing time, quiz answer results, etc. The server uses the Python scikit-learn library to train a generative AI model and analyze the data.

[1742] Feedback Generation: Based on the analysis results, the server generates appropriate feedback, including suggesting next learning steps and providing personalized learning materials.

[1743] Terminal configuration and operation

[1744] The terminal performs the following main functions:

[1745] User authentication and profile management: The user enters authentication information on the login screen from the device and sends it to the server. After authentication, the learning content begins to be displayed based on the profile data received from the server.

[1746] Displaying educational materials: The device displays educational materials sent from the server to the user. For example, a YouTube video player can be embedded to play video lectures.

[1747] Learning activity tracking: The device sends the user's learning activity data (viewing time, quiz correct answer rate, etc.) to the server in real time. This data is accumulated on the server and used for analysis.

[1748] Show feedback: Show feedback sent by the server to the user, for example, "It seems you don't understand the basics of quadratic equations. Please study the supplementary material below."

[1749] User Actions

[1750] The user performs the following main actions:

[1751] Login and authentication: The user logs in by entering their ID and password on the terminal. For example, they enter "student123" and "password!" and press the login button.

[1752] Learning Progress: Users can view and learn the materials presented on their devices. They can watch videos, read text materials, and answer quizzes.

[1753] Receive feedback and take next steps: Users review the feedback displayed on their device and take the suggested next learning step. For example, they can click on "Additional learning materials for quadratic equations" to begin additional learning.

[1754] Specific examples

[1755] For example, consider a case where a junior high school student user begins studying "quadratic equations." The user logs in by entering their ID and password on their device. If authentication is successful, profile data is retrieved from the server. Next, the user selects learning materials for "quadratic equations," and the server retrieves the relevant materials from the database and sends them to the device. The user watches videos, reads text materials, and answers quizzes. The device sends this learning activity data to the server. The server analyzes the collected data and evaluates the level of understanding using a generative AI model. Feedback is generated and the next learning step is suggested. The user continues studying according to this feedback.

[1756] Prompt Sentence Examples

[1757] "How can we use generative AI models to tailor optimal feedback to users based on their learning progress?"

[1758] In this way, the system of the present invention provides optimal education according to the student's learning progress, enabling the student to progress in learning efficiently.

[1759] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1760] Step 1: Enter and submit your credentials

[1761] User: The user enters their ID and password on their device and clicks the login button. For example, they enter "student123" and "password!"

[1762] Input: User ID and password.

[1763] Terminal: The terminal sends the entered authentication information to the server.

[1764] Output: Sending authentication information to the server.

[1765] Step 2: User authentication

[1766] Server: The server checks the received authentication information against a database and authenticates the user.

[1767] Input: Authentication information sent from the device (user ID and password).

[1768] Data manipulation: Using an SQL query, the authentication information is matched against the user table in the database.

[1769] Output: Authentication result (success or failure) and profile data.

[1770] Specific operation: Connect to the MySQL database and search the user table. If authentication is successful, obtain profile data and send it to the device. If authentication fails, send an error message.

[1771] Step 3: View your profile data

[1772] Terminal: The terminal saves the profile data received from the server and transitions to the learning screen.

[1773] Input: Profile data from the server.

[1774] Output: Transition to the learning screen after logging in.

[1775] Specific operation: Upon successful authentication, a dashboard showing the user's specific learning progress and history will be displayed.

[1776] Step 4: Request and submit materials

[1777] User: Select the material you want to learn on the learning screen. For example, click "Math > Quadratic Equations."

[1778] Input: The ID of the selected material.

[1779] Terminal: Sends a request including the learning material ID to the server.

[1780] Output: Sending a learning material request to the server.

[1781] Step 5: Acquire and distribute materials

[1782] Server: Receives the learning material request and retrieves the corresponding learning material data from the database.

[1783] Input: Teaching material request from the terminal (teaching material ID).

[1784] Data processing: Retrieve teaching material data from the database using SQL queries.

[1785] Output: Sending teaching material data to the terminal.

[1786] Specific operation: The relevant video URL, text material, quiz, and other data are read from the database and sent to the device.

[1787] Step 6: Displaying the materials

[1788] Terminal: The terminal receives the teaching material data sent from the server and displays it to the user.

[1789] Input: Teaching material data from the server (video URL, text materials, quizzes).

[1790] Output: Display of teaching materials.

[1791] What it does: Embeds a video player using YouTube's API, plays videos, displays text material, and starts an interactive quiz.

[1792] Step 7: Tracking learning activities

[1793] Terminal: Sends user learning activity data (viewing time, quiz answer results, etc.) to the server in real time.

[1794] Input: User learning activity data.

[1795] Data processing: Collect various activity data (e.g., start time, end time, correct answer rate) and convert them into an appropriate format.

[1796] Output: Sending learning activity data to the server.

[1797] Specific operation: Generate a log of learning activities and periodically send it to the server.

[1798] Step 8: Collect and analyze training data

[1799] Server: Analyzes the collected data and evaluates comprehension using a generative AI model.

[1800] Input: Learning activity data sent from the device.

[1801] Data processing: Preprocess the data using Python's scikit-learn library and analyze it using a generative AI model.

[1802] Output: Generate comprehension assessment results and feedback.

[1803] Specific operation: Learning activity data is fed into an AI model to generate performance metrics (e.g., grades, comprehension scores).

[1804] Step 9: Generate feedback

[1805] Server: Generates feedback based on the analysis results, including suggesting next learning steps and providing personalized learning materials.

[1806] Input: Comprehension assessment results and output of the generative AI model.

[1807] Data processing: Generating feedback messages and selecting the next teaching materials.

[1808] Output: Sending feedback information to a terminal.

[1809] What it does: Based on a specific comprehension score, generate links to additional practice questions or supplemental materials and send them to your device.

[1810] Step 10: View feedback and take next steps

[1811] Terminal: The terminal displays the feedback received from the server to the user.

[1812] Input: Feedback information from the server.

[1813] Output: Displaying feedback to the user and suggesting next learning steps.

[1814] What it does: A dialog box appears on the screen, showing the next learning step and links to additional learning materials. It tells the user to click on "Additional learning materials for quadratic equations."

[1815] Example prompt

[1816] "How can we use generative AI models to tailor optimal feedback to users based on their learning progress?"

[1817] Through the above process, this system can provide optimal education according to students' learning progress, realizing an efficient and personalized learning experience.

[1818] (Application example 1)

[1819] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1820] Modern factories require efficient and effective training for new workers and existing workers learning new processes. However, conventional methods have difficulty providing education and training tailored to each worker's individual level of understanding, making it difficult to contribute to improving overall production efficiency and safety. The present invention aims to solve these problems by providing a system that collects and analyzes workers' learning activity data in real time and provides training materials and feedback based on each worker's individual level of understanding.

[1821] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1822] In this invention, the server includes means for receiving authentication information from a student, means for authenticating the student based on the authentication information, means for retrieving selected learning materials from a database, means for transmitting the learning materials to the student's terminal, means for collecting learning activity data of the student from the terminal, means for analyzing the learning activity data and evaluating the student's level of understanding, means for generating feedback based on the evaluation results and transmitting the feedback to the terminal, means for supporting the learning progress of factory workers based on the feedback, and means for providing the factory workers with training materials for learning new processes and machine operations. This enables factory workers to receive effective training tailored to their individual levels of understanding.

[1823] "Students" refers to factory workers who are users, and includes beginners who need to learn new processes or machine operations, as well as existing experts.

[1824] "Authentication Information" refers to information such as ID and password required for a user to access the system.

[1825] "Database" refers to the centralized management system where training materials, user profiles, learning activity data, etc. are stored.

[1826] "Instructional Materials" means educational content provided to factory workers, including video lectures, text materials, interactive quizzes, etc.

[1827] "Devices" refers to devices used by factory workers to view learning materials and carry out learning activities. Examples include computers and tablets.

[1828] "Learning activity data" is data related to the worker's learning activities, including viewing time and quiz answer results.

[1829] "Feedback" refers to information generated based on the analysis of learning activity data, suggesting the next learning step or additional training content.

[1830] "Factory workers" refers to people who operate machines and manage processes at production sites and receive training.

[1831] "Training materials" refers to educational materials provided to factory workers to learn new processes or machine operations.

[1832] The present invention is a system for efficiently providing training to factory workers and tracking and analyzing their progress in real time. An embodiment of this system will be described in detail below.

[1833] System Configuration

[1834] server

[1835] The server has multiple functions and plays a central role in managing and implementing training for factory workers. Specifically, it implements the following functions:

[1836] 1. User Authentication

[1837] The server authenticates the factory worker by checking the information in the database based on the authentication information (ID and password) of the factory worker received from the terminal. If authentication is successful, the server sends the worker's profile data to the terminal.

[1838] 2. Management and distribution of teaching materials

[1839] The server manages the training materials (videos, text, interactive quizzes, etc.) used by factory workers in a database, retrieves the materials requested by the workers from their terminals, and sends them to the terminals.

[1840] 3. Collection and analysis of training data

[1841] The server collects learning activity data of factory workers in real time, including viewing time, quiz answer results, etc. The server analyzes this data and evaluates the worker's level of understanding.

[1842] 4. Generate feedback

[1843] Based on the analysis results, the server generates appropriate feedback, including suggestions for next learning steps and the provision of additional training materials, and sends the generated feedback to the device.

[1844] Terminal

[1845] The terminal is a device that is directly operated by factory workers and provides an interface for training. Specifically, it implements the following functions:

[1846] 1. User authentication and profile management

[1847] The worker enters authentication information on the login screen from their terminal and sends it to the server. After authentication, the learning content begins to be displayed based on the profile data received from the server.

[1848] 2. Display of teaching materials

[1849] The terminals display educational materials sent from the server to the workers, for example, playing video lectures on machine operation, displaying text materials on operating procedures, and providing interactive quizzes.

[1850] 3. Tracking learning activities

[1851] The terminal collects data on the worker's learning activities (such as viewing time and quiz correct answer rate) in real time and sends it to the server.

[1852] 4. Viewing Feedback

[1853] Feedback sent from the server is displayed to the worker, suggesting next learning steps and additional training content to help them continue learning.

[1854] user

[1855] Factory workers can be efficiently trained using this system, specifically through the following process:

[1856] 1. Login and Authentication

[1857] Workers log in to the system by entering their authentication information on a terminal.

[1858] 2. Learning Progression

[1859] Workers can access the learning materials provided on their devices and progress through their studies, watching videos, reading textbooks, and answering quizzes.

[1860] 3. Receive feedback and take next steps

[1861] The worker reviews the feedback displayed on the terminal and executes the suggested next learning steps.

[1862] Specific examples

[1863] For example, when a new worker begins training to operate a new machine, the steps would be as follows:

[1864] 1. Login and Authentication

[1865] The worker logs in by entering their ID and password on the terminal. The server performs authentication and, if successful, obtains their profile data.

[1866] 2. Acquiring and displaying educational materials

[1867] A worker selects a new machine's operating manual. The server retrieves the relevant learning materials (video lectures, text materials, interactive quizzes) and sends them to the terminal. The terminal displays them, and the worker begins learning.

[1868] 3. Learning Progress and Data Collection

[1869] Workers watch videos, read textual materials on operating procedures, and answer quizzes, and their devices send this learning activity data to a server.

[1870] 4. Data analysis and feedback generation

[1871] The server analyzes the collected learning activity data, evaluates the worker's level of understanding, and generates feedback (such as suggestions for the next learning step or additional practice questions) based on the analysis results and sends it to the terminal.

[1872] 5. View feedback and suggest next steps

[1873] The device displays feedback to the worker, who can see what to do next and implement any additional techniques suggested.

[1874] Example prompts for generative AI models

[1875] "Please explain how to implement a factory worker operation training system. Please explain in detail the specific steps, including user authentication, learning material management, learning data collection and analysis, and feedback generation."

[1876] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1877] Step 1:

[1878] A factory worker (hereafter referred to as the user) enters authentication information (ID and password) into a terminal. The terminal receives this information and sends it to the server. The server compares it with user data in a database, and if authentication is successful, it returns profile data to the terminal. The input is the user's ID and password, and the output is the authentication result and profile data.

[1879] Step 2:

[1880] The terminal displays the user's learning dashboard based on the profile data received from the server. This dashboard contains a list of available training materials. The user selects the training material they want to learn. The input is the profile data and the user's material selection, and the output is a display of the selected material.

[1881] Step 3:

[1882] Based on the user's selection, the terminal sends a learning material request to the server. The server retrieves the requested learning material from the database and sends it to the terminal. The input is the learning material request, and the output is the learning material data.

[1883] Step 4:

[1884] The device displays the learning material data (videos, text, quizzes) received from the server to the user. The user progresses through learning by watching or reading the data and answering quizzes. The input is the learning material data, and the output is the user's learning activity data.

[1885] Step 5:

[1886] The device collects the user's learning activity data in real time and sends it to the server. The learning activity data includes viewing time, answer results, correct answer rate, etc. The server receives this data and stores it in a database. The input is the learning activity data, and the output is the stored learning activity data.

[1887] Step 6:

[1888] The server analyzes the saved learning activity data and evaluates the user's level of understanding. Here, the analysis is based on factors such as viewing time and quiz correct answer rate. The input is the learning activity data, and the output is the comprehension evaluation result.

[1889] Step 7:

[1890] The server generates individually customized feedback based on the comprehension assessment results. This feedback may include suggestions for the next learning step or additional practice questions. The generated feedback is sent to the device. The input is the comprehension assessment results, and the output is the feedback.

[1891] Step 8:

[1892] The terminal displays the feedback received from the server to the user, and the user performs the next learning step based on the feedback. The input is the feedback, and the output is the user's continued learning.

[1893] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1894] This invention is a system that provides high-quality education to students around the world, reduces the burden on teachers, and enables individualized instruction according to learning progress. By combining this invention with an emotion engine that recognizes students' emotions, it is possible to provide more precise feedback and adjust learning. This system works in cooperation between the server, the terminal, and the user, and realizes learning support that takes into account the user's emotions through the emotion engine.

[1895] System Configuration

[1896] server

[1897] 1. User Authentication

[1898] The server authenticates the user (student) by checking the information in the database based on the authentication information (ID and password) received from the device. If authentication is successful, the server sends the user's profile data to the device.

[1899] 2. Management and distribution of teaching materials

[1900] The server manages various educational materials (videos, text, interactive quizzes, etc.) stored in a database. It retrieves educational materials requested by users from their devices and sends them to the devices.

[1901] 3. Collection and analysis of training data

[1902] The server collects learning activity data sent from the device, including viewing time, quiz answer results, etc. The server analyzes this data and evaluates the user's level of understanding.

[1903] 4. Generate feedback

[1904] Based on the analysis results, the server generates appropriate feedback, such as suggestions for next learning steps or the provision of individually customized supplementary learning materials, and sends the generated feedback to the device.

[1905] 5. Emotion engine integration and analysis

[1906] The emotion engine analyzes the user's facial expressions, tone of voice, input data, etc. sent from the device during learning, and recognizes the user's emotions. The recognition results are sent to the server, and the feedback content is adjusted based on this.

[1907] Terminal

[1908] 1. User authentication and profile management

[1909] The user enters authentication information on the login screen from the terminal and sends it to the server. After authentication, the system prepares to display the learning content based on the profile data received from the server.

[1910] 2. Display of teaching materials

[1911] The terminal displays the educational materials sent from the server to the user, for example, playing lecture videos in a video player, displaying text materials, and providing interactive quizzes.

[1912] 3. Tracking learning activities

[1913] The terminal collects the user's learning activity data (viewing time, quiz correct answer rate, etc.) and transmits it to the server in real time.

[1914] 4. Viewing Feedback

[1915] Feedback sent from the server is displayed to the user, suggesting next steps and supplementary learning materials to help them continue their learning.

[1916] 5. Collecting and transmitting emotional data

[1917] The device sends the user's facial expressions, tone of voice, input data, etc. to the emotion engine to recognize emotions. The recognition results are then sent to the server.

[1918] User

[1919] 1. Login and Authentication

[1920] The user logs in to the system by entering authentication information from the terminal.

[1921] 2. Learning Progression

[1922] Users can view and learn from the learning materials presented on their devices, including videos, textbooks, and quizzes.

[1923] 3. Receive feedback and take next steps

[1924] The user reviews the feedback displayed on the device and performs the suggested next learning step.

[1925] Specific examples

[1926] For example, consider the case where a junior high school student user begins learning "quadratic equations."

[1927] 1. Login and Authentication

[1928] The user logs in by entering their ID and password on their device. The server performs authentication, and if authentication is successful, the profile data is retrieved.

[1929] 2. Acquiring and displaying educational materials

[1930] The user selects the learning material for "quadratic equations." The server retrieves the relevant learning material (video lectures, text materials, quizzes) from the database and sends it to the terminal. The terminal displays it, and the user begins learning.

[1931] 3. Learning Progress and Data Collection

[1932] Users watch videos, read text materials, and answer quizzes. The device sends learning activity data (such as viewing time and quiz answers) to the server.

[1933] 4. Data analysis and feedback generation

[1934] The server analyzes the collected learning activity data, evaluates the user's level of understanding, and generates feedback (such as suggestions for the next learning step or additional practice questions) based on the analysis results and sends it to the device.

[1935] 5. Emotion recognition and feedback regulation

[1936] The device sends the user's facial expressions and tone of voice to the emotion engine to recognize emotions. For example, if a student is feeling stressed, the emotion engine reports this to the server, which then generates feedback suggesting a break to relax.

[1937] 6. View feedback and suggest next steps

[1938] The device displays feedback to the user, who can then review the next learning step or suggestions for relaxation and continue learning.

[1939] In this way, the system not only provides optimal content according to students' learning progress, but also provides more effective learning support by taking emotions into consideration. The server, devices, and users work together to provide high-quality education through the emotion engine, while reducing the burden on teachers.

[1940] The processing flow will be explained below.

[1941] Step 1:

[1942] The user accesses the login screen from the terminal and enters their ID and password.

[1943] Step 2:

[1944] The terminal transmits the input authentication information to the server.

[1945] Step 3:

[1946] The server compares the received authentication information with information in a database and authenticates the user.

[1947] Step 4:

[1948] If the authentication is successful, the server sends the user's profile data to the terminal.

[1949] Step 5:

[1950] The terminal prepares to display the learning content based on the profile data received from the server.

[1951] Step 6:

[1952] The user selects the learning material they want to study on the screen of their device, for example, "quadratic equations."

[1953] Step 7:

[1954] The terminal requests the selected teaching material information from the server.

[1955] Step 8:

[1956] The server retrieves the relevant teaching materials (video lectures, text materials, quizzes) from the database and sends them to the terminal.

[1957] Step 9:

[1958] The terminal displays the educational materials sent from the server to the user, for example, playing lecture videos on a video player and displaying text materials.

[1959] Step 10:

[1960] Users watch videos, read text materials, and answer interactive quizzes on their devices.

[1961] Step 11:

[1962] The terminal collects the user's learning activity data (viewing time, quiz answers) and transmits it to the server in real time.

[1963] Step 12:

[1964] The device sends the user's facial expressions, tone of voice, input data, etc. that are being learned to the emotion engine to recognize emotions.

[1965] Step 13:

[1966] The emotion engine analyzes the user's facial expressions and tone of voice to understand the user's emotions.

[1967] Step 14:

[1968] The terminal transmits the recognition result of the emotion engine to the server.

[1969] Step 15:

[1970] The server analyzes the learning activity data sent from the device and the recognition results of the emotion engine to evaluate the user's level of understanding and emotional state.

[1971] Step 16:

[1972] The server generates feedback based on the analysis results, for example, if the user is feeling stressed, it generates feedback suggesting a break.

[1973] Step 17:

[1974] The server transmits the generated feedback to the terminal.

[1975] Step 18:

[1976] The terminal displays the feedback received from the server to the user.

[1977] Step 19:

[1978] The user reviews the feedback and takes suggested next learning steps or breaks.

[1979] Through this series of steps, the system provides optimal educational support that takes into account the user's learning progress and emotional state. The server, device, and emotion engine work together to deliver a high-quality learning experience and reduce the burden on teachers.

[1980] Example 2

[1981] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1982] In the current educational system, it is difficult to assess each student's individual learning progress and level of understanding in real time and provide appropriate feedback. Furthermore, there is a lack of learning support that takes students' emotions into consideration, which often leads to a decline in students' motivation to learn. This leads to a decline in the quality of education and an increase in the burden on teachers. A system that efficiently and effectively addresses these challenges is needed.

[1983] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1984] In this invention, the server includes means for receiving authentication information from a student, means for authenticating the student based on the authentication information, means for retrieving selected learning materials from a database, means for transmitting the learning materials to the student's terminal, means for collecting learning activity data of the student from the terminal, means for analyzing the learning activity data and evaluating the student's level of understanding, means for generating feedback based on the evaluation results and transmitting the feedback to the terminal, means for transmitting the student's facial expressions and tone of voice to an emotion engine for analysis, and means for adjusting the feedback content based on the emotion analysis results. This makes it possible to provide feedback according to the student's learning progress and support that takes emotions into consideration.

[1985] "Authentication information" refers to information that uniquely identifies a student, such as the ID and password that a student enters when logging in to the system.

[1986] "Study materials" refers to data such as videos, texts, and quizzes that are provided as learning materials necessary for students to advance their studies.

[1987] "Learning activity data" is a record of the activities that students performed while studying, including information such as viewing time and quiz correct answer rate.

[1988] "Feedback" is information that provides students with advice on their studies and suggests next steps based on the learning activity data analyzed by the server.

[1989] The "emotion engine" is computer software that analyzes students' facial expressions and tone of voice to recognize their emotions at that time.

[1990] "Emotion analysis results" are information about a student's emotional state obtained by the emotion engine by analyzing data such as facial expressions and tone of voice.

[1991] "Devices" are electronic devices used by students, such as computers, tablets, and smartphones.

[1992] The "server" is a central control computer that manages authentication information, distributes learning materials, analyzes learning activity data, generates feedback, and so on.

[1993] "Database" means a data storage system that the server accesses to store learning materials and student information.

[1994] This invention is an educational support system that provides high-quality education to students around the world, reduces the burden on teachers, and enables individualized instruction according to learning progress. This system is composed of three main elements: a server, a terminal, and a user, and realizes learning support that takes into account students' emotions through an emotion engine.

[1995] System Configuration

[1996] server

[1997] The server acts as a central control computer and is responsible for the following major processes:

[1998] 1. User Authentication

[1999] The server receives the authentication information (ID and password) sent by the student and authenticates the student by comparing it with the information in the database. If authentication is successful, the server sends the profile data to the device.

[2000] 2. Management and distribution of teaching materials

[2001] The server retrieves learning materials (videos, texts, quizzes, etc.) from the database and sends the materials selected by the student to the terminal.

[2002] 3. Collection and analysis of training data

[2003] The server collects learning activity data sent from the device, analyzes it to evaluate the level of understanding, and generates feedback based on the analysis results and sends it to the device.

[2004] 4. Emotion engine integration and analysis

[2005] The emotion engine analyzes facial expressions and tone of voice to recognize the student's emotions and sends the results to the server, which then adjusts the feedback content based on the emotion analysis results.

[2006] Terminal

[2007] Devices are electronic devices used by students, such as computers, tablets, and smartphones, and have the following functions:

[2008] 1. User authentication and profile management

[2009] The terminal sends the authentication information entered by the user to the server, and when the authentication result is returned, it displays the learning content based on the profile data.

[2010] 2. Display of teaching materials

[2011] The device displays the learning materials sent from the server and supports learning by, for example, playing lecture videos in a video player, displaying materials in a text viewer, and providing interactive quizzes.

[2012] 3. Tracking learning activities

[2013] The terminal collects students' learning activity data (viewing time, quiz correct answer rate, etc.) in real time and sends it to the server.

[2014] 4. Viewing Feedback

[2015] The device displays the feedback sent from the server and suggests next learning steps and supplementary learning materials.

[2016] 5. Collecting and transmitting emotional data

[2017] The device uses a built-in camera and microphone to transmit the student's facial expressions and tone of voice to the emotion engine, which then recognizes their emotions. The recognition results are then sent to the server.

[2018] User

[2019] The users are students who operate the system. They use the system as follows:

[2020] 1. Login and Authentication

[2021] The user logs in by entering their ID and password into the terminal. The server performs authentication, and if successful, the profile data is displayed on the terminal.

[2022] 2. Learning Progression

[2023] Users can view and learn from the learning materials presented on their devices, including videos, textbooks, and quizzes.

[2024] 3. Receive feedback and take next steps

[2025] The user reviews the feedback displayed on the terminal and performs the suggested next learning step.

[2026] Specific examples

[2027] For example, consider the case where a junior high school student user begins learning "quadratic equations."

[2028] 1. Login and Authentication

[2029] The user logs in by entering their ID and password on the terminal. The server performs authentication, and if authentication is successful, the profile data is acquired.

[2030] 2. Acquiring and displaying educational materials

[2031] The user selects the learning material for "quadratic equations." The server retrieves the corresponding learning material from the database and sends it to the terminal. The terminal displays it and begins learning.

[2032] 3. Learning Progress and Data Collection

[2033] Users watch videos, read text materials, and answer quizzes, and their devices send this learning activity data to a server.

[2034] 4. Data analysis and feedback generation

[2035] The server analyzes the collected learning activity data, evaluates the user's level of understanding, and generates feedback based on the analysis results and sends it to the device.

[2036] 5. Emotion recognition and feedback regulation

[2037] The device sends the user's facial expressions and tone of voice to the emotion engine to recognize emotions. For example, if a student is feeling stressed, the emotion engine reports this to the server and generates feedback suggesting a break to relax.

[2038] 6. View feedback and suggest next steps

[2039] The device displays feedback to the user, who can then review the next learning step or suggestions for relaxation and continue learning.

[2040] Example prompt

[2041] Use the following prompt to provide input to the generative AI model:

[2042] "Based on the following conditions, please explain the process of displaying teaching materials and generating feedback for students learning 'quadratic equations.'"

[2043] The flow of the identification process in the second embodiment will be described with reference to FIG.

[2044] Step 1:

[2045] User Login and Authentication

[2046] The user starts up the device and enters their ID and password on the login screen. The entered authentication information is sent from the device to the server.

[2047] The server compares the received authentication information (ID and password) with the information in the database. If the comparison is successful, the server obtains the profile data and sends it to the device.

[2048] Specifically, the server queries the database to see if the matching ID and password combination exists.

[2049] Input: User ID and password

[2050] Output: Authentication result (if authentication is successful, profile data is sent to the device)

[2051] Step 2:

[2052] Requesting and receiving materials

[2053] The user selects the material they wish to study using the device interface, and this selection is sent as a request from the device to the server.

[2054] The server retrieves the requested learning materials from the database and sends them to the terminal.

[2055] Specifically, the server searches the database for learning materials (videos, texts, quizzes, etc.) that correspond to the specified learning material ID and retrieves them.

[2056] Input: ID of selected teaching material

[2057] Output: Send learning materials (videos, text, quizzes) to your device

[2058] Step 3:

[2059] Learning progression and data collection

[2060] The terminal displays the received learning materials to the user, who then uses them to progress with their learning.

[2061] All user activities (watching videos, reading texts, answering quizzes, etc.) are tracked by the device and sent to the server as learning activity data.

[2062] Specifically, the device collects the start and end times of the study session, quiz scores, and so on.

[2063] Input: User learning activity (video playback, quiz answers, etc.)

[2064] Output: Send learning activity data to the server

[2065] Step 4:

[2066] Analyzing training data and generating feedback

[2067] The server analyzes the collected learning activity data and evaluates the user's level of understanding, using criteria such as viewing time and quiz accuracy.

[2068] Based on the evaluation results, the server generates feedback including suggestions for next learning steps and additional practice questions and sends this to the device.

[2069] Specifically, the server uses an analysis algorithm to assess comprehension and determine appropriate feedback content.

[2070] Input: Learning activity data

[2071] Output: Send generated feedback to the terminal

[2072] Step 5:

[2073] Emotion data collection and analysis

[2074] The device sends the user's facial expressions and tone of voice to the emotion engine during training to recognize emotions, and the recognition results are sent to the server.

[2075] Specifically, the device captures facial expressions with its built-in camera and collects voice tone with its microphone.

[2076] The emotion engine analyzes this data and recognizes the user's emotions, and sends the results to the server.

[2077] Input: User's facial expression data, voice tone data

[2078] Output: Send the sentiment analysis results to the server

[2079] Step 6:

[2080] Adjusting and viewing feedback

[2081] The server adjusts the feedback based on the results of the emotion analysis. For example, if the user is feeling stressed, it generates feedback suggesting a break to relax.

[2082] The device displays the adjusted feedback to the user.

[2083] Specifically, the server dynamically changes the content of the feedback depending on the emotion analysis results.

[2084] Input: Sentiment analysis results

[2085] Output: Sends adjusted feedback to the device

[2086] In this way, the system optimizes the user's learning experience based on a flow that clearly indicates the specific processing and input / output at each step.

[2087] (Application example 2)

[2088] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2089] With the spread of online education, there is a need to provide high-quality education to students around the world. However, it is difficult to respond appropriately to the different levels of understanding and learning progress of each student. Furthermore, there is no support that takes into account emotional changes during learning. Therefore, there is a need for a system that can maximize students' learning effectiveness and reduce the burden on teachers.

[2090] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[2091] In this invention, the server includes means for receiving authentication information from the student, means for authenticating the student based on the authentication information, means for retrieving selected learning materials from a database, means for transmitting the learning materials to the student's terminal, means for collecting the student's learning activity data from the terminal, means for analyzing the learning activity data and evaluating the student's level of understanding, means for generating feedback based on the evaluation results and transmitting the feedback to the terminal, means for analyzing the student's facial expressions and tone of voice to recognize emotions, and means for adjusting the feedback content in accordance with the recognized emotions. This enables personalized feedback that takes into account the student's level of understanding and emotional state, thereby improving learning effectiveness.

[2092] "Student" refers to a user of the learning content.

[2093] "Authentication information" refers to information such as ID and password that a user uses to log in to a system.

[2094] A "database" is an information management system for storing educational materials and learning data.

[2095] "Instructional Materials" refers to educational resources such as videos, textbooks, and interactive quizzes provided for learning purposes.

[2096] "Device" refers to a device (smartphone, tablet, PC, etc.) that students use to display and operate learning content.

[2097] "Learning activity data" refers to information about students' activities, such as the amount of time they spend viewing learning materials and the results of quizzes.

[2098] A "server" is a central computing unit that verifies authentication information, manages and distributes learning materials, analyzes learning data, and so on.

[2099] "Analysis" refers to the process of evaluating collected learning activity data and determining students' level of understanding.

[2100] "Feedback" refers to information provided to students based on the assessment results, such as suggestions for next learning steps or the provision of supplementary learning materials.

[2101] "Facial expressions" refers to data that infers emotions based on students' facial movements and expressions.

[2102] "Tone of voice" refers to data that infers emotions based on the way a student speaks, the pitch and intensity of their voice.

[2103] "Emotions" refers to the psychological state of students that can be inferred from their facial expressions and tone of voice.

[2104] An "emotion engine" is software or hardware that analyzes students' facial expressions and tone of voice to recognize their emotions.

[2105] "Personalization" means providing support that is individually tailored to each student's learning progress and emotional state.

[2106] System Overview

[2107] The system that realizes this application example provides high-quality education by linking the server, terminals, and users, and collecting, analyzing, and providing feedback on students' learning activity and emotional data. In particular, by combining it with an emotion engine, it is possible to recognize students' emotional states in real time and provide appropriate feedback according to their learning situation.

[2108] Hardware and Software Configuration

[2109] Server: A central computing unit that manages databases, authenticates users, analyzes data, and generates feedback. For example, a cloud server such as AWS or Google Cloud Platform can be used.

[2110] Device: A device on which students view and interact with learning content. Examples include smartphones, tablets, and PCs. This device is equipped with a camera and microphone, which are used to collect emotional data.

[2111] Emotion engine: Software that analyzes students' facial expressions and tone of voice to recognize their emotions. For example, libraries such as OpenCV and Google Cloud Vision AI can be used.

[2112] Processing steps

[2113] 1. User authentication: The user enters authentication information (ID and password) on the device and sends it to the server. The server verifies the authentication information, authenticates the user, and sends profile data to the device.

[2114] 2. Acquisition of learning materials: The server acquires the requested learning materials from the database and sends them to the user's terminal.

[2115] 3. Collection of learning activity data: The terminal collects learning activity data such as the user's viewing time and quiz correct answer rate, and transmits it to the server in real time.

[2116] 4. Data analysis: The server analyzes the collected learning activity data and evaluates the user's level of understanding.

[2117] 5. Feedback generation: Based on the evaluation results, feedback is generated suggesting next learning steps and supplementary learning materials, and this is sent to the device.

[2118] 6. Emotion data collection and analysis: The device sends the user's facial expressions and tone of voice during training via a camera and microphone to the emotion engine to recognize emotions. The server receives the recognition results and adjusts the feedback content.

[2119] Specific examples

[2120] For example, below is a specific scenario in which a junior high school student is learning about quadratic equations.

[2121] 1. The user logs into the app on their smartphone and selects the quadratic equations study material.

[2122] 2. The server sends the teaching materials (video lectures, text materials, quizzes, etc.) to the terminal.

[2123] 3. The user watches videos and answers quizzes. The device collects this data and sends it to the server.

[2124] 4. The server analyzes the learning activity data and evaluates the user's level of understanding.

[2125] 5. If the emotion engine recognizes "confusion" from the user's facial expression while watching a movie, the server generates feedback including supplementary explanations and sends it to the device.

[2126] 6. Furthermore, if the user is feeling "stressed," suggestions such as "take a five-minute break" will be made.

[2127] Example prompt sentence:

[2128] "My students are stressed. What are some ways to help them relax?"

[2129] "A student is confused. Please provide additional explanation for quadratic equations."

[2130] This allows for personalized feedback that takes into account the student's level of understanding and emotional state, maximizing learning outcomes while providing high-quality education.

[2131] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[2132] Step 1: User authentication

[2133] The user enters authentication information (ID and password) from the terminal and sends it to the server, which collates this authentication information with information in a database and authenticates the user.

[2134] Input: ID, password

[2135] Output: Authentication result (profile data if successful)

[2136] Operation: The ID and password entered by the user are sent to the server, which then authenticates them by checking them against the authentication information in the database. If authentication is successful, the user's profile data is sent to the device.

[2137] Step 2: Acquire the study materials

[2138] The server retrieves the educational material requested by the user from the database and transmits it to the terminal.

[2139] Input: User request (e.g., teaching material ID)

[2140] Output: Teaching material data (videos, text, quizzes, etc.)

[2141] Operation: When a user sends a requested educational material ID from a terminal to the server, the server retrieves the corresponding educational material from the database and sends it to the terminal.

[2142] Step 3: Collect learning activity data

[2143] The terminal collects the user's viewing time, quiz correct answer rate, etc. in real time and transmits them to the server.

[2144] Input: Learning activity data (viewing time, quiz answer results, etc.)

[2145] Output: Sending learning activity data to the server

[2146] Operation: When a user watches or listens to educational materials or answers quizzes, their activity data is collected on the device and sent to the server in real time.

[2147] Step 4: Data analysis

[2148] The server analyzes the collected learning activity data and evaluates the user's level of understanding.

[2149] Input: Learning activity data

[2150] Output: Comprehension assessment results

[2151] Operation: The server analyzes learning activity data and performs data calculations using statistical analysis and machine learning algorithms to evaluate the user's level of understanding.

[2152] Step 5: Generate feedback

[2153] Based on the evaluation results, the server generates feedback suggesting the next learning step or supplementary learning materials and sends this to the terminal.

[2154] Input: Comprehension assessment results

[2155] Output: Feedback (suggestions for next learning steps, supplementary materials, etc.)

[2156] Operation: Based on the comprehension assessment results, the server uses a generative AI model to generate feedback suggesting next steps and supplementary learning materials. The generated feedback is then sent to the device.

[2157] Step 6: Collect and analyze emotion data

[2158] The device sends the user's facial expressions and tone of voice during training via a camera and microphone to the emotion engine to recognize emotions.

[2159] Input: facial expression data, tone of voice

[2160] Output: Emotion recognition result

[2161] How it works: Facial expression data and voice tone captured by the device's camera and microphone are s...

Claims

1. a means for receiving authentication information from the student; means for authenticating the student based on the authentication information; means for retrieving selected educational materials from a database; means for transmitting the teaching materials to the student's terminal; means for collecting learning activity data of the students from the terminals; means for analyzing the learning activity data and evaluating the student's level of understanding; means for generating feedback based on the evaluation result and transmitting the feedback to the terminal; A system including:

2. The system of claim 1 further comprising means for suggesting a next learning step based on the evaluation results.

3. The system of claim 1 , further comprising means for generating the feedback individually and suggesting customized next learning material according to the student's progress.

Images