System

Abstract

A system is provided.SOLUTION: A learning system, comprising: means for collecting and storing information on a learning style and a learning history of a student; means for generating an optimal learning plan based on the collected learning style and learning history; means for presenting learning contents to the student based on the generated learning plan; and means for checking answers submitted by the student and providing explanations if necessary.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Traditional school education is based on the premise of group education, making it difficult to provide learning instruction tailored to each student's level of understanding, learning progress, and learning style. This calls for supplementary learning methods tailored to each individual, but due to constraints on the number of teachers and their working hours, it is difficult to provide homework and learning methods optimized for each student. Furthermore, individualized instruction methods such as cram schools and private tutors are expensive, and not all families can afford them. As a result, the learning gap between students widens, and fairness in educational opportunities is undermined. [Means for solving the problem]

[0005] To solve the above problems, the present invention provides a learning system that includes a means for collecting and saving information about a student's learning style and learning history, a means for generating an optimal learning plan based on the collected learning style and learning history, a means for presenting learning content to the student based on the generated learning plan, and a means for checking answers submitted by the student and providing explanations as needed.Furthermore, this system also includes a means for generating a learning plan by calling a generation AI to generate an optimal learning plan, and a means for analyzing the answers using image information to check the answers to the presented questions, making it possible to efficiently and effectively provide learning content and methods tailored to each student.

[0006] "Student" refers to a child or young person studying at an educational institution.

[0007] "Learning style" refers to the characteristics of individual methods and approaches that students use to learn most effectively.

[0008] "Learning history" refers to a record of what a student has learned so far, their grades, level of understanding, etc.

[0009] "Means for collecting and storing information" refers to the function of collecting data on students' learning styles and learning histories and recording it in a database, etc.

[0010] "Means for generating learning plans" refers to the function of designing optimal learning content and methods for each student based on collected information.

[0011] "Means for presenting learning content" refers to a function for visually or audibly presenting the content to be learned to students based on the generated learning plan.

[0012] "Means for checking answers and providing explanations" refers to the function of checking whether the answers submitted by students are correct or incorrect, and explaining the reasons for the errors and the correct answers if necessary.

[0013] "Generative AI" refers to artificial intelligence that automatically generates learning plans using AI technology.

[0014] "Means for analyzing answers using image information" refers to a function that uses image data to perform analysis in order to accurately check the answers submitted by students. [Brief explanation of the drawings]

[0015] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13]FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0016] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0017] First, the terms used in the following description will be explained.

[0018] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0019] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0020] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0021] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0022] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0023] [First embodiment]

[0024] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0025] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0026] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0027] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0028] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0029] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0030] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0031] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0032] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0033] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0034] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0035] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0036] This invention is a learning system that uses generative AI and image analysis technology to provide optimal learning plans based on each student's individual learning style and learning history. Below, we will explain the program processing of this system in natural language.

[0037] Collecting student information

[0038] 1. User (student) enters information

[0039] Users input information about their learning style and learning history into a device (such as a tablet or PC), such as "I'm good at math," "I'm good at repeated learning," or "I got 80 points on the last test."

[0040] 2. The device collects and transmits information

[0041] The device collects the information entered by the user and sends it to the server, which stores it in a database.

[0042] Suggestions for learning content and methods

[0043] 1. The server analyzes the information

[0044] The server extracts the student's learning style and past learning history from the database and analyzes this information.

[0045] 2. The server calls the generated AI

[0046] The server calls a generative AI (e.g., an AI model) and instructs it to generate an optimal learning plan for each student based on the analysis results. For example, it might instruct the server to "generate a set of math problems for students who are good at repetitive learning."

[0047] 3. Generative AI generates a learning plan

[0048] Generative AI creates customized study plans based on a student's learning style and history, such as a plan that includes a series of math problems or a set amount of reading each day.

[0049] 4. The server organizes and sends the study plan

[0050] The server organizes the learning plan received from the generation AI and sends it to the device.

[0051] Presentation of learning content

[0052] 1. The server sends the study plan to the device

[0053] The server sends the generated study plan to the terminal, which receives it and presents it to the user.

[0054] 2. The device displays the learning content

[0055] The device displays questions and learning materials to the user based on the received study plan, for example, displaying math problems or reading material on the screen.

[0056] 3. Users engage in learning

[0057] The user solves problems based on the learning content presented to them, either by typing the answers into the device or by speaking them through the audio glasses.

[0058] Check answers and explain

[0059] 1. User submits answer

[0060] Users can enter answers to the questions they have solved into the device, or they can enter answers by voice using audio glasses.

[0061] 2. The device sends the answer to the server

[0062] The terminal sends the answer entered by the user to the server, which accepts it and performs the necessary processing.

[0063] 3. The server checks the answer

[0064] The server calls the AI ​​generator and evaluates the user's answer against the correct answer. For example, it checks whether the answer to the question "What is 234 + 567?" is "801."

[0065] 4. The server generates the description

[0066] If the user's answer is incorrect, the server uses generative AI to generate text or audio that explains the error, for example, providing step-by-step instructions on how to arrive at the correct answer.

[0067] 5. The server sends the results and explanations to the device.

[0068] The server sends the results of the check and any necessary explanations to the terminal, which displays them to the user.

[0069] 6. The device displays the results and explanations

[0070] The device will show students whether their answers are correct or incorrect, and if they are incorrect, a detailed explanation will be displayed to help them further their learning.

[0071] Specific examples

[0072] For example, here is a fifth-grade student solving a math problem:

[0073] 1. Collecting student information

[0074] A user (e.g., Mr. Tanaka) inputs his / her learning style (he / she is good at repeated learning) and past test scores (80 points) into the terminal. The terminal sends this information to the server, which stores it in a database.

[0075] 2. Suggested learning content and methods

[0076] The server analyzes Tanaka's information, calls the generation AI, and instructs it to "generate a math problem set for students who are good at repeated learning." The generation AI generates the optimal problem set for Tanaka and returns it to the server. The server then sends it to the device.

[0077] 3. Presentation of learning content

[0078] The terminal displays the question "What is 234 + 567?" to Tanaka. Tanaka works on the question and enters the answer as "801."

[0079] 4. Check and explain your answers

[0080] The answer entered by Tanaka is sent from the device to the server, which checks the answer using the generation AI. If it is determined to be correct, the device displays "Correct!" If it is incorrect, the server uses the generation AI to generate an explanation for the mistake and sends it to the device. The device displays this to Tanaka and presents the next practice question.

[0081] This allows us to provide an optimal learning experience tailored to each student's individual characteristics.

[0082] The processing flow will be explained below.

[0083] Program processing flow

[0084] Collecting Student Information

[0085] Step 1:

[0086] The user (student) inputs information about their learning style and learning history into the terminal, such as "I'm good at repeating my learning" or "I got 80 points on the last math test."

[0087] Step 2:

[0088] The device collects the information entered by the user and sends it to the server, including the student ID, grade, learning style, and learning history.

[0089] Step 3:

[0090] The server stores the received information in a database, assigning an ID to each student's information to uniquely identify them.

[0091] Suggestions for learning content and methods

[0092] Step 1:

[0093] The server extracts the student's learning style and past learning history from the database, thereby understanding the student's current learning situation.

[0094] Step 2:

[0095] The server calls the AI ​​generator and instructs it to generate an optimal learning plan based on the extracted data (learning style, learning history), for example, by issuing a specific instruction such as "Generate math problems for a fifth-grade student who is good at repetitive learning."

[0096] Step 3:

[0097] Generative AI creates a personalized learning plan for each student, including multiple practice exercises and regular review.

[0098] Step 4:

[0099] The server receives the generated lesson plan, organizes it, and sends it to the device in the appropriate format.

[0100] Presentation of learning content

[0101] Step 1:

[0102] The server then sends the generated study plan to the device, which includes questions, study materials, and a study schedule.

[0103] Step 2:

[0104] The device then displays the received learning plan to the user, for example, displaying math exercises on the screen and providing audio and text explanations.

[0105] Step 3:

[0106] The user works on the presented problems and either types the answers into the terminal or submits the answers aloud via audio glasses.

[0107] Check answers and explain

[0108] Step 1:

[0109] After solving the problem, the user submits the answer by typing it into the terminal or by voice, and the answer is registered in the terminal.

[0110] Step 2:

[0111] The device sends the user's response to the server, which may include image information and audio data.

[0112] Step 3:

[0113] The server calls the generation AI and checks the answer submitted by the user against the correct answer, for example, checking whether '801' is the correct answer to 234 + 567.

[0114] Step 4:

[0115] The server determines whether the answer is correct, and if it is incorrect, the generative AI creates an explanation including the reason and the correct solution.

[0116] Step 5:

[0117] The server checks the answers and sends them to the device along with explanations, which may include reasons for the correct answers and additional practice questions.

[0118] Step 6:

[0119] The device will display the result and explanation to the student. Specifically, it will say "That's correct!" or "That's wrong. The correct answer is '801'" on the screen, along with a detailed explanation of why the answer was wrong.

[0120] Specific examples

[0121] For example, the processing flow when Tanaka, a fifth-grader, solves a math problem is shown below.

[0122] Step 1:

[0123] Tanaka types into the terminal, "I'm good at repeated learning and got 80 points on the last test."

[0124] Step 2:

[0125] The device collects Tanaka's information and sends it to the server.

[0126] Step 3:

[0127] The server stores the information in a database and calls a generative AI to instruct it to generate a "math problem set for students who excel at repetitive learning."

[0128] Step 4:

[0129] The generation AI generates a plan and sends it back to the server.

[0130] Step 5:

[0131] The server organizes the study plan and sends it to the device.

[0132] Step 6:

[0133] The terminal displays the question "What is 234 + 567?" to Tanaka, who then enters the answer "801."

[0134] Step 7:

[0135] The device sends the entered answers to the server, which checks the answers using a generation AI.

[0136] Step 8:

[0137] The server determines that the answer is correct and sends the result to the terminal.

[0138] Step 9:

[0139] The device displays to Tanaka, "That's correct! Well done!"

[0140] As a result, this learning system provides optimal learning plans and feedback tailored to each student's learning needs.

[0141] Example 1

[0142] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0143] Conventional learning systems face the challenge of providing optimal learning plans tailored to each student's individual characteristics. In particular, there are few systems that can provide individually customized learning plans in real time that properly reflect a student's learning style and past learning history. Furthermore, the process of generating learning plans using generative AI models requires both convenience and accuracy, making it a highly advanced technology. Therefore, there is a need for the development of a system that can efficiently provide learning support tailored to each student.

[0144] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0145] In this invention, the server includes means for collecting and storing information on students' learning styles and learning histories, means for generating an optimal learning plan based on the collected learning styles and learning histories, and means for presenting learning content to students based on the generated learning plan. This makes it possible to provide customized learning plans suited to the learning characteristics of individual students in real time, thereby improving learning efficiency.

[0146] "Learning style" refers to the characteristics that indicate how a student learns best, such as retrieval, visual learning, or reading-based learning.

[0147] "Learning history" refers to information that records a student's past learning activities and achievements, such as past test scores, study time, and what they learned.

[0148] "Means of collection" refers to devices and software that acquire information from users and organize and store it. Examples include tablets, PCs, and dedicated input systems.

[0149] "Storage means" refers to methods or systems for recording and storing collected data in an appropriate format, such as database management systems or cloud storage.

[0150] "Generative means" refers to methods or systems that generate new information or plans based on given input data, specifically generative AI models and algorithms.

[0151] "Presenting means" refers to a method or system for visually or audibly presenting the generated information or plan to the user, including, for example, display on a display or audio announcement.

[0152] "Means of checking" refers to techniques or methods for evaluating submitted data or responses to determine their accuracy. Examples include automated software evaluation and analysis using artificial intelligence.

[0153] "Means for providing explanations" refers to methods or systems for explaining the evaluation results or errors in a way that is easy for users to understand. For example, this could be a text-based explanation or a voice assistant explanation.

[0154] "Device" refers to the input / output device used by a user, including tablets, PCs, smartphones, etc.

[0155] "Server" refers to a central computer that stores and processes data via a network and manages and operates the entire system in cooperation with terminals.

[0156] "Database" refers to a digital information management system that systematically stores collected information and enables it to be quickly accessed and retrieved when needed.

[0157] A "generative AI model" refers to an artificial intelligence mechanism that automatically generates various outputs (such as learning plans) based on input data.

[0158] A "prompt sentence" refers to a text sentence that is input to give specific instructions to a generative AI model.

[0159] "REST API" refers to a set of definitions and protocols that allow different software applications to communicate with each other and exchange data.

[0160] "JSON" stands for JavaScript Object Notation and refers to a lightweight data interchange format for structuring, storing, and transferring data.

[0161] An "HTTP POST request" is an HTTP method for sending data to a web server, which can save or update data.

[0162] This invention is a learning system that utilizes generative AI models and image analysis technology to provide optimal learning plans based on each student's individual learning style and learning history. Specific embodiments of this system are described in detail below.

[0163] First, the user (student) uses a device (tablet or PC) to input information about their learning style and past learning history. For example, they provide information such as "I'm good at math," "I like repeated learning," and "I got 80 points on the last test." The device then organizes this information and sends it to the server.

[0164] The server uses a database management system (e.g., MySQL or PostgreSQL) to store the received information in a database. The server then extracts students' learning styles and past learning histories from the database and analyzes these data using data analysis tools (e.g., the Pandas module in Python).

[0165] Based on the analysis results, the server calls a generative AI model (e.g., GPT-3). It gives instructions to the AI ​​using prompts. Examples of specific prompts include:

[0166] "Generating appropriate math problem sets for users who are good at iterative learning"

[0167] etc.

[0168] The generative AI model generates an optimal study plan based on the provided prompts, which includes math problem sets, reading materials, etc. The generated study plan is sent back to the server in JSON format, which the server parses and sends to the device.

[0169] The device presents the user with learning content based on the received learning plan. Specifically, it uses HTML, CSS, and JavaScript to display various questions and learning materials on the screen. The user's answers to the questions are entered into the device or collected via a voice input device (e.g., audio glasses).

[0170] When a user submits an answer, the device sends it to the server. The server uses a generative AI model to check the user's answer and determine whether it is correct or incorrect. If the answer is incorrect, the server uses the generative AI model to generate an explanation for the mistake. This explanation is also sent to the device in JSON format, for example, and the device displays it to the user.

[0171] As a concrete example, consider a fifth-grade student solving a math problem. The user (student) inputs their learning style (good at repetitive learning) and past test scores (80 points) into the device. The device sends this information to the server, which stores it in a database. The server then sends a prompt to the generative AI model.

[0172] "Generating appropriate math problem sets for users who are good at iterative learning"

[0173] The generative AI model generates an optimal problem set and sends it back to the server. The server sends this to the device. The device then displays a question to the user, such as "What is 234 + 567?", to which the user answers, for example, "801." The user's answer is again sent to the server and checked by the generative AI model. If the answer is correct, a message indicating the correct answer is displayed, and if it is incorrect, an explanation is displayed.

[0174] This makes it possible to provide an optimal learning experience tailored to each student's individual characteristics.

[0175] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0176] Step 1: User Enters Information

[0177] Users use a device (tablet or PC) to input information such as their learning style and past grades. For example, they might say, "I'm good at math," "I'm good at repeated learning," or "I got 80 points on the last test." This information is collected as input data.

[0178] Step 2: Device collects and sends information

[0179] The terminal internally organizes the data entered by the user and converts it into JSON format, then uses a REST API to send this information to the server, which uses an HTTP POST request to transfer the data to the server. The server receives this data and stores it in a database.

[0180] Step 3: The server analyzes the information

[0181] The server extracts the user's learning style and learning history from the database. It then analyzes the data using Python's Pandas module, for example. For example, it calculates and visualizes the distribution of past test scores and learning style trends. The input here is the data extracted from the database, and the output is the analysis results.

[0182] Step 4: The server invokes the generative AI model

[0183] The server sends a prompt to the generative AI model based on the analysis results. A specific prompt might be, "Generate the optimal set of math problems for a user who is good at repeated learning." The input is the analysis results and the prompt, and the output is the generated learning plan.

[0184] Step 5: Generative AI model generates a learning plan

[0185] The generative AI model generates an optimal lesson plan based on the prompt it receives, including a set of math problems and reading material, and outputs a JSON-formatted lesson plan that is sent back to the server.

[0186] Step 6: The server organizes and sends the lesson plan

[0187] The server parses the JSON-formatted learning plan returned by the generative AI model, formats it as needed, and then sends the data to the device using a REST API. The output is a curated learning plan.

[0188] Step 7: Your device will display what it learned

[0189] The device uses HTML, CSS, and JavaScript to display the learning content based on the received lesson plan. For example, it displays math problems or reading material on the screen. In this case, the input is the lesson plan, and the output is the learning content on the user interface.

[0190] Step 8: Users engage in learning

[0191] The user progresses with their learning based on the displayed learning content, for example, by answering math questions. The input is the learning content, and the output is the user's answer.

[0192] Step 9: User submits answer

[0193] The user enters the answer to the problem into the terminal, for example, by typing "801" on a keyboard or by using voice input to answer "234 plus 567 is 801." The input is the user's answer, and the output is the data entered into the terminal.

[0194] Step 10: The device sends the answer to the server

[0195] The terminal organizes the user's answers and sends them to the server using an HTTP POST request. The input is the user's answer data, and the output is the data transferred to the server.

[0196] Step 11: Server checks the answer

[0197] The server calls the generative AI model and checks whether the user's answer is correct. For example, it evaluates whether "234 + 567 is 801." The input is the user's answer and the model's correct answer data, and the output is the correctness of the answer.

[0198] Step 12: Server generates commentary

[0199] If the answer is incorrect, the server uses a generative AI model to generate a detailed explanation for the mistake, for example, creating a step-by-step explanation for arriving at the correct answer. The input is the user's answer data and the prompt text, and the output is the explanation text.

[0200] Step 13: The server sends the results and explanations to the device.

[0201] The server sends the answer's correctness and explanation to the terminal. The output is the correctness and explanation data.

[0202] Step 14: Your device will display the results and explanations.

[0203] The device displays the received results and explanations to the user, for example, a message such as "Your answer is correct!" or "You made a mistake here. Please refer to the next step." The input is the result and explanation data, and the output is the display on the user interface.

[0204] (Application example 1)

[0205] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0206] Existing learning systems have limitations in providing optimal learning plans that are tailored to each student's individual learning style and learning history. They also lack the ability to provide real-time learning support and feedback, resulting in insufficient support for students to learn efficiently. Furthermore, the methods of presenting learning content and providing feedback are inconsistent, resulting in reduced learning effectiveness. Therefore, there is a need for an effective learning system that meets the individual needs of each student.

[0207] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0208] In this invention, the server includes means for collecting and storing information about the student's learning style and learning history, means for generating an optimal learning plan based on the collected learning style and learning history, means for presenting learning content to the student based on the generated learning plan, means for displaying the learning content and feedback using a glasses-type device worn by the student, and means for checking answers submitted by the student and providing explanations as necessary. This allows for the provision of a learning plan optimal for each student's individual learning style in real time, enabling efficient learning support.

[0209] "Student" refers to a learner who uses the learning system.

[0210] "Learning style" refers to the individual characteristics that indicate how students learn most effectively.

[0211] "Learning history" refers to a record of what a student has learned in the past and their grades.

[0212] "Collection and storage means" refers to methods and devices for capturing and storing information about students' learning styles and learning histories.

[0213] A "learning plan" refers to a plan that organizes learning content and methods in a way that is optimal for each student.

[0214] "Generating means" refers to a method or system that creates a learning plan based on the collected information.

[0215] "Presenting means" refers to a method or device for displaying the generated lesson plan to students.

[0216] "Eyeglasses-type device" refers to a wearable device that students wear and use to provide visual information.

[0217] "Feedback" refers to the evaluation and explanation given to students in response to their learning results and answers.

[0218] "Means of checking" refers to the method or device that evaluates the answers submitted by students and determines whether they are correct or incorrect.

[0219] "Means for providing explanations as needed" refers to a method or system that provides guidance or explanations to the correct answer when the answer is incorrect.

[0220] "Generative AI" refers to a system that uses artificial intelligence to generate new data and plans.

[0221] A "prompt sentence" refers to an input sentence that gives specific generation instructions to the generation AI.

[0222] The following system components and process steps are used to implement the invention.

[0223] System Components

[0224] 1. Eyeglass-type device worn by students

[0225] Hardware: Eyeglasses (e.g. Google Glass)

[0226] Software: Application for viewing learning content and receiving feedback

[0227] 2. Server

[0228] Hardware: High-performance server

[0229] Software: Database management systems (e.g., MySQL), generative AI (e.g., OpenAI GPT models), data analysis tools (e.g., Python-based analysis systems)

[0230] Process Overview

[0231] The server generates an optimal learning plan based on the student's learning style and learning history, and displays the learning content and feedback in real time on the glasses-type device worn by the student. Every time the student takes an action, that information is sent to the server, which analyzes the data and generates and provides the next appropriate learning content.

[0232] Detailed processing contents

[0233] 1. Information gathering

[0234] User: Students provide their learning style and learning history by speaking into the glasses, scanning a QR code, or manually entering information.

[0235] Device: The glasses send the collected information to a smartphone via Bluetooth, which then transmits it to a server via Wi-Fi.

[0236] 2. Data storage and analysis

[0237] Server: The server stores the received information in a database (MySQL) and uses data analysis tools to analyze learning styles and history.

[0238] 3. Create a learning plan

[0239] Server: Based on the analysis results, the server sends prompts to a generative AI model (e.g., OpenAI's GPT model) to generate an optimal learning plan. Example prompts:

[0240] Generate math problem sets suitable for middle school students who enjoy repetition.

[0241] 4. Presentation of learning content

[0242] Server: Sends the generated learning plan to the eyeglasses device.

[0243] Device: The glasses-type device displays the learning content according to the received learning plan.

[0244] 5. Providing Feedback

[0245] User: The student engages with the learning content and provides answers via voice or manual input into the glasses.

[0246] Device: The answer is sent to the server, where the generation AI determines whether it is correct or incorrect and generates feedback.

[0247] 6. Representation

[0248] Server: Sends feedback to the glasses-type device and prepares the next appropriate learning content.

[0249] Device: The glasses display feedback and new learnings.

[0250] Specific examples

[0251] For example, here is a scenario where a middle school student is taking extra math classes:

[0252] 1. Information gathering: Students speak into the glasses-like device, saying, "I want to solve a math problem."

[0253] 2. Data storage and analysis: The server receives the voice data and stores it in a database. The server analyzes the student's learning history and determines whether the student is a "junior high school student who excels at repeated learning."

[0254] 3. Learning plan generation: The server sends the following prompt to the generative AI model:

[0255] Generate math problem sets suitable for middle school students who enjoy repetition.

[0256] The generated problem set (e.g., "What is 234 + 567?") is sent to the eyeglass-type device.

[0257] 4. Presentation of learning content: The eyeglasses display "What is 234 + 567?"

[0258] 5. Providing feedback: The student types "801" by voice. The server uses generative AI to determine whether the answer is correct or not, and provides feedback saying, "That's correct!"

[0259] 6. Re-presentation: The server generates the next question and sends it to the glasses-type device. The learning content is displayed again.

[0260] This allows us to provide a learning experience that best suits each student's individual learning style.

[0261] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0262] Step 1:

[0263] Information collection and transmission

[0264] User: Students provide their learning style and learning history by speaking into the glasses, scanning a QR code, or manually entering their learning history.

[0265] Input: Learning style characteristics and learning history (e.g., you are good at repeating things, you got 80 points on the last test, etc.).

[0266] Output: Audio data and manual input data.

[0267] How it works: The glasses send the collected information to a smartphone via Bluetooth, and then to a server via Wi-Fi.

[0268] Step 2:

[0269] Data storage and analysis

[0270] Server: The server stores the received information in a database (e.g., MySQL) and uses a data analysis tool (e.g., a Python-based analysis system) to analyze learning styles and histories.

[0271] Input: Voice data and manual input data.

[0272] Output: Analysis results based on learning style and history.

[0273] Specific operation: After storing the information in the database, the server analyzes the learning style and history and generates prompts for the generative AI model.

[0274] Step 3:

[0275] Generate a learning plan

[0276] Server: Based on the analysis results, it sends prompts to a generative AI model (e.g., OpenAI's GPT model) to generate an optimal learning plan.

[0277] Input: Analysis results based on learning style and history, prompt statement.

[0278] Output: The generated learning plan.

[0279] Specific operation: The server sends the following prompt to the generative AI model: "Please generate a set of math problems suitable for junior high school students who are good at repetitive learning." The generative AI then generates a study plan and sends it back to the server.

[0280] Step 4:

[0281] Presentation of learning content

[0282] Server and device: The server sends the generated learning plan to the eyeglass device, which then displays the learning content to the student.

[0283] Input: The generated learning plan.

[0284] Output: Learning content displayed on the eyeglasses device.

[0285] Specific operation: The glasses-type device presents students with questions such as "What is 234 + 567?"

[0286] Step 5:

[0287] Enter and submit your answers

[0288] User: The student engages with the learning content and provides answers via voice or manual input into the glasses.

[0289] Input: Student's answer (e.g., "801").

[0290] Output: Audio data or manually input data.

[0291] How it works: The glasses send students' responses via Bluetooth to a smartphone, which then transmits them to a server via Wi-Fi.

[0292] Step 6:

[0293] Checking answers and generating feedback

[0294] Server: Checks the received answers using the generative AI model and generates feedback (explanation) if necessary.

[0295] Input: Student answers.

[0296] Output: Correct / incorrect judgment and feedback (explanation).

[0297] What it does: The server matches the student's answer with the correct answer and uses a generative AI model to generate feedback, such as "That's right!" or an explanation if the answer is incorrect.

[0298] Step 7:

[0299] Providing feedback and next steps

[0300] Server and device: The server sends the generated feedback to the eyeglasses device, which prepares the next appropriate learning content. The eyeglasses device displays the feedback and new learning content to the student.

[0301] Input: Feedback and next learning plan.

[0302] Output: Feedback and next steps displayed on the glasses.

[0303] What it does: The glasses provide feedback and next steps to students, providing continuous support for their learning.

[0304] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0305] This invention is a learning system that recognizes a student's learning style, learning history, and emotions and provides an optimal learning plan. The program processing of this system will be specifically explained below.

[0306] Collecting student information

[0307] 1. User (student) enters information

[0308] The user inputs information about their learning style and learning history into the terminal, such as "I'm good at repeating my learning" or "I got 80 points on the last test."

[0309] 2. The device collects and transmits information

[0310] The device collects the information entered by the user and sends it to the server, including the student ID, grade, learning style, and learning history.

[0311] 3. The server stores the information

[0312] The server stores the received information in a database, assigning an ID to each student's information to uniquely identify them.

[0313] Emotion recognition

[0314] 1. Real-time video and audio input by users

[0315] While studying, users input video and audio into the device in real time using a webcam and microphone.

[0316] 2. The device performs emotion analysis

[0317] The device sends the collected video and audio data to an emotion engine that recognizes the student's emotions, such as "concentrating," "tired," or "enjoying" in real time.

[0318] 3. Sending Emotional Data

[0319] The analyzed emotional data is sent from the device to a server and used when generating a learning plan.

[0320] Suggestions for learning content and methods

[0321] 1. The server analyzes the information

[0322] The server extracts information from the database about students' learning styles and past learning histories, as well as emotional data collected in real time, and analyzes this information.

[0323] 2. The server calls the generated AI

[0324] The server calls the AI ​​generator and instructs it to generate an optimal learning plan based on the analysis results, for example, by issuing specific instructions such as "generate math problems for students who are good at repetitive learning and who are in a focused state."

[0325] 3. Generative AI generates a learning plan

[0326] The AI ​​generates optimal learning plans based on a student's learning style and emotions. For example, if a student is concentrating, it will increase the difficulty of the questions slightly, and if it determines that the student is tired, it will generate a plan that includes instructions to take a break.

[0327] 4. The server organizes and sends the study plan

[0328] The server organizes the learning plan received from the generation AI and sends it to the device in the appropriate format.

[0329] Presentation of learning content

[0330] 1. The server sends the study plan to the device

[0331] The server then sends the generated study plan to the device, which includes questions, study materials, and a study schedule.

[0332] 2. Your device will display your study plan

[0333] The device then displays the received study plan to the user, for example, displaying math practice problems on the screen and providing dynamic instructions such as "concentrate" and "take breaks as you progress."

[0334] 3. Users engage in learning

[0335] The user studies based on the presented study plan, either by typing answers into the device or by submitting answers aloud via audio glasses.

[0336] Check answers and explain

[0337] 1. User submits answer

[0338] The user submits the answer to the problem by typing it into the terminal or by voice, and the answer is registered in the terminal.

[0339] 2. The device sends the answer to the server

[0340] The device sends the user's response to the server, along with their emotional data if necessary.

[0341] 3. The server checks the answer

[0342] The server calls the generation AI and checks the answer submitted by the user against the correct answer, for example, checking whether '801' is the correct answer to 234 + 567.

[0343] 4. The server generates the description

[0344] If the user's answer is incorrect, the server uses a generative AI to create an explanation including the reason and the correct solution. It also provides feedback according to the user's emotional state.

[0345] 5. The server sends the results and explanations to the device.

[0346] The server checks the answers and sends them to the device along with explanations, including reasons for the correct answers, additional practice questions, and even encouragement and advice based on the user's emotional state.

[0347] 6. The device displays the results and explanations

[0348] The device displays the results and explanations to the student. Specifically, it displays messages such as "That's right! Well done!" or "That's wrong. The correct answer is '801'" on the screen, and also provides feedback that takes emotions into account.

[0349] Specific examples

[0350] For example, the processing flow when Tanaka, a fifth-grader, solves a math problem is shown below.

[0351] Step 1:

[0352] Tanaka types into the terminal, "I'm good at repeated learning and got 80 points on the last test."

[0353] Step 2:

[0354] Video and audio data collected in real time was analyzed to determine that Tanaka was "concentrating."

[0355] Step 3:

[0356] The device collects Tanaka's information and emotional data and sends it to the server.

[0357] Step 4:

[0358] The server stores the information in a database and calls the generation AI, instructing it to generate a "math problem set suitable for Tanaka, a student who excels at repeated learning, and who is currently concentrating."

[0359] Step 5:

[0360] The generation AI generates a plan and sends it back to the server.

[0361] Step 6:

[0362] The server organizes the learning plan and sends it to the device, displaying a question such as "What is 234 + 567?"

[0363] Step 7:

[0364] Tanaka responds by entering "801." The device then sends the response and emotion data to the server.

[0365] Step 8:

[0366] The server checks the answer using the generation AI and determines it to be correct.

[0367] Step 9:

[0368] The server sends the result to the device, which then displays to Tanaka, "That's correct! Maintain your current concentration."

[0369] This allows the learning system to provide optimal learning plans and feedback tailored to each student's learning needs, promoting efficient and effective learning.

[0370] The processing flow will be explained below.

[0371] Program processing flow

[0372] Collecting student information

[0373] Step 1:

[0374] The user (student) inputs information about their learning style and learning history into the terminal. Specifically, they input information such as "I'm good at repeating my learning" or "I got 80 points on the last test."

[0375] Step 2:

[0376] The device collects the information entered by the user and sends it to the server, including the student ID, grade, learning style, and learning history.

[0377] Step 3:

[0378] The server stores the received information in a database, assigning an ID to each student's information to uniquely identify them.

[0379] Emotion recognition

[0380] Step 1:

[0381] While learning, users input video and audio into the device in real time, and the device uses a webcam and microphone to collect the user's facial expressions and tone of voice.

[0382] Step 2:

[0383] The device sends the collected video and audio data to the emotion engine, which analyzes this data and recognizes the user's emotions, such as "concentrated," "tired," or "enjoyed."

[0384] Step 3:

[0385] The emotion data analyzed by the emotion engine is sent to the server along with the user's learning style and learning history.

[0386] Suggestions for learning content and methods

[0387] Step 1:

[0388] The server extracts the student's learning style and past learning history from the database, as well as emotional data collected in real time, and analyzes this information.

[0389] Step 2:

[0390] The server calls the AI ​​and instructs it to generate an optimal learning plan based on the extracted data (learning style, learning history, emotional data).For example, it gives specific instructions such as, "Generate math problems for students who are good at repetitive learning and are in a focused state."

[0391] Step 3:

[0392] The generative AI generates optimal study plans based on a student's learning style and emotions. For example, if a student is concentrating, it might increase the difficulty of the questions slightly, or if it detects fatigue, it might provide a plan that includes instructions to take a break.

[0393] Step 4:

[0394] The server receives the generated lesson plan, organizes it, and sends it to the device in the appropriate format.

[0395] Presentation of learning content

[0396] Step 1:

[0397] The server then sends the generated study plan to the device, which includes questions, study materials, and a study schedule.

[0398] Step 2:

[0399] The device then displays the received study plan to the user, for example, displaying math exercises on the screen and providing dynamic instructions such as "stay focused" or "you look tired, take a break."

[0400] Step 3:

[0401] The user studies based on the presented study plan, either by typing answers into the device or by submitting answers aloud via audio glasses.

[0402] Check answers and explain

[0403] Step 1:

[0404] After solving the problem, the user submits the answer by typing it into the terminal or by voice, and the answer is registered in the terminal.

[0405] Step 2:

[0406] The device sends the user's response to the server, along with their emotional data if necessary.

[0407] Step 3:

[0408] The server calls the generation AI and checks the answer submitted by the user against the correct answer, for example, checking whether '801' is the correct answer to 234 + 567.

[0409] Step 4:

[0410] The server determines whether the answer is correct, and if it is incorrect, the generative AI creates an explanation including the reason and the correct solution. It also provides feedback based on the user's emotional state.

[0411] Step 5:

[0412] The server checks the answers and sends them to the device along with explanations, including reasons for the correct answers, additional practice questions, and even encouragement and advice based on the user's emotional state.

[0413] Step 6:

[0414] The device displays the results and explanations to the student, such as "That's right! Well done!" or "That's wrong. The correct answer is '801'," and also provides emotional feedback.

[0415] Specific examples

[0416] For example, the processing flow when Tanaka, a fifth-grader, solves a math problem is shown below.

[0417] Step 1:

[0418] Tanaka types into the terminal, "I'm good at repeated learning and got 80 points on the last test."

[0419] Step 2:

[0420] Video and audio data collected in real time was analyzed to determine that Tanaka was "concentrating."

[0421] Step 3:

[0422] The device collects Tanaka's information and emotional data and sends it to the server.

[0423] Step 4:

[0424] The server stores the information in a database and calls the generation AI, instructing it to generate a "math problem set suitable for Tanaka, a student who excels at repeated learning, and who is currently concentrating."

[0425] Step 5:

[0426] The generation AI generates a plan and sends it back to the server.

[0427] Step 6:

[0428] The server organizes the study plan and sends it to the device, which then displays the question "What is 234 + 567?" to Tanaka.

[0429] Step 7:

[0430] Tanaka responds by entering "801." The device then sends the response and emotion data to the server.

[0431] Step 8:

[0432] The server checks the answer using the generation AI, and if it is determined to be correct, it sends the result to the device.

[0433] Step 9:

[0434] The device displays to Tanaka, "That's correct! Maintain your current concentration."

[0435] This allows the learning system to provide optimal learning plans and feedback tailored to each student's learning needs, promoting efficient and effective learning.

[0436] Example 2

[0437] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0438] Modern learning systems are expected to provide individualized learning plans based on students' learning styles and learning histories. However, traditional systems only use static data and are unable to adjust learning plans to take students' real-time emotions into account. This makes it difficult to provide dynamic learning plans that optimally maintain students' focus and motivation.

[0439] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0440] In this invention, the server includes means for collecting and storing information on students' learning styles and learning histories, means for generating optimal learning plans based on the collected learning styles and learning histories, means for collecting real-time video and audio data, performing emotion analysis, and storing the data, and means for adjusting and generating learning plans based on the analyzed emotion data, thereby making it possible to provide learning plans that take into account the dynamic emotional states of students.

[0441] "Student learning style" refers to the individual characteristics of how and how students prefer to learn.

[0442] "Learning history" refers to data recorded about what learning activities a student has engaged in in the past, including the results and achievements.

[0443] "Means for collecting and storing information" refers to devices and systems for inputting, recording, and storing information about students' learning styles and learning histories.

[0444] "Means for generating optimal learning plans" refers to devices or systems that create learning content and schedules that are most suitable for individual students based on collected information.

[0445] "Means for presenting learning content to students" refers to devices or systems that display and provide specific learning tasks and learning materials to students based on the generated learning plan.

[0446] "Means for checking answers and providing explanations" refers to a device or system that verifies the answers submitted by students, determines whether they are correct, and provides explanations or feedback as needed.

[0447] "Real-time video and audio data" refers to video and audio information that records students' situations and reactions in real time while they are learning.

[0448] "Emotion analysis" refers to the process of analyzing and recognizing students' emotional states (e.g., focused, tired, or enjoying) from video and audio data collected in real time.

[0449] "Generative AI" refers to artificial intelligence models that automatically perform specific tasks (e.g., generating lesson plans, checking answers) based on collected data.

[0450] "Study plan adjustment" refers to the process of dynamically modifying and optimizing existing study plans based on data obtained through sentiment analysis.

[0451] This invention is a learning system that recognizes a student's learning style, learning history, and emotions to provide an optimal learning plan. Next, the program processing of this system will be specifically explained.

[0452] Hardware and software used

[0453] This system uses the following hardware and software:

[0454] Server: Database management system (e.g., MySQL), generative AI (e.g., OpenAI GPT-4), data analysis library (e.g., pandas, NumPy)

[0455] Device: Webcam, microphone, emotion analysis engine (e.g., OpenCV, TensorFlow), user interface

[0456] Communication protocol: HTTPS

[0457] Processing flow

[0458] Collecting student information

[0459] 1. The user enters information

[0460] The user inputs information about their learning style and learning history into the terminal, such as "I'm good at repeating my learning" or "I got 80 points on the last test."

[0461] 2. The device collects information and sends it to the server

[0462] The terminal collects the information entered by the user (student ID, grade, learning style, learning history) and sends it to the server via an HTTPS request.

[0463] 3. The server stores the information in a database

[0464] The server stores the received information in a database, using an SQL query to insert the input data using the student ID as the primary key.

[0465] Emotion recognition

[0466] 1. Users input real-time video and audio

[0467] While studying, users input video and audio into the device using a webcam and microphone.

[0468] 2. The device performs emotion analysis

[0469] The device sends the collected video and audio data to an emotion analysis engine, which analyzes the user's emotional state (concentrated, tired, enjoying, etc.) in real time.

[0470] 3. Sending Emotional Data

[0471] The device sends the analyzed emotion data to the server in JSON format.

[0472] Suggestions for learning content and methods

[0473] 1. The server analyzes the information

[0474] The server extracts students' learning styles, past learning history, and real-time emotional data from the database and analyzes the data using Python scripts.

[0475] 2. The server calls the generated AI

[0476] The server calls the generative AI and sends prompts that generate the optimal learning plan based on the analysis results.

[0477] Example: "Generate math problems for students who are good at repetition and are in a focused state."

[0478] 3. Generative AI generates learning plans

[0479] Based on the prompts, the generative AI generates an optimal study plan that includes the difficulty of the questions, the content to be studied, and a schedule.

[0480] 4. The server organizes and sends the generated lesson plans

[0481] The server organizes the learning plan received from the generation AI and sends it to the terminal in JSON format.

[0482] Presentation of learning content

[0483] 1. The server sends the study plan to the device

[0484] The server sends the organized lesson plan to the device, which is sent as an HTTP response.

[0485] 2. Your device will display your study plan

[0486] The device displays the received lesson plan on the screen to the user. For example, the device displays a math problem such as "What is 234 + 567?"

[0487] 3. Users engage in learning

[0488] The user solves problems based on the presented learning plan, and submits answers by typing them into the device or by voice.

[0489] Check answers and explain

[0490] 1. User submits answer

[0491] Users submit their answers to the questions they have solved by typing them into the device or by voice, and the answer data is saved on the device.

[0492] 2. The device sends the answer to the server

[0493] The device sends the user's response data to the server, including emotion data if necessary.

[0494] 3. The server checks the answer

[0495] The server calls the AI ​​generator and checks the user's answer against the correct answer. It sends a prompt to the AI ​​generator asking if '801' is the correct answer to 234 + 567.

[0496] 4. The server generates the description

[0497] If the user answers incorrectly, the server uses the generative AI again to generate an explanation, including the reason for the incorrect answer and the correct solution. It also provides feedback based on the user's emotional state.

[0498] 5. The server sends the results and explanations to the device.

[0499] The server then sends the results of the check and explanation to the device, including reasons for the correct answer, additional practice questions, and encouragement or advice based on the user's emotional state.

[0500] 6. The device will display the results and explanations.

[0501] The device displays the results and explanations on the screen, such as "That's right! Well done!" or "That's wrong. The correct answer is '801'," and also provides emotional feedback.

[0502] This allows the learning system to provide optimal learning plans and feedback tailored to each student's individual needs, promoting efficient and effective learning.

[0503] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0504] Step 1:

[0505] The user enters information

[0506] The user inputs information about their learning style and learning history into the device. For example, they might input information such as "I'm good at repeating my learning" or "I got 80 points on the last test." The input data includes student ID, grade, learning style, learning history, etc. The device temporarily stores this input data in its memory.

[0507] Step 2:

[0508] The device collects information and sends it to the server

[0509] The device collects information entered by the user and sends it to the server using an HTTPS request (input: user information, output: data sent to the server). The collected data is converted into JSON format and sent to the server using the HTTPS protocol.

[0510] Step 3:

[0511] The server stores the information in a database

[0512] The server saves the received information in a database. Specifically, it inserts the information into the database as a new record using an SQL query (input: user information, output: save to database). The saved data includes a unique student ID.

[0513] Step 4:

[0514] Users input real-time video and audio

[0515] While studying, users input real-time video and audio data into the device using a webcam and microphone. For example, while the user is solving a math problem, the device collects video of the user's facial expressions and records audio of their reactions (input: real-time video and audio data, output: temporary storage on the device).

[0516] Step 5:

[0517] The device performs emotion analysis

[0518] The device sends the collected video and audio data to an emotion analysis engine, which analyzes the user's emotions in real time. Specifically, OpenCV and TensorFlow are used to recognize emotional states such as "concentrated," "tired," and "enjoyed" (input: video and audio data, output: emotional state data).

[0519] Step 6:

[0520] The device sends emotion data to the server.

[0521] The device sends the analyzed emotion data to the server in JSON format, which includes the emotion data acquired in real time and the student ID (input: emotion state data, output: data transmission to the server).

[0522] Step 7:

[0523] The server analyzes the information

[0524] The server extracts and analyzes students' learning styles, learning histories, and real-time emotional data from the database. It then analyzes the data using Python scripts and data analysis libraries (e.g., pandas, NumPy) to determine the optimal learning plan for each student (input: student information, learning history, emotional data; output: analysis results).

[0525] Step 8:

[0526] The server calls the generated AI

[0527] The server calls a generative AI (e.g., OpenAI GPT-4) and sends a prompt that generates an optimal learning plan based on the analysis results (input: analysis results, output: generation of prompt).

[0528] Example: "Generate math problems for students who are good at repetition and are in a focused state."

[0529] Step 9:

[0530] Generative AI generates learning plans

[0531] The generation AI generates an optimal learning plan based on the prompt received from the server. The plan includes the difficulty level of the questions, learning content, and schedule (input: prompt, output: learning plan).

[0532] Step 10:

[0533] The server organizes and sends the generated lesson plans

[0534] The server organizes the learning plan received from the generation AI and sends it to the device. It converts it into the appropriate format in JSON format and sends it as an HTTPS response (input: learning plan, output: transmission to device).

[0535] Step 11:

[0536] Your device will display your study plan

[0537] The device displays the learning plan received from the server to the user. For example, a math problem such as "What is 234 + 567?" is displayed on the screen (input: learning plan, output: display to user).

[0538] Step 12:

[0539] Users engage in learning

[0540] The user solves the exercises based on the presented learning plan, and either enters the answers into the device or submits them by voice (input: user's answers, output: input to the device).

[0541] Step 13:

[0542] User submits answer

[0543] The user inputs the answer to the problem they solved into the terminal, which temporarily stores this data (input: user's answer, output: saved in the terminal).

[0544] Step 14:

[0545] The device sends the answer to the server

[0546] The device sends the user's response in JSON format to the server, including real-time emotional data if necessary (input: user's response, output: transmission to the server).

[0547] Step 15:

[0548] The server checks the answer

[0549] The server calls the generation AI and checks the answer submitted by the user by comparing it with the correct answer. For example, it checks whether "801" is the correct answer to 234 + 567" (input: user answer, output: correct or incorrect).

[0550] Step 16:

[0551] The server generates the description

[0552] When a user gives an incorrect answer, the server uses a generative AI to generate an explanation including the reason and the correct solution. It also provides feedback according to the user's emotional state (input: judgment result of correctness, output: explanation and feedback).

[0553] Step 17:

[0554] The server sends the results and explanations to the device.

[0555] The server checks the answers and sends them to the device together with an explanation. The results include reasons for the correct answer, additional practice questions, and encouragement and advice based on the user's emotional state (input: explanation and feedback, output: transmission to device).

[0556] Step 18:

[0557] The device displays the results and explanations

[0558] The device displays the result and explanation on the screen. Specifically, it displays "That's right! Well done!" or "That's wrong. The correct answer is '801'," and also provides feedback that takes emotions into account (input: result and explanation, output: displayed to the user).

[0559] (Application example 2)

[0560] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0561] Conventional learning and work support systems have had difficulty providing optimal plans that accurately and quickly reflect an individual's learning and work style, as well as their emotional state, which changes in real time. As a result, it has been difficult to achieve effective learning and efficient work support, leading to problems such as reduced learning effectiveness and work efficiency.

[0562] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for collecting and saving information on students' learning styles and learning histories, means for generating an optimal learning plan based on the collected learning styles and learning histories, means for presenting learning content to students based on the generated learning plan, means for checking answers submitted by students and providing explanations as necessary, means for collecting and saving information on workers' working styles and working histories, means for generating an optimal work plan based on the collected working styles and working histories, means for recognizing emotional states in real time and adjusting the work plan based thereon, means for recognizing the worker's fatigue level and concentration level using an emotion analysis engine, means for transmitting the collected emotional data to the server, means for calling a generation AI based on the collected information and generating an optimal work plan using prompt sentences, and means for presenting work content to workers based on the generated work plan and providing appropriate feedback. This makes it possible to provide plans that optimally correspond to individual learning and working styles and emotional states that change in real time.

[0563] "Learning style" refers to the particular methods and attitudes that students exhibit toward learning.

[0564] "Learning history" refers to a record of what a student has learned and their grades.

[0565] "Generative AI" refers to a system that uses artificial intelligence to generate new information or plans based on specific input information.

[0566] A "prompt" is an instruction given to a generating AI to perform a specific task.

[0567] "Work style" refers to the specific methods and attitudes that a worker exhibits toward work.

[0568] "Work history" refers to a record of the work that a worker has done so far and their achievements.

[0569] An "emotion analysis engine" refers to a system that analyzes collected video and audio data to recognize an individual's emotional state.

[0570] "Fatigue level" refers to an index that indicates the degree of fatigue of workers or students.

[0571] "Concentration" refers to an indicator that shows the current level of concentration of a worker or student.

[0572] A "learning plan" refers to a curriculum designed to help students learn effectively.

[0573] A "work plan" refers to steps and tasks planned to help workers carry out their work efficiently.

[0574] This invention is a system aimed at improving the learning and work efficiency of students and workers. The system generates and presents optimal plans by taking into account the student's learning style and learning history, the worker's working style and work history, and their real-time emotional state.

[0575] Hardware and software used

[0576] Hardware: Tablet device, camera, microphone, AR glasses

[0577] software:

[0578] EmotionRecognition (emotion analysis engine)

[0579] AITaskOptimizer (generative AI engine)

[0580] ServerCommunication (server communication module)

[0581] Processing flow

[0582] Collection of information

[0583] The terminal collects learning and work style and history information from the users (students and workers). For example, students input "I'm good at repeated learning" or "I got 80 points on the last test," while workers input "I'm good at concentrating" or "I made few errors on this line."

[0584] Emotion recognition

[0585] While students and workers are working, real-time video and audio data is collected via cameras and microphones, and EmotionRecognition is used to analyze their emotional states, such as their level of concentration and fatigue.

[0586] Sending data

[0587] The collected information and emotion data are sent to a server, which stores this information in a database and assigns a unique ID to each user.

[0588] Generate a plan

[0589] The server uses generative AI to generate optimal learning and work plans for the user, creating customized plans based on collected learning and work styles, past history, and current emotional state.

[0590] An example of a prompt is as follows:

[0591] "The current worker is very fatigued, please suggest a break."

[0592] "Because you're in a focused state, suggest specific tasks that will take your work speed to the next level."

[0593] Presenting the proposal

[0594] The generated plan is then sent back to the device and presented to the user via a tablet or AR glasses, with dynamic instructions such as "concentrate" or "take a break."

[0595] Answer checking and feedback

[0596] When a user submits the results of their progress based on their learning and work plan, the server uses generative AI to check the accuracy of the answers. It examines the answers to the presented questions and tasks, generates explanations and feedback as needed, and displays them on the device.

[0597] In this way, this system can greatly improve the efficiency of users' learning and work.

[0598] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0599] Step 1:

[0600] Users input information. Specifically, students use tablet devices to input things like, "I'm good at repeated learning" or "I got 80 points on the last test." Workers input information like, "I'm good at concentrating on work" or "I made few errors on this line." This allows data to be collected about learning and work styles and past history. Learning style and history information is obtained as input data and temporarily saved on the device.

[0601] Step 2:

[0602] The device collects video and audio data in real time. It uses a camera and microphone to capture images and voices of users while they are studying or working. The EmotionRecognition engine analyzes this data and recognizes emotional states such as "concentrated" or "tired." The input is video and audio data, and the output is emotional state information.

[0603] Step 3:

[0604] The device sends the collected learning / working style, history data, and emotional state information to the server, which stores this information in a database and assigns a unique ID to each user. The input is the user's learning / working information and emotional information, and the output is a database entry in which this information is stored.

[0605] Step 4:

[0606] The server calls its generative AI model based on the collected information. It issues a prompt to the generative AI, saying, "Please generate a learning plan or work plan that is optimal for the user's current state." The input is learning and work information and emotional information, and the output is the generated plan.

[0607] Step 5:

[0608] The generative AI follows instructions and generates an optimal learning or work plan for the user. For example, it suggests more difficult tasks for a focused user, or a schedule that includes breaks for a tired user. The input is a prompt and user information, and the output is a specific plan.

[0609] Step 6:

[0610] The server organizes the generated plan and sends it to the terminal, where it is converted into the appropriate format for presentation to the user. The input is the generated plan and the output is the formatted plan.

[0611] Step 7:

[0612] The device then displays the received learning or work plan to the user, using a tablet or AR glasses to provide instructions such as "concentrate" or "move to the next task." The input is the formatted plan, and the output is the displayed instructions.

[0613] Step 8:

[0614] The user works based on the presented learning plan or work plan. The student enters the answers to the problems he or she solves into the tablet, and the worker completes the specified task. The input is the result of the user's work, and the output is the answer or task result entered into the device.

[0615] Step 9:

[0616] The device sends the answers and work results submitted by the user to the server. If necessary, emotional data is also sent. The input is the answers and work results, and the output is the data sent to the server.

[0617] Step 10:

[0618] The server uses a generation AI to check the user's answers and work results, and generates a judgment of whether they are correct or incorrect and any necessary explanations. For example, it generates feedback such as "The calculation result of 801 is correct" or "You should take a break." The input is the user's answers and work results, and the output is the results with feedback.

[0619] Step 11:

[0620] The server sends the results and explanations to the device, which include the correct solution, improvements, and advice based on the user's emotional state. The input is the result with feedback, and the output is the explanation sent to the device.

[0621] Step 12:

[0622] The device displays the received result and explanation to the user. For example, it displays "That's right! Well done!" or "That's wrong. The correct answer is 801," providing emotional feedback. The input is the result with feedback, and the output is the displayed explanation and feedback.

[0623] Through these steps, this system can significantly improve the efficiency of users' learning and work.

[0624] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0625] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0626] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0627] [Second embodiment]

[0628] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0629] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0630] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0631] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0632] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0633] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0634] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0635] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0636] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0637] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0638] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0639] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0640] This invention is a learning system that uses generative AI and image analysis technology to provide optimal learning plans based on each student's individual learning style and learning history. Below, we will explain the program processing of this system in natural language.

[0641] Collecting student information

[0642] 1. User (student) enters information

[0643] Users input information about their learning style and learning history into a device (such as a tablet or PC), such as "I'm good at math," "I'm good at repeated learning," or "I got 80 points on the last test."

[0644] 2. The device collects and transmits information

[0645] The device collects the information entered by the user and sends it to the server, which stores it in a database.

[0646] Suggestions for learning content and methods

[0647] 1. The server analyzes the information

[0648] The server extracts the student's learning style and past learning history from the database and analyzes this information.

[0649] 2. The server calls the generated AI

[0650] The server calls a generative AI (e.g., an AI model) and instructs it to generate an optimal learning plan for each student based on the analysis results. For example, it might instruct the server to "generate a set of math problems for students who are good at repetitive learning."

[0651] 3. Generative AI generates a learning plan

[0652] Generative AI creates customized study plans based on a student's learning style and history, such as a plan that includes a series of math problems or a set amount of reading each day.

[0653] 4. The server organizes and sends the study plan

[0654] The server organizes the learning plan received from the generation AI and sends it to the device.

[0655] Presentation of learning content

[0656] 1. The server sends the study plan to the device

[0657] The server sends the generated study plan to the terminal, which receives it and presents it to the user.

[0658] 2. The device displays the learning content

[0659] The device displays questions and learning materials to the user based on the received study plan, for example, displaying math problems or reading material on the screen.

[0660] 3. Users engage in learning

[0661] The user solves problems based on the learning content presented to them, either by typing the answers into the device or by speaking them through the audio glasses.

[0662] Check answers and explain

[0663] 1. User submits answer

[0664] Users can enter answers to the questions they have solved into the device, or they can enter answers by voice using audio glasses.

[0665] 2. The device sends the answer to the server

[0666] The terminal sends the answer entered by the user to the server, which accepts it and performs the necessary processing.

[0667] 3. The server checks the answer

[0668] The server calls the AI ​​generator and evaluates the user's answer against the correct answer. For example, it checks whether the answer to the question "What is 234 + 567?" is "801."

[0669] 4. The server generates the description

[0670] If the user's answer is incorrect, the server uses generative AI to generate text or audio that explains the error, for example, providing step-by-step instructions on how to arrive at the correct answer.

[0671] 5. The server sends the results and explanations to the device.

[0672] The server sends the results of the check and any necessary explanations to the terminal, which displays them to the user.

[0673] 6. The device displays the results and explanations

[0674] The device will show students whether their answers are correct or incorrect, and if they are incorrect, a detailed explanation will be displayed to help them further their learning.

[0675] Specific examples

[0676] For example, here is a fifth-grade student solving a math problem:

[0677] 1. Collecting student information

[0678] A user (e.g., Mr. Tanaka) inputs his / her learning style (he / she is good at repeated learning) and past test scores (80 points) into the terminal. The terminal sends this information to the server, which stores it in a database.

[0679] 2. Suggested learning content and methods

[0680] The server analyzes Tanaka's information, calls the generation AI, and instructs it to "generate a math problem set for students who are good at repeated learning." The generation AI generates the optimal problem set for Tanaka and returns it to the server. The server then sends it to the device.

[0681] 3. Presentation of learning content

[0682] The terminal displays the question "What is 234 + 567?" to Tanaka. Tanaka works on the question and enters the answer as "801."

[0683] 4. Check and explain your answers

[0684] The answer entered by Tanaka is sent from the device to the server, which checks the answer using the generation AI. If it is determined to be correct, the device displays "Correct!" If it is incorrect, the server uses the generation AI to generate an explanation for the mistake and sends it to the device. The device displays this to Tanaka and presents the next practice question.

[0685] This allows us to provide an optimal learning experience tailored to each student's individual characteristics.

[0686] The processing flow will be explained below.

[0687] Program processing flow

[0688] Collecting Student Information

[0689] Step 1:

[0690] The user (student) inputs information about their learning style and learning history into the terminal, such as "I'm good at repeating my learning" or "I got 80 points on the last math test."

[0691] Step 2:

[0692] The device collects the information entered by the user and sends it to the server, including the student ID, grade, learning style, and learning history.

[0693] Step 3:

[0694] The server stores the received information in a database, assigning an ID to each student's information to uniquely identify them.

[0695] Suggestions for learning content and methods

[0696] Step 1:

[0697] The server extracts the student's learning style and past learning history from the database, thereby understanding the student's current learning situation.

[0698] Step 2:

[0699] The server calls the AI ​​generator and instructs it to generate an optimal learning plan based on the extracted data (learning style, learning history), for example, by issuing a specific instruction such as "Generate math problems for a fifth-grade student who is good at repetitive learning."

[0700] Step 3:

[0701] Generative AI creates a personalized learning plan for each student, including multiple practice exercises and regular review.

[0702] Step 4:

[0703] The server receives the generated lesson plan, organizes it, and sends it to the device in the appropriate format.

[0704] Presentation of learning content

[0705] Step 1:

[0706] The server then sends the generated study plan to the device, which includes questions, study materials, and a study schedule.

[0707] Step 2:

[0708] The device then displays the received learning plan to the user, for example, displaying math exercises on the screen and providing audio and text explanations.

[0709] Step 3:

[0710] The user works on the presented problems and either types the answers into the terminal or submits the answers aloud via audio glasses.

[0711] Check answers and explain

[0712] Step 1:

[0713] After solving the problem, the user submits the answer by typing it into the terminal or by voice, and the answer is registered in the terminal.

[0714] Step 2:

[0715] The device sends the user's response to the server, which may include image information and audio data.

[0716] Step 3:

[0717] The server calls the generation AI and checks the answer submitted by the user against the correct answer, for example, checking whether '801' is the correct answer to 234 + 567.

[0718] Step 4:

[0719] The server determines whether the answer is correct, and if it is incorrect, the generative AI creates an explanation including the reason and the correct solution.

[0720] Step 5:

[0721] The server checks the answers and sends them to the device along with explanations, which may include reasons for the correct answers and additional practice questions.

[0722] Step 6:

[0723] The device will display the result and explanation to the student. Specifically, it will say "That's correct!" or "That's wrong. The correct answer is '801'" on the screen, along with a detailed explanation of why the answer was wrong.

[0724] Specific examples

[0725] For example, the processing flow when Tanaka, a fifth-grader, solves a math problem is shown below.

[0726] Step 1:

[0727] Tanaka types into the terminal, "I'm good at repeated learning and got 80 points on the last test."

[0728] Step 2:

[0729] The device collects Tanaka's information and sends it to the server.

[0730] Step 3:

[0731] The server stores the information in a database and calls a generative AI to instruct it to generate a "math problem set for students who excel at repetitive learning."

[0732] Step 4:

[0733] The generation AI generates a plan and sends it back to the server.

[0734] Step 5:

[0735] The server organizes the study plan and sends it to the device.

[0736] Step 6:

[0737] The terminal displays the question "What is 234 + 567?" to Tanaka, who then enters the answer "801."

[0738] Step 7:

[0739] The device sends the entered answers to the server, which checks the answers using a generation AI.

[0740] Step 8:

[0741] The server determines that the answer is correct and sends the result to the terminal.

[0742] Step 9:

[0743] The device displays to Tanaka, "That's correct! Well done!"

[0744] As a result, this learning system provides optimal learning plans and feedback tailored to each student's learning needs.

[0745] Example 1

[0746] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0747] Conventional learning systems face the challenge of providing optimal learning plans tailored to each student's individual characteristics. In particular, there are few systems that can provide individually customized learning plans in real time that properly reflect a student's learning style and past learning history. Furthermore, the process of generating learning plans using generative AI models requires both convenience and accuracy, making it a highly advanced technology. Therefore, there is a need for the development of a system that can efficiently provide learning support tailored to each student.

[0748] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0749] In this invention, the server includes means for collecting and storing information on students' learning styles and learning histories, means for generating an optimal learning plan based on the collected learning styles and learning histories, and means for presenting learning content to students based on the generated learning plan. This makes it possible to provide customized learning plans suited to the learning characteristics of individual students in real time, thereby improving learning efficiency.

[0750] "Learning style" refers to the characteristics that indicate how a student learns best, such as retrieval, visual learning, or reading-based learning.

[0751] "Learning history" refers to information that records a student's past learning activities and achievements, such as past test scores, study time, and what they learned.

[0752] "Means of collection" refers to devices and software that acquire information from users and organize and store it. Examples include tablets, PCs, and dedicated input systems.

[0753] "Storage means" refers to methods or systems for recording and storing collected data in an appropriate format, such as database management systems or cloud storage.

[0754] "Generative means" refers to methods or systems that generate new information or plans based on given input data, specifically generative AI models and algorithms.

[0755] "Presenting means" refers to a method or system for visually or audibly presenting the generated information or plan to the user, including, for example, display on a display or audio announcement.

[0756] "Means of checking" refers to techniques or methods for evaluating submitted data or responses to determine their accuracy. Examples include automated software evaluation and analysis using artificial intelligence.

[0757] "Means for providing explanations" refers to methods or systems for explaining the evaluation results or errors in a way that is easy for users to understand. For example, this could be a text-based explanation or a voice assistant explanation.

[0758] "Device" refers to the input / output device used by a user, including tablets, PCs, smartphones, etc.

[0759] "Server" refers to a central computer that stores and processes data via a network and manages and operates the entire system in cooperation with terminals.

[0760] "Database" refers to a digital information management system that systematically stores collected information and enables it to be quickly accessed and retrieved when needed.

[0761] A "generative AI model" refers to an artificial intelligence mechanism that automatically generates various outputs (such as learning plans) based on input data.

[0762] A "prompt sentence" refers to a text sentence that is input to give specific instructions to a generative AI model.

[0763] "REST API" refers to a set of definitions and protocols that allow different software applications to communicate with each other and exchange data.

[0764] "JSON" stands for JavaScript Object Notation and refers to a lightweight data interchange format for structuring, storing, and transferring data.

[0765] An "HTTP POST request" is an HTTP method for sending data to a web server, which can save or update data.

[0766] This invention is a learning system that utilizes generative AI models and image analysis technology to provide optimal learning plans based on each student's individual learning style and learning history. Specific embodiments of this system are described in detail below.

[0767] First, the user (student) uses a device (tablet or PC) to input information about their learning style and past learning history. For example, they provide information such as "I'm good at math," "I like repeated learning," and "I got 80 points on the last test." The device then organizes this information and sends it to the server.

[0768] The server uses a database management system (e.g., MySQL or PostgreSQL) to store the received information in a database. The server then extracts students' learning styles and past learning histories from the database and analyzes these data using data analysis tools (e.g., the Pandas module in Python).

[0769] Based on the analysis results, the server calls a generative AI model (e.g., GPT-3). It gives instructions to the AI ​​using prompts. Examples of specific prompts include:

[0770] "Generating appropriate math problem sets for users who are good at iterative learning"

[0771] etc.

[0772] The generative AI model generates an optimal study plan based on the provided prompts, which includes math problem sets, reading materials, etc. The generated study plan is sent back to the server in JSON format, which the server parses and sends to the device.

[0773] The device presents the user with learning content based on the received learning plan. Specifically, it uses HTML, CSS, and JavaScript to display various questions and learning materials on the screen. The user's answers to the questions are entered into the device or collected via a voice input device (e.g., audio glasses).

[0774] When a user submits an answer, the device sends it to the server. The server uses a generative AI model to check the user's answer and determine whether it is correct or incorrect. If the answer is incorrect, the server uses the generative AI model to generate an explanation for the mistake. This explanation is also sent to the device in JSON format, for example, and the device displays it to the user.

[0775] As a concrete example, consider a fifth-grade student solving a math problem. The user (student) inputs their learning style (good at repetitive learning) and past test scores (80 points) into the device. The device sends this information to the server, which stores it in a database. The server then sends a prompt to the generative AI model.

[0776] "Generating appropriate math problem sets for users who are good at iterative learning"

[0777] The generative AI model generates an optimal problem set and sends it back to the server. The server sends this to the device. The device then displays a question to the user, such as "What is 234 + 567?", to which the user answers, for example, "801." The user's answer is again sent to the server and checked by the generative AI model. If the answer is correct, a message indicating the correct answer is displayed, and if it is incorrect, an explanation is displayed.

[0778] This makes it possible to provide an optimal learning experience tailored to each student's individual characteristics.

[0779] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0780] Step 1: User Enters Information

[0781] Users use a device (tablet or PC) to input information such as their learning style and past grades. For example, they might say, "I'm good at math," "I'm good at repeated learning," or "I got 80 points on the last test." This information is collected as input data.

[0782] Step 2: Device collects and sends information

[0783] The terminal internally organizes the data entered by the user and converts it into JSON format, then uses a REST API to send this information to the server, which uses an HTTP POST request to transfer the data to the server. The server receives this data and stores it in a database.

[0784] Step 3: The server analyzes the information

[0785] The server extracts the user's learning style and learning history from the database. It then analyzes the data using Python's Pandas module, for example. For example, it calculates and visualizes the distribution of past test scores and learning style trends. The input here is the data extracted from the database, and the output is the analysis results.

[0786] Step 4: The server invokes the generative AI model

[0787] The server sends a prompt to the generative AI model based on the analysis results. A specific prompt might be, "Generate the optimal set of math problems for a user who is good at repeated learning." The input is the analysis results and the prompt, and the output is the generated learning plan.

[0788] Step 5: Generative AI model generates a learning plan

[0789] The generative AI model generates an optimal lesson plan based on the prompt it receives, including a set of math problems and reading material, and outputs a JSON-formatted lesson plan that is sent back to the server.

[0790] Step 6: The server organizes and sends the lesson plan

[0791] The server parses the JSON-formatted learning plan returned by the generative AI model, formats it as needed, and then sends the data to the device using a REST API. The output is a curated learning plan.

[0792] Step 7: Your device will display what it learned

[0793] The device uses HTML, CSS, and JavaScript to display the learning content based on the received lesson plan. For example, it displays math problems or reading material on the screen. In this case, the input is the lesson plan, and the output is the learning content on the user interface.

[0794] Step 8: Users engage in learning

[0795] The user progresses with their learning based on the displayed learning content, for example, by answering math questions. The input is the learning content, and the output is the user's answer.

[0796] Step 9: User submits answer

[0797] The user enters the answer to the problem into the terminal, for example, by typing "801" on a keyboard or by using voice input to answer "234 plus 567 is 801." The input is the user's answer, and the output is the data entered into the terminal.

[0798] Step 10: The device sends the answer to the server

[0799] The terminal organizes the user's answers and sends them to the server using an HTTP POST request. The input is the user's answer data, and the output is the data transferred to the server.

[0800] Step 11: Server checks the answer

[0801] The server calls the generative AI model and checks whether the user's answer is correct. For example, it evaluates whether "234 + 567 is 801." The input is the user's answer and the model's correct answer data, and the output is the correctness of the answer.

[0802] Step 12: Server generates commentary

[0803] If the answer is incorrect, the server uses a generative AI model to generate a detailed explanation for the mistake, for example, creating a step-by-step explanation for arriving at the correct answer. The input is the user's answer data and the prompt text, and the output is the explanation text.

[0804] Step 13: The server sends the results and explanations to the device.

[0805] The server sends the answer's correctness and explanation to the terminal. The output is the correctness and explanation data.

[0806] Step 14: Your device will display the results and explanations.

[0807] The device displays the received results and explanations to the user, for example, a message such as "Your answer is correct!" or "You made a mistake here. Please refer to the next step." The input is the result and explanation data, and the output is the display on the user interface.

[0808] (Application example 1)

[0809] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0810] Existing learning systems have limitations in providing optimal learning plans that are tailored to each student's individual learning style and learning history. They also lack the ability to provide real-time learning support and feedback, resulting in insufficient support for students to learn efficiently. Furthermore, the methods of presenting learning content and providing feedback are inconsistent, resulting in reduced learning effectiveness. Therefore, there is a need for an effective learning system that meets the individual needs of each student.

[0811] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0812] In this invention, the server includes means for collecting and storing information about the student's learning style and learning history, means for generating an optimal learning plan based on the collected learning style and learning history, means for presenting learning content to the student based on the generated learning plan, means for displaying the learning content and feedback using a glasses-type device worn by the student, and means for checking answers submitted by the student and providing explanations as necessary. This allows for the provision of a learning plan optimal for each student's individual learning style in real time, enabling efficient learning support.

[0813] "Student" refers to a learner who uses the learning system.

[0814] "Learning style" refers to the individual characteristics that indicate how students learn most effectively.

[0815] "Learning history" refers to a record of what a student has learned in the past and their grades.

[0816] "Collection and storage means" refers to methods and devices for capturing and storing information about students' learning styles and learning histories.

[0817] A "learning plan" refers to a plan that organizes learning content and methods in a way that is optimal for each student.

[0818] "Generating means" refers to a method or system that creates a learning plan based on the collected information.

[0819] "Presenting means" refers to a method or device for displaying the generated lesson plan to students.

[0820] "Eyeglasses-type device" refers to a wearable device that students wear and use to provide visual information.

[0821] "Feedback" refers to the evaluation and explanation given to students in response to their learning results and answers.

[0822] "Means of checking" refers to the method or device that evaluates the answers submitted by students and determines whether they are correct or incorrect.

[0823] "Means for providing explanations as needed" refers to a method or system that provides guidance or explanations to the correct answer when the answer is incorrect.

[0824] "Generative AI" refers to a system that uses artificial intelligence to generate new data and plans.

[0825] A "prompt sentence" refers to an input sentence that gives specific generation instructions to the generation AI.

[0826] The following system components and process steps are used to implement the invention.

[0827] System Components

[0828] 1. Eyeglass-type device worn by students

[0829] Hardware: Eyeglasses (e.g. Google Glass)

[0830] Software: Application for viewing learning content and receiving feedback

[0831] 2. Server

[0832] Hardware: High-performance server

[0833] Software: Database management systems (e.g., MySQL), generative AI (e.g., OpenAI GPT models), data analysis tools (e.g., Python-based analysis systems)

[0834] Process Overview

[0835] The server generates an optimal learning plan based on the student's learning style and learning history, and displays the learning content and feedback in real time on the glasses-type device worn by the student. Every time the student takes an action, that information is sent to the server, which analyzes the data and generates and provides the next appropriate learning content.

[0836] Detailed processing contents

[0837] 1. Information gathering

[0838] User: Students provide their learning style and learning history by speaking into the glasses, scanning a QR code, or manually entering information.

[0839] Device: The glasses send the collected information to a smartphone via Bluetooth, which then transmits it to a server via Wi-Fi.

[0840] 2. Data storage and analysis

[0841] Server: The server stores the received information in a database (MySQL) and uses data analysis tools to analyze learning styles and history.

[0842] 3. Create a learning plan

[0843] Server: Based on the analysis results, the server sends prompts to a generative AI model (e.g., OpenAI's GPT model) to generate an optimal learning plan. Example prompts:

[0844] Generate math problem sets suitable for middle school students who enjoy repetition.

[0845] 4. Presentation of learning content

[0846] Server: Sends the generated learning plan to the eyeglasses device.

[0847] Device: The glasses-type device displays the learning content according to the received learning plan.

[0848] 5. Providing Feedback

[0849] User: The student engages with the learning content and provides answers via voice or manual input into the glasses.

[0850] Device: The answer is sent to the server, where the generation AI determines whether it is correct or incorrect and generates feedback.

[0851] 6. Representation

[0852] Server: Sends feedback to the glasses-type device and prepares the next appropriate learning content.

[0853] Device: The glasses display feedback and new learnings.

[0854] Specific examples

[0855] For example, here is a scenario where a middle school student is taking extra math classes:

[0856] 1. Information gathering: Students speak into the glasses-like device, saying, "I want to solve a math problem."

[0857] 2. Data storage and analysis: The server receives the voice data and stores it in a database. The server analyzes the student's learning history and determines whether the student is a "junior high school student who excels at repeated learning."

[0858] 3. Learning plan generation: The server sends the following prompt to the generative AI model:

[0859] Generate math problem sets suitable for middle school students who enjoy repetition.

[0860] The generated problem set (e.g., "What is 234 + 567?") is sent to the eyeglass-type device.

[0861] 4. Presentation of learning content: The eyeglasses display "What is 234 + 567?"

[0862] 5. Providing feedback: The student types "801" by voice. The server uses generative AI to determine whether the answer is correct or not, and provides feedback saying, "That's correct!"

[0863] 6. Re-presentation: The server generates the next question and sends it to the glasses-type device. The learning content is displayed again.

[0864] This allows us to provide a learning experience that best suits each student's individual learning style.

[0865] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0866] Step 1:

[0867] Information collection and transmission

[0868] User: Students provide their learning style and learning history by speaking into the glasses, scanning a QR code, or manually entering their learning history.

[0869] Input: Learning style characteristics and learning history (e.g., you are good at repeating things, you got 80 points on the last test, etc.).

[0870] Output: Audio data and manual input data.

[0871] How it works: The glasses send the collected information to a smartphone via Bluetooth, and then to a server via Wi-Fi.

[0872] Step 2:

[0873] Data storage and analysis

[0874] Server: The server stores the received information in a database (e.g., MySQL) and uses a data analysis tool (e.g., a Python-based analysis system) to analyze learning styles and histories.

[0875] Input: Voice data and manual input data.

[0876] Output: Analysis results based on learning style and history.

[0877] Specific operation: After storing the information in the database, the server analyzes the learning style and history and generates prompts for the generative AI model.

[0878] Step 3:

[0879] Generate a learning plan

[0880] Server: Based on the analysis results, it sends prompts to a generative AI model (e.g., OpenAI's GPT model) to generate an optimal learning plan.

[0881] Input: Analysis results based on learning style and history, prompt statement.

[0882] Output: The generated learning plan.

[0883] Specific operation: The server sends the following prompt to the generative AI model: "Please generate a set of math problems suitable for junior high school students who are good at repetitive learning." The generative AI then generates a study plan and sends it back to the server.

[0884] Step 4:

[0885] Presentation of learning content

[0886] Server and device: The server sends the generated learning plan to the eyeglass device, which then displays the learning content to the student.

[0887] Input: The generated learning plan.

[0888] Output: Learning content displayed on the eyeglasses device.

[0889] Specific operation: The glasses-type device presents students with questions such as "What is 234 + 567?"

[0890] Step 5:

[0891] Enter and submit your answers

[0892] User: The student engages with the learning content and provides answers via voice or manual input into the glasses.

[0893] Input: Student's answer (e.g., "801").

[0894] Output: Audio data or manually input data.

[0895] How it works: The glasses send students' responses via Bluetooth to a smartphone, which then transmits them to a server via Wi-Fi.

[0896] Step 6:

[0897] Checking answers and generating feedback

[0898] Server: Checks the received answers using the generative AI model and generates feedback (explanation) if necessary.

[0899] Input: Student answers.

[0900] Output: Correct / incorrect judgment and feedback (explanation).

[0901] What it does: The server matches the student's answer with the correct answer and uses a generative AI model to generate feedback, such as "That's right!" or an explanation if the answer is incorrect.

[0902] Step 7:

[0903] Providing feedback and next steps

[0904] Server and device: The server sends the generated feedback to the eyeglasses device, which prepares the next appropriate learning content. The eyeglasses device displays the feedback and new learning content to the student.

[0905] Input: Feedback and next learning plan.

[0906] Output: Feedback and next steps displayed on the glasses.

[0907] What it does: The glasses provide feedback and next steps to students, providing continuous support for their learning.

[0908] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0909] This invention is a learning system that recognizes a student's learning style, learning history, and emotions and provides an optimal learning plan. The program processing of this system will be specifically explained below.

[0910] Collecting student information

[0911] 1. User (student) enters information

[0912] The user inputs information about their learning style and learning history into the terminal, such as "I'm good at repeating my learning" or "I got 80 points on the last test."

[0913] 2. The device collects and transmits information

[0914] The device collects the information entered by the user and sends it to the server, including the student ID, grade, learning style, and learning history.

[0915] 3. The server stores the information

[0916] The server stores the received information in a database, assigning an ID to each student's information to uniquely identify them.

[0917] Emotion recognition

[0918] 1. Real-time video and audio input by users

[0919] While studying, users input video and audio into the device in real time using a webcam and microphone.

[0920] 2. The device performs emotion analysis

[0921] The device sends the collected video and audio data to an emotion engine that recognizes the student's emotions, such as "concentrating," "tired," or "enjoying" in real time.

[0922] 3. Sending Emotional Data

[0923] The analyzed emotional data is sent from the device to a server and used when generating a learning plan.

[0924] Suggestions for learning content and methods

[0925] 1. The server analyzes the information

[0926] The server extracts information from the database about students' learning styles and past learning histories, as well as emotional data collected in real time, and analyzes this information.

[0927] 2. The server calls the generated AI

[0928] The server calls the AI ​​generator and instructs it to generate an optimal learning plan based on the analysis results, for example, by issuing specific instructions such as "generate math problems for students who are good at repetitive learning and who are in a focused state."

[0929] 3. Generative AI generates a learning plan

[0930] The AI ​​generates optimal learning plans based on a student's learning style and emotions. For example, if a student is concentrating, it will increase the difficulty of the questions slightly, and if it determines that the student is tired, it will generate a plan that includes instructions to take a break.

[0931] 4. The server organizes and sends the study plan

[0932] The server organizes the learning plan received from the generation AI and sends it to the device in the appropriate format.

[0933] Presentation of learning content

[0934] 1. The server sends the study plan to the device

[0935] The server then sends the generated study plan to the device, which includes questions, study materials, and a study schedule.

[0936] 2. Your device will display your study plan

[0937] The device then displays the received study plan to the user, for example, displaying math practice problems on the screen and providing dynamic instructions such as "concentrate" and "take breaks as you progress."

[0938] 3. Users engage in learning

[0939] The user studies based on the presented study plan, either by typing answers into the device or by submitting answers aloud via audio glasses.

[0940] Check answers and explain

[0941] 1. User submits answer

[0942] The user submits the answer to the problem by typing it into the terminal or by voice, and the answer is registered in the terminal.

[0943] 2. The device sends the answer to the server

[0944] The device sends the user's response to the server, along with their emotional data if necessary.

[0945] 3. The server checks the answer

[0946] The server calls the generation AI and checks the answer submitted by the user against the correct answer, for example, checking whether '801' is the correct answer to 234 + 567.

[0947] 4. The server generates the description

[0948] If the user's answer is incorrect, the server uses a generative AI to create an explanation including the reason and the correct solution. It also provides feedback according to the user's emotional state.

[0949] 5. The server sends the results and explanations to the device.

[0950] The server checks the answers and sends them to the device along with explanations, including reasons for the correct answers, additional practice questions, and even encouragement and advice based on the user's emotional state.

[0951] 6. The device displays the results and explanations

[0952] The device displays the results and explanations to the student. Specifically, it displays messages such as "That's right! Well done!" or "That's wrong. The correct answer is '801'" on the screen, and also provides feedback that takes emotions into account.

[0953] Specific examples

[0954] For example, the processing flow when Tanaka, a fifth-grader, solves a math problem is shown below.

[0955] Step 1:

[0956] Tanaka types into the terminal, "I'm good at repeated learning and got 80 points on the last test."

[0957] Step 2:

[0958] Video and audio data collected in real time was analyzed to determine that Tanaka was "concentrating."

[0959] Step 3:

[0960] The device collects Tanaka's information and emotional data and sends it to the server.

[0961] Step 4:

[0962] The server stores the information in a database and calls the generation AI, instructing it to generate a "math problem set suitable for Tanaka, a student who excels at repeated learning, and who is currently concentrating."

[0963] Step 5:

[0964] The generation AI generates a plan and sends it back to the server.

[0965] Step 6:

[0966] The server organizes the learning plan and sends it to the device, displaying a question such as "What is 234 + 567?"

[0967] Step 7:

[0968] Tanaka responds by entering "801." The device then sends the response and emotion data to the server.

[0969] Step 8:

[0970] The server checks the answer using the generation AI and determines it to be correct.

[0971] Step 9:

[0972] The server sends the result to the device, which then displays to Tanaka, "That's correct! Maintain your current concentration."

[0973] This allows the learning system to provide optimal learning plans and feedback tailored to each student's learning needs, promoting efficient and effective learning.

[0974] The processing flow will be explained below.

[0975] Program processing flow

[0976] Collecting student information

[0977] Step 1:

[0978] The user (student) inputs information about their learning style and learning history into the terminal. Specifically, they input information such as "I'm good at repeating my learning" or "I got 80 points on the last test."

[0979] Step 2:

[0980] The device collects the information entered by the user and sends it to the server, including the student ID, grade, learning style, and learning history.

[0981] Step 3:

[0982] The server stores the received information in a database, assigning an ID to each student's information to uniquely identify them.

[0983] Emotion recognition

[0984] Step 1:

[0985] While learning, users input video and audio into the device in real time, and the device uses a webcam and microphone to collect the user's facial expressions and tone of voice.

[0986] Step 2:

[0987] The device sends the collected video and audio data to the emotion engine, which analyzes this data and recognizes the user's emotions, such as "concentrated," "tired," or "enjoyed."

[0988] Step 3:

[0989] The emotion data analyzed by the emotion engine is sent to the server along with the user's learning style and learning history.

[0990] Suggestions for learning content and methods

[0991] Step 1:

[0992] The server extracts the student's learning style and past learning history from the database, as well as emotional data collected in real time, and analyzes this information.

[0993] Step 2:

[0994] The server calls the AI ​​and instructs it to generate an optimal learning plan based on the extracted data (learning style, learning history, emotional data).For example, it gives specific instructions such as, "Generate math problems for students who are good at repetitive learning and are in a focused state."

[0995] Step 3:

[0996] The generative AI generates optimal study plans based on a student's learning style and emotions. For example, if a student is concentrating, it might increase the difficulty of the questions slightly, or if it detects fatigue, it might provide a plan that includes instructions to take a break.

[0997] Step 4:

[0998] The server receives the generated lesson plan, organizes it, and sends it to the device in the appropriate format.

[0999] Presentation of learning content

[1000] Step 1:

[1001] The server then sends the generated study plan to the device, which includes questions, study materials, and a study schedule.

[1002] Step 2:

[1003] The device then displays the received study plan to the user, for example, displaying math exercises on the screen and providing dynamic instructions such as "stay focused" or "you look tired, take a break."

[1004] Step 3:

[1005] The user studies based on the presented study plan, either by typing answers into the device or by submitting answers aloud via audio glasses.

[1006] Check answers and explain

[1007] Step 1:

[1008] After solving the problem, the user submits the answer by typing it into the terminal or by voice, and the answer is registered in the terminal.

[1009] Step 2:

[1010] The device sends the user's response to the server, along with their emotional data if necessary.

[1011] Step 3:

[1012] The server calls the generation AI and checks the answer submitted by the user against the correct answer, for example, checking whether '801' is the correct answer to 234 + 567.

[1013] Step 4:

[1014] The server determines whether the answer is correct, and if it is incorrect, the generative AI creates an explanation including the reason and the correct solution. It also provides feedback based on the user's emotional state.

[1015] Step 5:

[1016] The server checks the answers and sends them to the device along with explanations, including reasons for the correct answers, additional practice questions, and even encouragement and advice based on the user's emotional state.

[1017] Step 6:

[1018] The device displays the results and explanations to the student, such as "That's right! Well done!" or "That's wrong. The correct answer is '801'," and also provides emotional feedback.

[1019] Specific examples

[1020] For example, the processing flow when Tanaka, a fifth-grader, solves a math problem is shown below.

[1021] Step 1:

[1022] Tanaka types into the terminal, "I'm good at repeated learning and got 80 points on the last test."

[1023] Step 2:

[1024] Video and audio data collected in real time was analyzed to determine that Tanaka was "concentrating."

[1025] Step 3:

[1026] The device collects Tanaka's information and emotional data and sends it to the server.

[1027] Step 4:

[1028] The server stores the information in a database and calls the generation AI, instructing it to generate a "math problem set suitable for Tanaka, a student who excels at repeated learning, and who is currently concentrating."

[1029] Step 5:

[1030] The generation AI generates a plan and sends it back to the server.

[1031] Step 6:

[1032] The server organizes the study plan and sends it to the device, which then displays the question "What is 234 + 567?" to Tanaka.

[1033] Step 7:

[1034] Tanaka responds by entering "801." The device then sends the response and emotion data to the server.

[1035] Step 8:

[1036] The server checks the answer using the generation AI, and if it is determined to be correct, it sends the result to the device.

[1037] Step 9:

[1038] The device displays to Tanaka, "That's correct! Maintain your current concentration."

[1039] This allows the learning system to provide optimal learning plans and feedback tailored to each student's learning needs, promoting efficient and effective learning.

[1040] Example 2

[1041] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1042] Modern learning systems are expected to provide individualized learning plans based on students' learning styles and learning histories. However, traditional systems only use static data and are unable to adjust learning plans to take students' real-time emotions into account. This makes it difficult to provide dynamic learning plans that optimally maintain students' focus and motivation.

[1043] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1044] In this invention, the server includes means for collecting and storing information on students' learning styles and learning histories, means for generating optimal learning plans based on the collected learning styles and learning histories, means for collecting real-time video and audio data, performing emotion analysis, and storing the data, and means for adjusting and generating learning plans based on the analyzed emotion data, thereby making it possible to provide learning plans that take into account the dynamic emotional states of students.

[1045] "Student learning style" refers to the individual characteristics of how and how students prefer to learn.

[1046] "Learning history" refers to data recorded about what learning activities a student has engaged in in the past, including the results and achievements.

[1047] "Means for collecting and storing information" refers to devices and systems for inputting, recording, and storing information about students' learning styles and learning histories.

[1048] "Means for generating optimal learning plans" refers to devices or systems that create learning content and schedules that are most suitable for individual students based on collected information.

[1049] "Means for presenting learning content to students" refers to devices or systems that display and provide specific learning tasks and learning materials to students based on the generated learning plan.

[1050] "Means for checking answers and providing explanations" refers to a device or system that verifies the answers submitted by students, determines whether they are correct, and provides explanations or feedback as needed.

[1051] "Real-time video and audio data" refers to video and audio information that records students' situations and reactions in real time while they are learning.

[1052] "Emotion analysis" refers to the process of analyzing and recognizing students' emotional states (e.g., focused, tired, or enjoying) from video and audio data collected in real time.

[1053] "Generative AI" refers to artificial intelligence models that automatically perform specific tasks (e.g., generating lesson plans, checking answers) based on collected data.

[1054] "Study plan adjustment" refers to the process of dynamically modifying and optimizing existing study plans based on data obtained through sentiment analysis.

[1055] This invention is a learning system that recognizes a student's learning style, learning history, and emotions to provide an optimal learning plan. Next, the program processing of this system will be specifically explained.

[1056] Hardware and software used

[1057] This system uses the following hardware and software:

[1058] Server: Database management system (e.g., MySQL), generative AI (e.g., OpenAI GPT-4), data analysis library (e.g., pandas, NumPy)

[1059] Device: Webcam, microphone, emotion analysis engine (e.g., OpenCV, TensorFlow), user interface

[1060] Communication protocol: HTTPS

[1061] Processing flow

[1062] Collecting student information

[1063] 1. The user enters information

[1064] The user inputs information about their learning style and learning history into the terminal, such as "I'm good at repeating my learning" or "I got 80 points on the last test."

[1065] 2. The device collects information and sends it to the server

[1066] The terminal collects the information entered by the user (student ID, grade, learning style, learning history) and sends it to the server via an HTTPS request.

[1067] 3. The server stores the information in a database

[1068] The server stores the received information in a database, using an SQL query to insert the input data using the student ID as the primary key.

[1069] Emotion recognition

[1070] 1. Users input real-time video and audio

[1071] While studying, users input video and audio into the device using a webcam and microphone.

[1072] 2. The device performs emotion analysis

[1073] The device sends the collected video and audio data to an emotion analysis engine, which analyzes the user's emotional state (concentrated, tired, enjoying, etc.) in real time.

[1074] 3. Sending Emotional Data

[1075] The device sends the analyzed emotion data to the server in JSON format.

[1076] Suggestions for learning content and methods

[1077] 1. The server analyzes the information

[1078] The server extracts students' learning styles, past learning history, and real-time emotional data from the database and analyzes the data using Python scripts.

[1079] 2. The server calls the generated AI

[1080] The server calls the generative AI and sends prompts that generate the optimal learning plan based on the analysis results.

[1081] Example: "Generate math problems for students who are good at repetition and are in a focused state."

[1082] 3. Generative AI generates learning plans

[1083] Based on the prompts, the generative AI generates an optimal study plan that includes the difficulty of the questions, the content to be studied, and a schedule.

[1084] 4. The server organizes and sends the generated lesson plans

[1085] The server organizes the learning plan received from the generation AI and sends it to the terminal in JSON format.

[1086] Presentation of learning content

[1087] 1. The server sends the study plan to the device

[1088] The server sends the organized lesson plan to the device, which is sent as an HTTP response.

[1089] 2. Your device will display your study plan

[1090] The device displays the received lesson plan on the screen to the user. For example, the device displays a math problem such as "What is 234 + 567?"

[1091] 3. Users engage in learning

[1092] The user solves problems based on the presented learning plan, and submits answers by typing them into the device or by voice.

[1093] Check answers and explain

[1094] 1. User submits answer

[1095] Users submit their answers to the questions they have solved by typing them into the device or by voice, and the answer data is saved on the device.

[1096] 2. The device sends the answer to the server

[1097] The device sends the user's response data to the server, including emotion data if necessary.

[1098] 3. The server checks the answer

[1099] The server calls the AI ​​generator and checks the user's answer against the correct answer. It sends a prompt to the AI ​​generator asking if '801' is the correct answer to 234 + 567.

[1100] 4. The server generates the description

[1101] If the user answers incorrectly, the server uses the generative AI again to generate an explanation, including the reason for the incorrect answer and the correct solution. It also provides feedback based on the user's emotional state.

[1102] 5. The server sends the results and explanations to the device.

[1103] The server then sends the results of the check and explanation to the device, including reasons for the correct answer, additional practice questions, and encouragement or advice based on the user's emotional state.

[1104] 6. The device will display the results and explanations.

[1105] The device displays the results and explanations on the screen, such as "That's right! Well done!" or "That's wrong. The correct answer is '801'," and also provides emotional feedback.

[1106] This allows the learning system to provide optimal learning plans and feedback tailored to each student's individual needs, promoting efficient and effective learning.

[1107] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1108] Step 1:

[1109] The user enters information

[1110] The user inputs information about their learning style and learning history into the device. For example, they might input information such as "I'm good at repeating my learning" or "I got 80 points on the last test." The input data includes student ID, grade, learning style, learning history, etc. The device temporarily stores this input data in its memory.

[1111] Step 2:

[1112] The device collects information and sends it to the server

[1113] The device collects information entered by the user and sends it to the server using an HTTPS request (input: user information, output: data sent to the server). The collected data is converted into JSON format and sent to the server using the HTTPS protocol.

[1114] Step 3:

[1115] The server stores the information in a database

[1116] The server saves the received information in a database. Specifically, it inserts the information into the database as a new record using an SQL query (input: user information, output: save to database). The saved data includes a unique student ID.

[1117] Step 4:

[1118] Users input real-time video and audio

[1119] While studying, users input real-time video and audio data into the device using a webcam and microphone. For example, while the user is solving a math problem, the device collects video of the user's facial expressions and records audio of their reactions (input: real-time video and audio data, output: temporary storage on the device).

[1120] Step 5:

[1121] The device performs emotion analysis

[1122] The device sends the collected video and audio data to an emotion analysis engine, which analyzes the user's emotions in real time. Specifically, OpenCV and TensorFlow are used to recognize emotional states such as "concentrated," "tired," and "enjoyed" (input: video and audio data, output: emotional state data).

[1123] Step 6:

[1124] The device sends emotion data to the server.

[1125] The device sends the analyzed emotion data to the server in JSON format, which includes the emotion data acquired in real time and the student ID (input: emotion state data, output: data transmission to the server).

[1126] Step 7:

[1127] The server analyzes the information

[1128] The server extracts and analyzes students' learning styles, learning histories, and real-time emotional data from the database. It then analyzes the data using Python scripts and data analysis libraries (e.g., pandas, NumPy) to determine the optimal learning plan for each student (input: student information, learning history, emotional data; output: analysis results).

[1129] Step 8:

[1130] The server calls the generated AI

[1131] The server calls a generative AI (e.g., OpenAI GPT-4) and sends a prompt that generates an optimal learning plan based on the analysis results (input: analysis results, output: generation of prompt).

[1132] Example: "Generate math problems for students who are good at repetition and are in a focused state."

[1133] Step 9:

[1134] Generative AI generates learning plans

[1135] The generation AI generates an optimal learning plan based on the prompt received from the server. The plan includes the difficulty level of the questions, learning content, and schedule (input: prompt, output: learning plan).

[1136] Step 10:

[1137] The server organizes and sends the generated lesson plans

[1138] The server organizes the learning plan received from the generation AI and sends it to the device. It converts it into the appropriate format in JSON format and sends it as an HTTPS response (input: learning plan, output: transmission to device).

[1139] Step 11:

[1140] Your device will display your study plan

[1141] The device displays the learning plan received from the server to the user. For example, a math problem such as "What is 234 + 567?" is displayed on the screen (input: learning plan, output: display to user).

[1142] Step 12:

[1143] Users engage in learning

[1144] The user solves the exercises based on the presented learning plan, and either enters the answers into the device or submits them by voice (input: user's answers, output: input to the device).

[1145] Step 13:

[1146] User submits answer

[1147] The user inputs the answer to the problem they solved into the terminal, which temporarily stores this data (input: user's answer, output: saved in the terminal).

[1148] Step 14:

[1149] The device sends the answer to the server

[1150] The device sends the user's response in JSON format to the server, including real-time emotional data if necessary (input: user's response, output: transmission to the server).

[1151] Step 15:

[1152] The server checks the answer

[1153] The server calls the generation AI and checks the answer submitted by the user by comparing it with the correct answer. For example, it checks whether "801" is the correct answer to 234 + 567" (input: user answer, output: correct or incorrect).

[1154] Step 16:

[1155] The server generates the description

[1156] When a user gives an incorrect answer, the server uses a generative AI to generate an explanation including the reason and the correct solution. It also provides feedback according to the user's emotional state (input: judgment result of correctness, output: explanation and feedback).

[1157] Step 17:

[1158] The server sends the results and explanations to the device.

[1159] The server checks the answers and sends them to the device together with an explanation. The results include reasons for the correct answer, additional practice questions, and encouragement and advice based on the user's emotional state (input: explanation and feedback, output: transmission to device).

[1160] Step 18:

[1161] The device displays the results and explanations

[1162] The device displays the result and explanation on the screen. Specifically, it displays "That's right! Well done!" or "That's wrong. The correct answer is '801'," and also provides feedback that takes emotions into account (input: result and explanation, output: displayed to the user).

[1163] (Application example 2)

[1164] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1165] Conventional learning and work support systems have had difficulty providing optimal plans that accurately and quickly reflect an individual's learning and work style, as well as their emotional state, which changes in real time. As a result, it has been difficult to achieve effective learning and efficient work support, leading to problems such as reduced learning effectiveness and work efficiency.

[1166] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for collecting and saving information on students' learning styles and learning histories, means for generating an optimal learning plan based on the collected learning styles and learning histories, means for presenting learning content to students based on the generated learning plan, means for checking answers submitted by students and providing explanations as necessary, means for collecting and saving information on workers' working styles and working histories, means for generating an optimal work plan based on the collected working styles and working histories, means for recognizing emotional states in real time and adjusting the work plan based thereon, means for recognizing the worker's fatigue level and concentration level using an emotion analysis engine, means for transmitting the collected emotional data to the server, means for calling a generation AI based on the collected information and generating an optimal work plan using prompt sentences, and means for presenting work content to workers based on the generated work plan and providing appropriate feedback. This makes it possible to provide plans that optimally correspond to individual learning and working styles and emotional states that change in real time.

[1167] "Learning style" refers to the particular methods and attitudes that students exhibit toward learning.

[1168] "Learning history" refers to a record of what a student has learned and their grades.

[1169] "Generative AI" refers to a system that uses artificial intelligence to generate new information or plans based on specific input information.

[1170] A "prompt" is an instruction given to a generating AI to perform a specific task.

[1171] "Work style" refers to the specific methods and attitudes that a worker exhibits toward work.

[1172] "Work history" refers to a record of the work that a worker has done so far and their achievements.

[1173] An "emotion analysis engine" refers to a system that analyzes collected video and audio data to recognize an individual's emotional state.

[1174] "Fatigue level" refers to an index that indicates the degree of fatigue of workers or students.

[1175] "Concentration" refers to an indicator that shows the current level of concentration of a worker or student.

[1176] A "learning plan" refers to a curriculum designed to help students learn effectively.

[1177] A "work plan" refers to steps and tasks planned to help workers carry out their work efficiently.

[1178] This invention is a system aimed at improving the learning and work efficiency of students and workers. The system generates and presents optimal plans by taking into account the student's learning style and learning history, the worker's working style and work history, and their real-time emotional state.

[1179] Hardware and software used

[1180] Hardware: Tablet device, camera, microphone, AR glasses

[1181] software:

[1182] EmotionRecognition (emotion analysis engine)

[1183] AITaskOptimizer (generative AI engine)

[1184] ServerCommunication (server communication module)

[1185] Processing flow

[1186] Collection of information

[1187] The terminal collects learning and work style and history information from the users (students and workers). For example, students input "I'm good at repeated learning" or "I got 80 points on the last test," while workers input "I'm good at concentrating" or "I made few errors on this line."

[1188] Emotion recognition

[1189] While students and workers are working, real-time video and audio data is collected via cameras and microphones, and EmotionRecognition is used to analyze their emotional states, such as their level of concentration and fatigue.

[1190] Sending data

[1191] The collected information and emotion data are sent to a server, which stores this information in a database and assigns a unique ID to each user.

[1192] Generate a plan

[1193] The server uses generative AI to generate optimal learning and work plans for the user, creating customized plans based on collected learning and work styles, past history, and current emotional state.

[1194] An example of a prompt is as follows:

[1195] "The current worker is very fatigued, please suggest a break."

[1196] "Because you're in a focused state, suggest specific tasks that will take your work speed to the next level."

[1197] Presenting the proposal

[1198] The generated plan is then sent back to the device and presented to the user via a tablet or AR glasses, with dynamic instructions such as "concentrate" or "take a break."

[1199] Answer checking and feedback

[1200] When a user submits the results of their progress based on their learning and work plan, the server uses generative AI to check the accuracy of the answers. It examines the answers to the presented questions and tasks, generates explanations and feedback as needed, and displays them on the device.

[1201] In this way, this system can greatly improve the efficiency of users' learning and work.

[1202] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1203] Step 1:

[1204] Users input information. Specifically, students use tablet devices to input things like, "I'm good at repeated learning" or "I got 80 points on the last test." Workers input information like, "I'm good at concentrating on work" or "I made few errors on this line." This allows data to be collected about learning and work styles and past history. Learning style and history information is obtained as input data and temporarily saved on the device.

[1205] Step 2:

[1206] The device collects video and audio data in real time. It uses a camera and microphone to capture images and voices of users while they are studying or working. The EmotionRecognition engine analyzes this data and recognizes emotional states such as "concentrated" or "tired." The input is video and audio data, and the output is emotional state information.

[1207] Step 3:

[1208] The device sends the collected learning / working style, history data, and emotional state information to the server, which stores this information in a database and assigns a unique ID to each user. The input is the user's learning / working information and emotional information, and the output is a database entry in which this information is stored.

[1209] Step 4:

[1210] The server calls its generative AI model based on the collected information. It issues a prompt to the generative AI, saying, "Please generate a learning plan or work plan that is optimal for the user's current state." The input is learning and work information and emotional information, and the output is the generated plan.

[1211] Step 5:

[1212] The generative AI follows instructions and generates an optimal learning or work plan for the user. For example, it suggests more difficult tasks for a focused user, or a schedule that includes breaks for a tired user. The input is a prompt and user information, and the output is a specific plan.

[1213] Step 6:

[1214] The server organizes the generated plan and sends it to the terminal, where it is converted into the appropriate format for presentation to the user. The input is the generated plan and the output is the formatted plan.

[1215] Step 7:

[1216] The device then displays the received learning or work plan to the user, using a tablet or AR glasses to provide instructions such as "concentrate" or "move to the next task." The input is the formatted plan, and the output is the displayed instructions.

[1217] Step 8:

[1218] The user works based on the presented learning plan or work plan. The student enters the answers to the problems he or she solves into the tablet, and the worker completes the specified task. The input is the result of the user's work, and the output is the answer or task result entered into the device.

[1219] Step 9:

[1220] The device sends the answers and work results submitted by the user to the server. If necessary, emotional data is also sent. The input is the answers and work results, and the output is the data sent to the server.

[1221] Step 10:

[1222] The server uses a generation AI to check the user's answers and work results, and generates a judgment of whether they are correct or incorrect and any necessary explanations. For example, it generates feedback such as "The calculation result of 801 is correct" or "You should take a break." The input is the user's answers and work results, and the output is the results with feedback.

[1223] Step 11:

[1224] The server sends the results and explanations to the device, which include the correct solution, improvements, and advice based on the user's emotional state. The input is the result with feedback, and the output is the explanation sent to the device.

[1225] Step 12:

[1226] The device displays the received result and explanation to the user. For example, it displays "That's right! Well done!" or "That's wrong. The correct answer is 801," providing emotional feedback. The input is the result with feedback, and the output is the displayed explanation and feedback.

[1227] Through these steps, this system can significantly improve the efficiency of users' learning and work.

[1228] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1229] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1230] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1231] [Third embodiment]

[1232] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1233] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1234] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1235] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1236] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1237] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1238] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1239] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1240] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1241] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1242] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1243] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1244] This invention is a learning system that uses generative AI and image analysis technology to provide optimal learning plans based on each student's individual learning style and learning history. Below, we will explain the program processing of this system in natural language.

[1245] Collecting student information

[1246] 1. User (student) enters information

[1247] Users input information about their learning style and learning history into a device (such as a tablet or PC), such as "I'm good at math," "I'm good at repeated learning," or "I got 80 points on the last test."

[1248] 2. The device collects and transmits information

[1249] The device collects the information entered by the user and sends it to the server, which stores it in a database.

[1250] Suggestions for learning content and methods

[1251] 1. The server analyzes the information

[1252] The server extracts the student's learning style and past learning history from the database and analyzes this information.

[1253] 2. The server calls the generated AI

[1254] The server calls a generative AI (e.g., an AI model) and instructs it to generate an optimal learning plan for each student based on the analysis results. For example, it might instruct the server to "generate a set of math problems for students who are good at repetitive learning."

[1255] 3. Generative AI generates a learning plan

[1256] Generative AI creates customized study plans based on a student's learning style and history, such as a plan that includes a series of math problems or a set amount of reading each day.

[1257] 4. The server organizes and sends the study plan

[1258] The server organizes the learning plan received from the generation AI and sends it to the device.

[1259] Presentation of learning content

[1260] 1. The server sends the study plan to the device

[1261] The server sends the generated study plan to the terminal, which receives it and presents it to the user.

[1262] 2. The device displays the learning content

[1263] The device displays questions and learning materials to the user based on the received study plan, for example, displaying math problems or reading material on the screen.

[1264] 3. Users engage in learning

[1265] The user solves problems based on the learning content presented to them, either by typing the answers into the device or by speaking them through the audio glasses.

[1266] Check answers and explain

[1267] 1. User submits answer

[1268] Users can enter answers to the questions they have solved into the device, or they can enter answers by voice using audio glasses.

[1269] 2. The device sends the answer to the server

[1270] The terminal sends the answer entered by the user to the server, which accepts it and performs the necessary processing.

[1271] 3. The server checks the answer

[1272] The server calls the AI ​​generator and evaluates the user's answer against the correct answer. For example, it checks whether the answer to the question "What is 234 + 567?" is "801."

[1273] 4. The server generates the description

[1274] If the user's answer is incorrect, the server uses generative AI to generate text or audio that explains the error, for example, providing step-by-step instructions on how to arrive at the correct answer.

[1275] 5. The server sends the results and explanations to the device.

[1276] The server sends the results of the check and any necessary explanations to the terminal, which displays them to the user.

[1277] 6. The device displays the results and explanations

[1278] The device will show students whether their answers are correct or incorrect, and if they are incorrect, a detailed explanation will be displayed to help them further their learning.

[1279] Specific examples

[1280] For example, here is a fifth-grade student solving a math problem:

[1281] 1. Collecting student information

[1282] A user (e.g., Mr. Tanaka) inputs his / her learning style (he / she is good at repeated learning) and past test scores (80 points) into the terminal. The terminal sends this information to the server, which stores it in a database.

[1283] 2. Suggested learning content and methods

[1284] The server analyzes Tanaka's information, calls the generation AI, and instructs it to "generate a math problem set for students who are good at repeated learning." The generation AI generates the optimal problem set for Tanaka and returns it to the server. The server then sends it to the device.

[1285] 3. Presentation of learning content

[1286] The terminal displays the question "What is 234 + 567?" to Tanaka. Tanaka works on the question and enters the answer as "801."

[1287] 4. Check and explain your answers

[1288] The answer entered by Tanaka is sent from the device to the server, which checks the answer using the generation AI. If it is determined to be correct, the device displays "Correct!" If it is incorrect, the server uses the generation AI to generate an explanation for the mistake and sends it to the device. The device displays this to Tanaka and presents the next practice question.

[1289] This allows us to provide an optimal learning experience tailored to each student's individual characteristics.

[1290] The processing flow will be explained below.

[1291] Program processing flow

[1292] Collecting Student Information

[1293] Step 1:

[1294] The user (student) inputs information about their learning style and learning history into the terminal, such as "I'm good at repeating my learning" or "I got 80 points on the last math test."

[1295] Step 2:

[1296] The device collects the information entered by the user and sends it to the server, including the student ID, grade, learning style, and learning history.

[1297] Step 3:

[1298] The server stores the received information in a database, assigning an ID to each student's information to uniquely identify them.

[1299] Suggestions for learning content and methods

[1300] Step 1:

[1301] The server extracts the student's learning style and past learning history from the database, thereby understanding the student's current learning situation.

[1302] Step 2:

[1303] The server calls the AI ​​generator and instructs it to generate an optimal learning plan based on the extracted data (learning style, learning history), for example, by issuing a specific instruction such as "Generate math problems for a fifth-grade student who is good at repetitive learning."

[1304] Step 3:

[1305] Generative AI creates a personalized learning plan for each student, including multiple practice exercises and regular review.

[1306] Step 4:

[1307] The server receives the generated lesson plan, organizes it, and sends it to the device in the appropriate format.

[1308] Presentation of learning content

[1309] Step 1:

[1310] The server then sends the generated study plan to the device, which includes questions, study materials, and a study schedule.

[1311] Step 2:

[1312] The device then displays the received learning plan to the user, for example, displaying math exercises on the screen and providing audio and text explanations.

[1313] Step 3:

[1314] The user works on the presented problems and either types the answers into the terminal or submits the answers aloud via audio glasses.

[1315] Check answers and explain

[1316] Step 1:

[1317] After solving the problem, the user submits the answer by typing it into the terminal or by voice, and the answer is registered in the terminal.

[1318] Step 2:

[1319] The device sends the user's response to the server, which may include image information and audio data.

[1320] Step 3:

[1321] The server calls the generation AI and checks the answer submitted by the user against the correct answer, for example, checking whether '801' is the correct answer to 234 + 567.

[1322] Step 4:

[1323] The server determines whether the answer is correct, and if it is incorrect, the generative AI creates an explanation including the reason and the correct solution.

[1324] Step 5:

[1325] The server checks the answers and sends them to the device along with explanations, which may include reasons for the correct answers and additional practice questions.

[1326] Step 6:

[1327] The device will display the result and explanation to the student. Specifically, it will say "That's correct!" or "That's wrong. The correct answer is '801'" on the screen, along with a detailed explanation of why the answer was wrong.

[1328] Specific examples

[1329] For example, the processing flow when Tanaka, a fifth-grader, solves a math problem is shown below.

[1330] Step 1:

[1331] Tanaka types into the terminal, "I'm good at repeated learning and got 80 points on the last test."

[1332] Step 2:

[1333] The device collects Tanaka's information and sends it to the server.

[1334] Step 3:

[1335] The server stores the information in a database and calls a generative AI to instruct it to generate a "math problem set for students who excel at repetitive learning."

[1336] Step 4:

[1337] The generation AI generates a plan and sends it back to the server.

[1338] Step 5:

[1339] The server organizes the study plan and sends it to the device.

[1340] Step 6:

[1341] The terminal displays the question "What is 234 + 567?" to Tanaka, who then enters the answer "801."

[1342] Step 7:

[1343] The device sends the entered answers to the server, which checks the answers using a generation AI.

[1344] Step 8:

[1345] The server determines that the answer is correct and sends the result to the terminal.

[1346] Step 9:

[1347] The device displays to Tanaka, "That's correct! Well done!"

[1348] As a result, this learning system provides optimal learning plans and feedback tailored to each student's learning needs.

[1349] Example 1

[1350] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1351] Conventional learning systems face the challenge of providing optimal learning plans tailored to each student's individual characteristics. In particular, there are few systems that can provide individually customized learning plans in real time that properly reflect a student's learning style and past learning history. Furthermore, the process of generating learning plans using generative AI models requires both convenience and accuracy, making it a highly advanced technology. Therefore, there is a need for the development of a system that can efficiently provide learning support tailored to each student.

[1352] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1353] In this invention, the server includes means for collecting and storing information on students' learning styles and learning histories, means for generating an optimal learning plan based on the collected learning styles and learning histories, and means for presenting learning content to students based on the generated learning plan. This makes it possible to provide customized learning plans suited to the learning characteristics of individual students in real time, thereby improving learning efficiency.

[1354] "Learning style" refers to the characteristics that indicate how a student learns best, such as retrieval, visual learning, or reading-based learning.

[1355] "Learning history" refers to information that records a student's past learning activities and achievements, such as past test scores, study time, and what they learned.

[1356] "Means of collection" refers to devices and software that acquire information from users and organize and store it. Examples include tablets, PCs, and dedicated input systems.

[1357] "Storage means" refers to methods or systems for recording and storing collected data in an appropriate format, such as database management systems or cloud storage.

[1358] "Generative means" refers to methods or systems that generate new information or plans based on given input data, specifically generative AI models and algorithms.

[1359] "Presenting means" refers to a method or system for visually or audibly presenting the generated information or plan to the user, including, for example, display on a display or audio announcement.

[1360] "Means of checking" refers to techniques or methods for evaluating submitted data or responses to determine their accuracy. Examples include automated software evaluation and analysis using artificial intelligence.

[1361] "Means for providing explanations" refers to methods or systems for explaining the evaluation results or errors in a way that is easy for users to understand. For example, this could be a text-based explanation or a voice assistant explanation.

[1362] "Device" refers to the input / output device used by a user, including tablets, PCs, smartphones, etc.

[1363] "Server" refers to a central computer that stores and processes data via a network and manages and operates the entire system in cooperation with terminals.

[1364] "Database" refers to a digital information management system that systematically stores collected information and enables it to be quickly accessed and retrieved when needed.

[1365] A "generative AI model" refers to an artificial intelligence mechanism that automatically generates various outputs (such as learning plans) based on input data.

[1366] A "prompt sentence" refers to a text sentence that is input to give specific instructions to a generative AI model.

[1367] "REST API" refers to a set of definitions and protocols that allow different software applications to communicate with each other and exchange data.

[1368] "JSON" stands for JavaScript Object Notation and refers to a lightweight data interchange format for structuring, storing, and transferring data.

[1369] An "HTTP POST request" is an HTTP method for sending data to a web server, which can save or update data.

[1370] This invention is a learning system that utilizes generative AI models and image analysis technology to provide optimal learning plans based on each student's individual learning style and learning history. Specific embodiments of this system are described in detail below.

[1371] First, the user (student) uses a device (tablet or PC) to input information about their learning style and past learning history. For example, they provide information such as "I'm good at math," "I like repeated learning," and "I got 80 points on the last test." The device then organizes this information and sends it to the server.

[1372] The server uses a database management system (e.g., MySQL or PostgreSQL) to store the received information in a database. The server then extracts students' learning styles and past learning histories from the database and analyzes these data using data analysis tools (e.g., the Pandas module in Python).

[1373] Based on the analysis results, the server calls a generative AI model (e.g., GPT-3). It gives instructions to the AI ​​using prompts. Examples of specific prompts include:

[1374] "Generating appropriate math problem sets for users who are good at iterative learning"

[1375] etc.

[1376] The generative AI model generates an optimal study plan based on the provided prompts, which includes math problem sets, reading materials, etc. The generated study plan is sent back to the server in JSON format, which the server parses and sends to the device.

[1377] The device presents the user with learning content based on the received learning plan. Specifically, it uses HTML, CSS, and JavaScript to display various questions and learning materials on the screen. The user's answers to the questions are entered into the device or collected via a voice input device (e.g., audio glasses).

[1378] When a user submits an answer, the device sends it to the server. The server uses a generative AI model to check the user's answer and determine whether it is correct or incorrect. If the answer is incorrect, the server uses the generative AI model to generate an explanation for the mistake. This explanation is also sent to the device in JSON format, for example, and the device displays it to the user.

[1379] As a concrete example, consider a fifth-grade student solving a math problem. The user (student) inputs their learning style (good at repetitive learning) and past test scores (80 points) into the device. The device sends this information to the server, which stores it in a database. The server then sends a prompt to the generative AI model.

[1380] "Generating appropriate math problem sets for users who are good at iterative learning"

[1381] The generative AI model generates an optimal problem set and sends it back to the server. The server sends this to the device. The device then displays a question to the user, such as "What is 234 + 567?", to which the user answers, for example, "801." The user's answer is again sent to the server and checked by the generative AI model. If the answer is correct, a message indicating the correct answer is displayed, and if it is incorrect, an explanation is displayed.

[1382] This makes it possible to provide an optimal learning experience tailored to each student's individual characteristics.

[1383] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1384] Step 1: User Enters Information

[1385] Users use a device (tablet or PC) to input information such as their learning style and past grades. For example, they might say, "I'm good at math," "I'm good at repeated learning," or "I got 80 points on the last test." This information is collected as input data.

[1386] Step 2: Device collects and sends information

[1387] The terminal internally organizes the data entered by the user and converts it into JSON format, then uses a REST API to send this information to the server, which uses an HTTP POST request to transfer the data to the server. The server receives this data and stores it in a database.

[1388] Step 3: The server analyzes the information

[1389] The server extracts the user's learning style and learning history from the database. It then analyzes the data using Python's Pandas module, for example. For example, it calculates and visualizes the distribution of past test scores and learning style trends. The input here is the data extracted from the database, and the output is the analysis results.

[1390] Step 4: The server invokes the generative AI model

[1391] The server sends a prompt to the generative AI model based on the analysis results. A specific prompt might be, "Generate the optimal set of math problems for a user who is good at repeated learning." The input is the analysis results and the prompt, and the output is the generated learning plan.

[1392] Step 5: Generative AI model generates a learning plan

[1393] The generative AI model generates an optimal lesson plan based on the prompt it receives, including a set of math problems and reading material, and outputs a JSON-formatted lesson plan that is sent back to the server.

[1394] Step 6: The server organizes and sends the lesson plan

[1395] The server parses the JSON-formatted learning plan returned by the generative AI model, formats it as needed, and then sends the data to the device using a REST API. The output is a curated learning plan.

[1396] Step 7: Your device will display what it learned

[1397] The device uses HTML, CSS, and JavaScript to display the learning content based on the received lesson plan. For example, it displays math problems or reading material on the screen. In this case, the input is the lesson plan, and the output is the learning content on the user interface.

[1398] Step 8: Users engage in learning

[1399] The user progresses with their learning based on the displayed learning content, for example, by answering math questions. The input is the learning content, and the output is the user's answer.

[1400] Step 9: User submits answer

[1401] The user enters the answer to the problem into the terminal, for example, by typing "801" on a keyboard or by using voice input to answer "234 plus 567 is 801." The input is the user's answer, and the output is the data entered into the terminal.

[1402] Step 10: The device sends the answer to the server

[1403] The terminal organizes the user's answers and sends them to the server using an HTTP POST request. The input is the user's answer data, and the output is the data transferred to the server.

[1404] Step 11: Server checks the answer

[1405] The server calls the generative AI model and checks whether the user's answer is correct. For example, it evaluates whether "234 + 567 is 801." The input is the user's answer and the model's correct answer data, and the output is the correctness of the answer.

[1406] Step 12: Server generates commentary

[1407] If the answer is incorrect, the server uses a generative AI model to generate a detailed explanation for the mistake, for example, creating a step-by-step explanation for arriving at the correct answer. The input is the user's answer data and the prompt text, and the output is the explanation text.

[1408] Step 13: The server sends the results and explanations to the device.

[1409] The server sends the answer's correctness and explanation to the terminal. The output is the correctness and explanation data.

[1410] Step 14: Your device will display the results and explanations.

[1411] The device displays the received results and explanations to the user, for example, a message such as "Your answer is correct!" or "You made a mistake here. Please refer to the next step." The input is the result and explanation data, and the output is the display on the user interface.

[1412] (Application example 1)

[1413] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1414] Existing learning systems have limitations in providing optimal learning plans that are tailored to each student's individual learning style and learning history. They also lack the ability to provide real-time learning support and feedback, resulting in insufficient support for students to learn efficiently. Furthermore, the methods of presenting learning content and providing feedback are inconsistent, resulting in reduced learning effectiveness. Therefore, there is a need for an effective learning system that meets the individual needs of each student.

[1415] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1416] In this invention, the server includes means for collecting and storing information about the student's learning style and learning history, means for generating an optimal learning plan based on the collected learning style and learning history, means for presenting learning content to the student based on the generated learning plan, means for displaying the learning content and feedback using a glasses-type device worn by the student, and means for checking answers submitted by the student and providing explanations as necessary. This allows for the provision of a learning plan optimal for each student's individual learning style in real time, enabling efficient learning support.

[1417] "Student" refers to a learner who uses the learning system.

[1418] "Learning style" refers to the individual characteristics that indicate how students learn most effectively.

[1419] "Learning history" refers to a record of what a student has learned in the past and their grades.

[1420] "Collection and storage means" refers to methods and devices for capturing and storing information about students' learning styles and learning histories.

[1421] A "learning plan" refers to a plan that organizes learning content and methods in a way that is optimal for each student.

[1422] "Generating means" refers to a method or system that creates a learning plan based on the collected information.

[1423] "Presenting means" refers to a method or device for displaying the generated lesson plan to students.

[1424] "Eyeglasses-type device" refers to a wearable device that students wear and use to provide visual information.

[1425] "Feedback" refers to the evaluation and explanation given to students in response to their learning results and answers.

[1426] "Means of checking" refers to the method or device that evaluates the answers submitted by students and determines whether they are correct or incorrect.

[1427] "Means for providing explanations as needed" refers to a method or system that provides guidance or explanations to the correct answer when the answer is incorrect.

[1428] "Generative AI" refers to a system that uses artificial intelligence to generate new data and plans.

[1429] A "prompt sentence" refers to an input sentence that gives specific generation instructions to the generation AI.

[1430] The following system components and process steps are used to implement the invention.

[1431] System Components

[1432] 1. Eyeglass-type device worn by students

[1433] Hardware: Eyeglasses (e.g. Google Glass)

[1434] Software: Application for viewing learning content and receiving feedback

[1435] 2. Server

[1436] Hardware: High-performance server

[1437] Software: Database management systems (e.g., MySQL), generative AI (e.g., OpenAI GPT models), data analysis tools (e.g., Python-based analysis systems)

[1438] Process Overview

[1439] The server generates an optimal learning plan based on the student's learning style and learning history, and displays the learning content and feedback in real time on the glasses-type device worn by the student. Every time the student takes an action, that information is sent to the server, which analyzes the data and generates and provides the next appropriate learning content.

[1440] Detailed processing contents

[1441] 1. Information gathering

[1442] User: Students provide their learning style and learning history by speaking into the glasses, scanning a QR code, or manually entering information.

[1443] Device: The glasses send the collected information to a smartphone via Bluetooth, which then transmits it to a server via Wi-Fi.

[1444] 2. Data storage and analysis

[1445] Server: The server stores the received information in a database (MySQL) and uses data analysis tools to analyze learning styles and history.

[1446] 3. Create a learning plan

[1447] Server: Based on the analysis results, the server sends prompts to a generative AI model (e.g., OpenAI's GPT model) to generate an optimal learning plan. Example prompts:

[1448] Generate math problem sets suitable for middle school students who enjoy repetition.

[1449] 4. Presentation of learning content

[1450] Server: Sends the generated learning plan to the eyeglasses device.

[1451] Device: The glasses-type device displays the learning content according to the received learning plan.

[1452] 5. Providing Feedback

[1453] User: The student engages with the learning content and provides answers via voice or manual input into the glasses.

[1454] Device: The answer is sent to the server, where the generation AI determines whether it is correct or incorrect and generates feedback.

[1455] 6. Representation

[1456] Server: Sends feedback to the glasses-type device and prepares the next appropriate learning content.

[1457] Device: The glasses display feedback and new learnings.

[1458] Specific examples

[1459] For example, here is a scenario where a middle school student is taking extra math classes:

[1460] 1. Information gathering: Students speak into the glasses-like device, saying, "I want to solve a math problem."

[1461] 2. Data storage and analysis: The server receives the voice data and stores it in a database. The server analyzes the student's learning history and determines whether the student is a "junior high school student who excels at repeated learning."

[1462] 3. Learning plan generation: The server sends the following prompt to the generative AI model:

[1463] Generate math problem sets suitable for middle school students who enjoy repetition.

[1464] The generated problem set (e.g., "What is 234 + 567?") is sent to the eyeglass-type device.

[1465] 4. Presentation of learning content: The eyeglasses display "What is 234 + 567?"

[1466] 5. Providing feedback: The student types "801" by voice. The server uses generative AI to determine whether the answer is correct or not, and provides feedback saying, "That's correct!"

[1467] 6. Re-presentation: The server generates the next question and sends it to the glasses-type device. The learning content is displayed again.

[1468] This allows us to provide a learning experience that best suits each student's individual learning style.

[1469] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1470] Step 1:

[1471] Information collection and transmission

[1472] User: Students provide their learning style and learning history by speaking into the glasses, scanning a QR code, or manually entering their learning history.

[1473] Input: Learning style characteristics and learning history (e.g., you are good at repeating things, you got 80 points on the last test, etc.).

[1474] Output: Audio data and manual input data.

[1475] How it works: The glasses send the collected information to a smartphone via Bluetooth, and then to a server via Wi-Fi.

[1476] Step 2:

[1477] Data storage and analysis

[1478] Server: The server stores the received information in a database (e.g., MySQL) and uses a data analysis tool (e.g., a Python-based analysis system) to analyze learning styles and histories.

[1479] Input: Voice data and manual input data.

[1480] Output: Analysis results based on learning style and history.

[1481] Specific operation: After storing the information in the database, the server analyzes the learning style and history and generates prompts for the generative AI model.

[1482] Step 3:

[1483] Generate a learning plan

[1484] Server: Based on the analysis results, it sends prompts to a generative AI model (e.g., OpenAI's GPT model) to generate an optimal learning plan.

[1485] Input: Analysis results based on learning style and history, prompt statement.

[1486] Output: The generated learning plan.

[1487] Specific operation: The server sends the following prompt to the generative AI model: "Please generate a set of math problems suitable for junior high school students who are good at repetitive learning." The generative AI then generates a study plan and sends it back to the server.

[1488] Step 4:

[1489] Presentation of learning content

[1490] Server and device: The server sends the generated learning plan to the eyeglass device, which then displays the learning content to the student.

[1491] Input: The generated learning plan.

[1492] Output: Learning content displayed on the eyeglasses device.

[1493] Specific operation: The glasses-type device presents students with questions such as "What is 234 + 567?"

[1494] Step 5:

[1495] Enter and submit your answers

[1496] User: The student engages with the learning content and provides answers via voice or manual input into the glasses.

[1497] Input: Student's answer (e.g., "801").

[1498] Output: Audio data or manually input data.

[1499] How it works: The glasses send students' responses via Bluetooth to a smartphone, which then transmits them to a server via Wi-Fi.

[1500] Step 6:

[1501] Checking answers and generating feedback

[1502] Server: Checks the received answers using the generative AI model and generates feedback (explanation) if necessary.

[1503] Input: Student answers.

[1504] Output: Correct / incorrect judgment and feedback (explanation).

[1505] What it does: The server matches the student's answer with the correct answer and uses a generative AI model to generate feedback, such as "That's right!" or an explanation if the answer is incorrect.

[1506] Step 7:

[1507] Providing feedback and next steps

[1508] Server and device: The server sends the generated feedback to the eyeglasses device, which prepares the next appropriate learning content. The eyeglasses device displays the feedback and new learning content to the student.

[1509] Input: Feedback and next learning plan.

[1510] Output: Feedback and next steps displayed on the glasses.

[1511] What it does: The glasses provide feedback and next steps to students, providing continuous support for their learning.

[1512] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1513] This invention is a learning system that recognizes a student's learning style, learning history, and emotions and provides an optimal learning plan. The program processing of this system will be specifically explained below.

[1514] Collecting student information

[1515] 1. User (student) enters information

[1516] The user inputs information about their learning style and learning history into the terminal, such as "I'm good at repeating my learning" or "I got 80 points on the last test."

[1517] 2. The device collects and transmits information

[1518] The device collects the information entered by the user and sends it to the server, including the student ID, grade, learning style, and learning history.

[1519] 3. The server stores the information

[1520] The server stores the received information in a database, assigning an ID to each student's information to uniquely identify them.

[1521] Emotion recognition

[1522] 1. Real-time video and audio input by users

[1523] While studying, users input video and audio into the device in real time using a webcam and microphone.

[1524] 2. The device performs emotion analysis

[1525] The device sends the collected video and audio data to an emotion engine that recognizes the student's emotions, such as "concentrating," "tired," or "enjoying" in real time.

[1526] 3. Sending Emotional Data

[1527] The analyzed emotional data is sent from the device to a server and used when generating a learning plan.

[1528] Suggestions for learning content and methods

[1529] 1. The server analyzes the information

[1530] The server extracts information from the database about students' learning styles and past learning histories, as well as emotional data collected in real time, and analyzes this information.

[1531] 2. The server calls the generated AI

[1532] The server calls the AI ​​generator and instructs it to generate an optimal learning plan based on the analysis results, for example, by issuing specific instructions such as "generate math problems for students who are good at repetitive learning and are in a focused state."

[1533] 3. Generative AI generates a learning plan

[1534] The AI ​​generates optimal learning plans based on a student's learning style and emotions. For example, if a student is concentrating, it will increase the difficulty of the questions slightly, and if it determines that the student is tired, it will generate a plan that includes instructions to take a break.

[1535] 4. The server organizes and sends the study plan

[1536] The server organizes the learning plan received from the generation AI and sends it to the device in the appropriate format.

[1537] Presentation of learning content

[1538] 1. The server sends the study plan to the device

[1539] The server then sends the generated study plan to the device, which includes questions, study materials, and a study schedule.

[1540] 2. Your device will display your study plan

[1541] The device then displays the received study plan to the user, for example, displaying math practice problems on the screen and providing dynamic instructions such as "concentrate" and "take breaks as you progress."

[1542] 3. Users engage in learning

[1543] The user studies based on the presented study plan, either by typing answers into the device or by submitting answers aloud via audio glasses.

[1544] Check answers and explain

[1545] 1. User submits answer

[1546] The user submits the answer to the problem by typing it into the terminal or by voice, and the answer is registered in the terminal.

[1547] 2. The device sends the answer to the server

[1548] The device sends the user's response to the server, along with their emotional data if necessary.

[1549] 3. The server checks the answer

[1550] The server calls the generation AI and checks the answer submitted by the user against the correct answer, for example, checking whether '801' is the correct answer to 234 + 567.

[1551] 4. The server generates the description

[1552] If the user's answer is incorrect, the server uses a generative AI to create an explanation including the reason and the correct solution. It also provides feedback according to the user's emotional state.

[1553] 5. The server sends the results and explanations to the device.

[1554] The server checks the answers and sends them to the device along with explanations, including reasons for the correct answers, additional practice questions, and even encouragement and advice based on the user's emotional state.

[1555] 6. The device displays the results and explanations

[1556] The device displays the results and explanations to the student. Specifically, it displays messages such as "That's right! Well done!" or "That's wrong. The correct answer is '801'" on the screen, and also provides feedback that takes emotions into account.

[1557] Specific examples

[1558] For example, the processing flow when Tanaka, a fifth-grader, solves a math problem is shown below.

[1559] Step 1:

[1560] Tanaka types into the terminal, "I'm good at repeated learning and got 80 points on the last test."

[1561] Step 2:

[1562] Video and audio data collected in real time was analyzed to determine that Tanaka was "concentrating."

[1563] Step 3:

[1564] The device collects Tanaka's information and emotional data and sends it to the server.

[1565] Step 4:

[1566] The server stores the information in a database and calls the generation AI, instructing it to generate a "math problem set suitable for Tanaka, a student who excels at repeated learning, and who is currently concentrating."

[1567] Step 5:

[1568] The generation AI generates a plan and sends it back to the server.

[1569] Step 6:

[1570] The server organizes the learning plan and sends it to the device, displaying a question such as "What is 234 + 567?"

[1571] Step 7:

[1572] Tanaka responds by entering "801." The device then sends the response and emotion data to the server.

[1573] Step 8:

[1574] The server checks the answer using the generation AI and determines it to be correct.

[1575] Step 9:

[1576] The server sends the result to the device, which then displays to Tanaka, "That's correct! Maintain your current concentration."

[1577] This allows the learning system to provide optimal learning plans and feedback tailored to each student's learning needs, promoting efficient and effective learning.

[1578] The processing flow will be explained below.

[1579] Program processing flow

[1580] Collecting student information

[1581] Step 1:

[1582] The user (student) inputs information about their learning style and learning history into the terminal. Specifically, they input information such as "I'm good at repeating my learning" or "I got 80 points on the last test."

[1583] Step 2:

[1584] The device collects the information entered by the user and sends it to the server, including the student ID, grade, learning style, and learning history.

[1585] Step 3:

[1586] The server stores the received information in a database, assigning an ID to each student's information to uniquely identify them.

[1587] Emotion recognition

[1588] Step 1:

[1589] While learning, users input video and audio into the device in real time, and the device uses a webcam and microphone to collect the user's facial expressions and tone of voice.

[1590] Step 2:

[1591] The device sends the collected video and audio data to the emotion engine, which analyzes this data and recognizes the user's emotions, such as "concentrated," "tired," or "enjoyed."

[1592] Step 3:

[1593] The emotion data analyzed by the emotion engine is sent to the server along with the user's learning style and learning history.

[1594] Suggestions for learning content and methods

[1595] Step 1:

[1596] The server extracts the student's learning style and past learning history from the database, as well as emotional data collected in real time, and analyzes this information.

[1597] Step 2:

[1598] The server calls the AI ​​and instructs it to generate an optimal learning plan based on the extracted data (learning style, learning history, emotional data).For example, it gives specific instructions such as, "Generate math problems for students who are good at repetitive learning and are in a focused state."

[1599] Step 3:

[1600] The generative AI generates optimal study plans based on a student's learning style and emotions. For example, if a student is concentrating, it might increase the difficulty of the questions slightly, or if it detects fatigue, it might provide a plan that includes instructions to take a break.

[1601] Step 4:

[1602] The server receives the generated lesson plan, organizes it, and sends it to the device in the appropriate format.

[1603] Presentation of learning content

[1604] Step 1:

[1605] The server then sends the generated study plan to the device, which includes questions, study materials, and a study schedule.

[1606] Step 2:

[1607] The device then displays the received study plan to the user, for example, displaying math exercises on the screen and providing dynamic instructions such as "stay focused" or "you look tired, take a break."

[1608] Step 3:

[1609] The user studies based on the presented study plan, either by typing answers into the device or by submitting answers aloud via audio glasses.

[1610] Check answers and explain

[1611] Step 1:

[1612] After solving the problem, the user submits the answer by typing it into the terminal or by voice, and the answer is registered in the terminal.

[1613] Step 2:

[1614] The device sends the user's response to the server, along with their emotional data if necessary.

[1615] Step 3:

[1616] The server calls the generation AI and checks the answer submitted by the user against the correct answer, for example, checking whether '801' is the correct answer to 234 + 567.

[1617] Step 4:

[1618] The server determines whether the answer is correct, and if it is incorrect, the generative AI creates an explanation including the reason and the correct solution. It also provides feedback based on the user's emotional state.

[1619] Step 5:

[1620] The server checks the answers and sends them to the device along with explanations, including reasons for the correct answers, additional practice questions, and even encouragement and advice based on the user's emotional state.

[1621] Step 6:

[1622] The device displays the results and explanations to the student, such as "That's right! Well done!" or "That's wrong. The correct answer is '801'," and also provides emotional feedback.

[1623] Specific examples

[1624] For example, the processing flow when Tanaka, a fifth-grader, solves a math problem is shown below.

[1625] Step 1:

[1626] Tanaka types into the terminal, "I'm good at repeated learning and got 80 points on the last test."

[1627] Step 2:

[1628] Video and audio data collected in real time was analyzed to determine that Tanaka was "concentrating."

[1629] Step 3:

[1630] The device collects Tanaka's information and emotional data and sends it to the server.

[1631] Step 4:

[1632] The server stores the information in a database and calls the generation AI, instructing it to generate a "math problem set suitable for Tanaka, a student who excels at repeated learning, and who is currently concentrating."

[1633] Step 5:

[1634] The generation AI generates a plan and sends it back to the server.

[1635] Step 6:

[1636] The server organizes the study plan and sends it to the device, which then displays the question "What is 234 + 567?" to Tanaka.

[1637] Step 7:

[1638] Tanaka responds by entering "801." The device then sends the response and emotion data to the server.

[1639] Step 8:

[1640] The server checks the answer using the generation AI, and if it is determined to be correct, it sends the result to the device.

[1641] Step 9:

[1642] The device displays to Tanaka, "That's correct! Maintain your current concentration."

[1643] This allows the learning system to provide optimal learning plans and feedback tailored to each student's learning needs, promoting efficient and effective learning.

[1644] Example 2

[1645] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1646] Modern learning systems are expected to provide individualized learning plans based on students' learning styles and learning histories. However, traditional systems only use static data and are unable to adjust learning plans to take students' real-time emotions into account. This makes it difficult to provide dynamic learning plans that optimally maintain students' focus and motivation.

[1647] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1648] In this invention, the server includes means for collecting and storing information on students' learning styles and learning histories, means for generating optimal learning plans based on the collected learning styles and learning histories, means for collecting real-time video and audio data, performing emotion analysis, and storing the data, and means for adjusting and generating learning plans based on the analyzed emotion data, thereby making it possible to provide learning plans that take into account the dynamic emotional states of students.

[1649] "Student learning style" refers to the individual characteristics of how and how students prefer to learn.

[1650] "Learning history" refers to data recorded about what learning activities a student has engaged in in the past, including the results and achievements.

[1651] "Means for collecting and storing information" refers to devices and systems for inputting, recording, and storing information about students' learning styles and learning histories.

[1652] "Means for generating optimal learning plans" refers to devices or systems that create learning content and schedules that are most suitable for individual students based on collected information.

[1653] "Means for presenting learning content to students" refers to devices or systems that display and provide specific learning tasks and learning materials to students based on the generated learning plan.

[1654] "Means for checking answers and providing explanations" refers to a device or system that verifies the answers submitted by students, determines whether they are correct, and provides explanations or feedback as needed.

[1655] "Real-time video and audio data" refers to video and audio information that records students' situations and reactions in real time while they are learning.

[1656] "Emotion analysis" refers to the process of analyzing and recognizing students' emotional states (e.g., focused, tired, or enjoying) from video and audio data collected in real time.

[1657] "Generative AI" refers to artificial intelligence models that automatically perform specific tasks (e.g., generating lesson plans, checking answers) based on collected data.

[1658] "Study plan adjustment" refers to the process of dynamically modifying and optimizing existing study plans based on data obtained through sentiment analysis.

[1659] This invention is a learning system that recognizes a student's learning style, learning history, and emotions to provide an optimal learning plan. Next, the program processing of this system will be specifically explained.

[1660] Hardware and software used

[1661] This system uses the following hardware and software:

[1662] Server: Database management system (e.g., MySQL), generative AI (e.g., OpenAI GPT-4), data analysis library (e.g., pandas, NumPy)

[1663] Device: Webcam, microphone, emotion analysis engine (e.g., OpenCV, TensorFlow), user interface

[1664] Communication protocol: HTTPS

[1665] Processing flow

[1666] Collecting student information

[1667] 1. The user enters information

[1668] The user inputs information about their learning style and learning history into the terminal, such as "I'm good at repeating my learning" or "I got 80 points on the last test."

[1669] 2. The device collects information and sends it to the server

[1670] The terminal collects the information entered by the user (student ID, grade, learning style, learning history) and sends it to the server via an HTTPS request.

[1671] 3. The server stores the information in a database

[1672] The server stores the received information in a database, using an SQL query to insert the input data using the student ID as the primary key.

[1673] Emotion recognition

[1674] 1. Users input real-time video and audio

[1675] While studying, users input video and audio into the device using a webcam and microphone.

[1676] 2. The device performs emotion analysis

[1677] The device sends the collected video and audio data to an emotion analysis engine, which analyzes the user's emotional state (concentrated, tired, enjoying, etc.) in real time.

[1678] 3. Sending Emotional Data

[1679] The device sends the analyzed emotion data to the server in JSON format.

[1680] Suggestions for learning content and methods

[1681] 1. The server analyzes the information

[1682] The server extracts students' learning styles, past learning history, and real-time emotional data from the database and analyzes the data using Python scripts.

[1683] 2. The server calls the generated AI

[1684] The server calls the generative AI and sends prompts that generate the optimal learning plan based on the analysis results.

[1685] Example: "Generate math problems for students who are good at repetition and are in a focused state."

[1686] 3. Generative AI generates learning plans

[1687] Based on the prompts, the generative AI generates an optimal study plan that includes the difficulty of the questions, the content to be studied, and a schedule.

[1688] 4. The server organizes and sends the generated lesson plans

[1689] The server organizes the learning plan received from the generation AI and sends it to the terminal in JSON format.

[1690] Presentation of learning content

[1691] 1. The server sends the study plan to the device

[1692] The server sends the organized lesson plan to the device, which is sent as an HTTP response.

[1693] 2. Your device will display your study plan

[1694] The device displays the received lesson plan on the screen to the user. For example, the device displays a math problem such as "What is 234 + 567?"

[1695] 3. Users engage in learning

[1696] The user solves problems based on the presented learning plan, and submits answers by typing them into the device or by voice.

[1697] Check answers and explain

[1698] 1. User submits answer

[1699] Users submit their answers to the questions they have solved by typing them into the device or by voice, and the answer data is saved on the device.

[1700] 2. The device sends the answer to the server

[1701] The device sends the user's response data to the server, including emotion data if necessary.

[1702] 3. The server checks the answer

[1703] The server calls the AI ​​generator and checks the user's answer against the correct answer. It sends a prompt to the AI ​​generator asking if '801' is the correct answer to 234 + 567.

[1704] 4. The server generates the description

[1705] If the user answers incorrectly, the server uses the generative AI again to generate an explanation, including the reason for the incorrect answer and the correct solution. It also provides feedback based on the user's emotional state.

[1706] 5. The server sends the results and explanations to the device.

[1707] The server then sends the results of the check and explanation to the device, including reasons for the correct answer, additional practice questions, and encouragement or advice based on the user's emotional state.

[1708] 6. The device will display the results and explanations.

[1709] The device displays the results and explanations on the screen, such as "That's right! Well done!" or "That's wrong. The correct answer is '801'," and also provides emotional feedback.

[1710] This allows the learning system to provide optimal learning plans and feedback tailored to each student's individual needs, promoting efficient and effective learning.

[1711] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1712] Step 1:

[1713] The user enters information

[1714] The user inputs information about their learning style and learning history into the device. For example, they might input information such as "I'm good at repeating my learning" or "I got 80 points on the last test." The input data includes student ID, grade, learning style, learning history, etc. The device temporarily stores this input data in its memory.

[1715] Step 2:

[1716] The device collects information and sends it to the server

[1717] The device collects information entered by the user and sends it to the server using an HTTPS request (input: user information, output: data sent to the server). The collected data is converted into JSON format and sent to the server using the HTTPS protocol.

[1718] Step 3:

[1719] The server stores the information in a database

[1720] The server saves the received information in a database. Specifically, it inserts the information into the database as a new record using an SQL query (input: user information, output: save to database). The saved data includes a unique student ID.

[1721] Step 4:

[1722] Users input real-time video and audio

[1723] While studying, users input real-time video and audio data into the device using a webcam and microphone. For example, while the user is solving a math problem, the device collects video of the user's facial expressions and records audio of their reactions (input: real-time video and audio data, output: temporary storage on the device).

[1724] Step 5:

[1725] The device performs emotion analysis

[1726] The device sends the collected video and audio data to an emotion analysis engine, which analyzes the user's emotions in real time. Specifically, OpenCV and TensorFlow are used to recognize emotional states such as "concentrated," "tired," and "enjoyed" (input: video and audio data, output: emotional state data).

[1727] Step 6:

[1728] The device sends emotion data to the server.

[1729] The device sends the analyzed emotion data to the server in JSON format, which includes the emotion data acquired in real time and the student ID (input: emotion state data, output: data transmission to the server).

[1730] Step 7:

[1731] The server analyzes the information

[1732] The server extracts and analyzes students' learning styles, learning histories, and real-time emotional data from the database. It then analyzes the data using Python scripts and data analysis libraries (e.g., pandas, NumPy) to determine the optimal learning plan for each student (input: student information, learning history, emotional data; output: analysis results).

[1733] Step 8:

[1734] The server calls the generated AI

[1735] The server calls a generative AI (e.g., OpenAI GPT-4) and sends a prompt that generates an optimal learning plan based on the analysis results (input: analysis results, output: generation of prompt).

[1736] Example: "Generate math problems for students who are good at repetition and are in a focused state."

[1737] Step 9:

[1738] Generative AI generates learning plans

[1739] The generation AI generates an optimal learning plan based on the prompt received from the server. The plan includes the difficulty level of the questions, learning content, and schedule (input: prompt, output: learning plan).

[1740] Step 10:

[1741] The server organizes and sends the generated lesson plans

[1742] The server organizes the learning plan received from the generation AI and sends it to the device. It converts it into the appropriate format in JSON format and sends it as an HTTPS response (input: learning plan, output: transmission to device).

[1743] Step 11:

[1744] Your device will display your study plan

[1745] The device displays the learning plan received from the server to the user. For example, a math problem such as "What is 234 + 567?" is displayed on the screen (input: learning plan, output: display to user).

[1746] Step 12:

[1747] Users engage in learning

[1748] The user solves the exercises based on the presented learning plan, and either enters the answers into the device or submits them by voice (input: user's answers, output: input to the device).

[1749] Step 13:

[1750] User submits answer

[1751] The user inputs the answer to the problem they solved into the terminal, which temporarily stores this data (input: user's answer, output: saved in the terminal).

[1752] Step 14:

[1753] The device sends the answer to the server

[1754] The device sends the user's response in JSON format to the server, including real-time emotional data if necessary (input: user's response, output: transmission to the server).

[1755] Step 15:

[1756] The server checks the answer

[1757] The server calls the generation AI and checks the answer submitted by the user by comparing it with the correct answer. For example, it checks whether "801" is the correct answer to 234 + 567" (input: user answer, output: correct or incorrect).

[1758] Step 16:

[1759] The server generates the description

[1760] When a user gives an incorrect answer, the server uses a generative AI to generate an explanation including the reason and the correct solution. It also provides feedback according to the user's emotional state (input: judgment result of correctness, output: explanation and feedback).

[1761] Step 17:

[1762] The server sends the results and explanations to the device.

[1763] The server checks the answers and sends them to the device together with an explanation. The results include reasons for the correct answer, additional practice questions, and encouragement and advice based on the user's emotional state (input: explanation and feedback, output: transmission to device).

[1764] Step 18:

[1765] The device displays the results and explanations

[1766] The device displays the result and explanation on the screen. Specifically, it displays "That's right! Well done!" or "That's wrong. The correct answer is '801'," and also provides feedback that takes emotions into account (input: result and explanation, output: displayed to the user).

[1767] (Application example 2)

[1768] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1769] Conventional learning and work support systems have had difficulty providing optimal plans that accurately and quickly reflect an individual's learning and work style, as well as their emotional state, which changes in real time. As a result, it has been difficult to achieve effective learning and efficient work support, leading to problems such as reduced learning effectiveness and work efficiency.

[1770] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for collecting and saving information on students' learning styles and learning histories, means for generating an optimal learning plan based on the collected learning styles and learning histories, means for presenting learning content to students based on the generated learning plan, means for checking answers submitted by students and providing explanations as necessary, means for collecting and saving information on workers' working styles and working histories, means for generating an optimal work plan based on the collected working styles and working histories, means for recognizing emotional states in real time and adjusting the work plan based thereon, means for recognizing the worker's fatigue level and concentration level using an emotion analysis engine, means for transmitting the collected emotional data to the server, means for calling a generation AI based on the collected information and generating an optimal work plan using prompt sentences, and means for presenting work content to workers based on the generated work plan and providing appropriate feedback. This makes it possible to provide plans that optimally correspond to individual learning and working styles and emotional states that change in real time.

[1771] "Learning style" refers to the particular methods and attitudes that students exhibit toward learning.

[1772] "Learning history" refers to a record of what a student has learned and their grades.

[1773] "Generative AI" refers to a system that uses artificial intelligence to generate new information or plans based on specific input information.

[1774] A "prompt" is an instruction given to a generating AI to perform a specific task.

[1775] "Work style" refers to the specific methods and attitudes that a worker exhibits toward work.

[1776] "Work history" refers to a record of the work that a worker has done so far and their achievements.

[1777] An "emotion analysis engine" refers to a system that analyzes collected video and audio data to recognize an individual's emotional state.

[1778] "Fatigue level" refers to an index that indicates the degree of fatigue of workers or students.

[1779] "Concentration" refers to an indicator that shows the current level of concentration of a worker or student.

[1780] A "learning plan" refers to a curriculum designed to help students learn effectively.

[1781] A "work plan" refers to steps and tasks planned to help workers carry out their work efficiently.

[1782] This invention is a system aimed at improving the learning and work efficiency of students and workers. The system generates and presents optimal plans by taking into account the student's learning style and learning history, the worker's working style and work history, and their real-time emotional state.

[1783] Hardware and software used

[1784] Hardware: Tablet device, camera, microphone, AR glasses

[1785] software:

[1786] EmotionRecognition (emotion analysis engine)

[1787] AITaskOptimizer (generative AI engine)

[1788] ServerCommunication (server communication module)

[1789] Processing flow

[1790] Collection of information

[1791] The terminal collects learning and work style and history information from the users (students and workers). For example, students input "I'm good at repeated learning" or "I got 80 points on the last test," while workers input "I'm good at concentrating" or "I made few errors on this line."

[1792] Emotion recognition

[1793] While students and workers are working, real-time video and audio data is collected via cameras and microphones, and EmotionRecognition is used to analyze their emotional states, such as their level of concentration and fatigue.

[1794] Sending data

[1795] The collected information and emotion data are sent to a server, which stores this information in a database and assigns a unique ID to each user.

[1796] Generate a plan

[1797] The server uses generative AI to generate optimal learning and work plans for the user, creating customized plans based on collected learning and work styles, past history, and current emotional state.

[1798] An example of a prompt is as follows:

[1799] "The current worker is very fatigued, please suggest a break."

[1800] "Because you're in a focused state, suggest specific tasks that will take your work speed to the next level."

[1801] Presenting the proposal

[1802] The generated plan is then sent back to the device and presented to the user via a tablet or AR glasses, with dynamic instructions such as "concentrate" or "take a break."

[1803] Answer checking and feedback

[1804] When a user submits the results of their progress based on their learning and work plan, the server uses generative AI to check the accuracy of the answers. It examines the answers to the presented questions and tasks, generates explanations and feedback as needed, and displays them on the device.

[1805] In this way, this system can greatly improve the efficiency of users' learning and work.

[1806] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1807] Step 1:

[1808] Users input information. Specifically, students use tablet devices to input things like, "I'm good at repeated learning" or "I got 80 points on the last test." Workers input information like, "I'm good at concentrating on work" or "I made few errors on this line." This allows data to be collected about learning and work styles and past history. Learning style and history information is obtained as input data and temporarily saved on the device.

[1809] Step 2:

[1810] The device collects video and audio data in real time. It uses a camera and microphone to capture images and voices of users while they are studying or working. The EmotionRecognition engine analyzes this data and recognizes emotional states such as "concentrated" or "tired." The input is video and audio data, and the output is emotional state information.

[1811] Step 3:

[1812] The device sends the collected learning / working style, history data, and emotional state information to the server, which stores this information in a database and assigns a unique ID to each user. The input is the user's learning / working information and emotional information, and the output is a database entry in which this information is stored.

[1813] Step 4:

[1814] The server calls its generative AI model based on the collected information. It issues a prompt to the generative AI, saying, "Please generate a learning plan or work plan that is optimal for the user's current state." The input is learning and work information and emotional information, and the output is the generated plan.

[1815] Step 5:

[1816] The generative AI follows instructions and generates an optimal learning or work plan for the user. For example, it suggests more difficult tasks for a focused user, or a schedule that includes breaks for a tired user. The input is a prompt and user information, and the output is a specific plan.

[1817] Step 6:

[1818] The server organizes the generated plan and sends it to the terminal, where it is converted into the appropriate format for presentation to the user. The input is the generated plan and the output is the formatted plan.

[1819] Step 7:

[1820] The device then displays the received learning or work plan to the user, using a tablet or AR glasses to provide instructions such as "concentrate" or "move to the next task." The input is the formatted plan, and the output is the displayed instructions.

[1821] Step 8:

[1822] The user works based on the presented learning plan or work plan. The student enters the answers to the problems he or she solves into the tablet, and the worker completes the specified task. The input is the result of the user's work, and the output is the answer or task result entered into the device.

[1823] Step 9:

[1824] The device sends the answers and work results submitted by the user to the server. If necessary, emotional data is also sent. The input is the answers and work results, and the output is the data sent to the server.

[1825] Step 10:

[1826] The server uses a generation AI to check the user's answers and work results, and generates a judgment of whether they are correct or incorrect and any necessary explanations. For example, it generates feedback such as "The calculation result of 801 is correct" or "You should take a break." The input is the user's answers and work results, and the output is the results with feedback.

[1827] Step 11:

[1828] The server sends the results and explanations to the device, which include the correct solution, improvements, and advice based on the user's emotional state. The input is the result with feedback, and the output is the explanation sent to the device.

[1829] Step 12:

[1830] The device displays the received result and explanation to the user. For example, it displays "That's right! Well done!" or "That's wrong. The correct answer is 801," providing emotional feedback. The input is the result with feedback, and the output is the displayed explanation and feedback.

[1831] Through these steps, this system can significantly improve the efficiency of users' learning and work.

[1832] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1833] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1834] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1835] [Fourth embodiment]

[1836] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1837] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1838] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1839] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1840] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1841] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1842] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1843] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1844] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1845] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1846] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1847] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1848] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1849] This invention is a learning system that uses generative AI and image analysis technology to provide optimal learning plans based on each student's individual learning style and learning history. Below, we will explain the program processing of this system in natural language.

[1850] Collecting student information

[1851] 1. User (student) enters information

[1852] Users input information about their learning style and learning history into a device (such as a tablet or PC), such as "I'm good at math," "I'm good at repeated learning," or "I got 80 points on the last test."

[1853] 2. The device collects and transmits information

[1854] The device collects the information entered by the user and sends it to the server, which stores it in a database.

[1855] Suggestions for learning content and methods

[1856] 1. The server analyzes the information

[1857] The server extracts the student's learning style and past learning history from the database and analyzes this information.

[1858] 2. The server calls the generated AI

[1859] The server calls a generative AI (e.g., an AI model) and instructs it to generate an optimal learning plan for each student based on the analysis results. For example, it might instruct the server to "generate a set of math problems for students who are good at repetitive learning."

[1860] 3. Generative AI generates a learning plan

[1861] Generative AI creates customized study plans based on a student's learning style and history, such as a plan that includes a series of math problems or a set amount of reading each day.

[1862] 4. The server organizes and sends the study plan

[1863] The server organizes the learning plan received from the generation AI and sends it to the device.

[1864] Presentation of learning content

[1865] 1. The server sends the study plan to the device

[1866] The server sends the generated study plan to the terminal, which receives it and presents it to the user.

[1867] 2. The device displays the learning content

[1868] The device displays questions and learning materials to the user based on the received study plan, for example, displaying math problems or reading material on the screen.

[1869] 3. Users engage in learning

[1870] The user solves problems based on the learning content presented to them, either by typing the answers into the device or by speaking them through the audio glasses.

[1871] Check answers and explain

[1872] 1. User submits answer

[1873] Users can enter answers to the questions they have solved into the device, or they can enter answers by voice using audio glasses.

[1874] 2. The device sends the answer to the server

[1875] The terminal sends the answer entered by the user to the server, which accepts it and performs the necessary processing.

[1876] 3. The server checks the answer

[1877] The server calls the AI ​​generator and evaluates the user's answer against the correct answer. For example, it checks whether the answer to the question "What is 234 + 567?" is "801."

[1878] 4. The server generates the description

[1879] If the user's answer is incorrect, the server uses generative AI to generate text or audio that explains the error, for example, providing step-by-step instructions on how to arrive at the correct answer.

[1880] 5. The server sends the results and explanations to the device.

[1881] The server sends the results of the check and any necessary explanations to the terminal, which displays them to the user.

[1882] 6. The device displays the results and explanations

[1883] The device will show students whether their answers are correct or incorrect, and if they are incorrect, a detailed explanation will be displayed to help them further their learning.

[1884] Specific examples

[1885] For example, here is a fifth-grade student solving a math problem:

[1886] 1. Collecting student information

[1887] A user (e.g., Mr. Tanaka) inputs his / her learning style (he / she is good at repeated learning) and past test scores (80 points) into the terminal. The terminal sends this information to the server, which stores it in a database.

[1888] 2. Suggested learning content and methods

[1889] The server analyzes Tanaka's information, calls the generation AI, and instructs it to "generate a math problem set for students who are good at repeated learning." The generation AI generates the optimal problem set for Tanaka and returns it to the server. The server then sends it to the device.

[1890] 3. Presentation of learning content

[1891] The terminal displays the question "What is 234 + 567?" to Tanaka. Tanaka works on the question and enters the answer as "801."

[1892] 4. Check and explain your answers

[1893] The answer entered by Tanaka is sent from the device to the server, which checks the answer using the generation AI. If it is determined to be correct, the device displays "Correct!" If it is incorrect, the server uses the generation AI to generate an explanation for the mistake and sends it to the device. The device displays this to Tanaka and presents the next practice question.

[1894] This allows us to provide an optimal learning experience tailored to each student's individual characteristics.

[1895] The processing flow will be explained below.

[1896] Program processing flow

[1897] Collecting Student Information

[1898] Step 1:

[1899] The user (student) inputs information about their learning style and learning history into the terminal, such as "I'm good at repeating my learning" or "I got 80 points on the last math test."

[1900] Step 2:

[1901] The device collects the information entered by the user and sends it to the server, including the student ID, grade, learning style, and learning history.

[1902] Step 3:

[1903] The server stores the received information in a database, assigning an ID to each student's information to uniquely identify them.

[1904] Suggestions for learning content and methods

[1905] Step 1:

[1906] The server extracts the student's learning style and past learning history from the database, thereby understanding the student's current learning situation.

[1907] Step 2:

[1908] The server calls the AI ​​generator and instructs it to generate an optimal learning plan based on the extracted data (learning style, learning history), for example, by issuing a specific instruction such as "Generate math problems for a fifth-grade student who is good at repetitive learning."

[1909] Step 3:

[1910] Generative AI creates a personalized learning plan for each student, including multiple practice exercises and regular review.

[1911] Step 4:

[1912] The server receives the generated lesson plan, organizes it, and sends it to the device in the appropriate format.

[1913] Presentation of learning content

[1914] Step 1:

[1915] The server then sends the generated study plan to the device, which includes questions, study materials, and a study schedule.

[1916] Step 2:

[1917] The device then displays the received learning plan to the user, for example, displaying math exercises on the screen and providing audio and text explanations.

[1918] Step 3:

[1919] The user works on the presented problems and either types the answers into the terminal or submits the answers aloud via audio glasses.

[1920] Check answers and explain

[1921] Step 1:

[1922] After solving the problem, the user submits the answer by typing it into the terminal or by voice, and the answer is registered in the terminal.

[1923] Step 2:

[1924] The device sends the user's response to the server, which may include image information and audio data.

[1925] Step 3:

[1926] The server calls the generation AI and checks the answer submitted by the user against the correct answer, for example, checking whether '801' is the correct answer to 234 + 567.

[1927] Step 4:

[1928] The server determines whether the answer is correct, and if it is incorrect, the generative AI creates an explanation including the reason and the correct solution.

[1929] Step 5:

[1930] The server checks the answers and sends them to the device along with explanations, which may include reasons for the correct answers and additional practice questions.

[1931] Step 6:

[1932] The device will display the result and explanation to the student. Specifically, it will say "That's correct!" or "That's wrong. The correct answer is '801'" on the screen, along with a detailed explanation of why the answer was wrong.

[1933] Specific examples

[1934] For example, the processing flow when Tanaka, a fifth-grader, solves a math problem is shown below.

[1935] Step 1:

[1936] Tanaka types into the terminal, "I'm good at repeated learning and got 80 points on the last test."

[1937] Step 2:

[1938] The device collects Tanaka's information and sends it to the server.

[1939] Step 3:

[1940] The server stores the information in a database and calls a generative AI to instruct it to generate a "math problem set for students who excel at repetitive learning."

[1941] Step 4:

[1942] The generation AI generates a plan and sends it back to the server.

[1943] Step 5:

[1944] The server organizes the study plan and sends it to the device.

[1945] Step 6:

[1946] The terminal displays the question "What is 234 + 567?" to Tanaka, who then enters the answer "801."

[1947] Step 7:

[1948] The device sends the entered answers to the server, which checks the answers using a generation AI.

[1949] Step 8:

[1950] The server determines that the answer is correct and sends the result to the terminal.

[1951] Step 9:

[1952] The device displays to Tanaka, "That's correct! Well done!"

[1953] As a result, this learning system provides optimal learning plans and feedback tailored to each student's learning needs.

[1954] Example 1

[1955] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1956] Conventional learning systems face the challenge of providing optimal learning plans tailored to each student's individual characteristics. In particular, there are few systems that can provide individually customized learning plans in real time that properly reflect a student's learning style and past learning history. Furthermore, the process of generating learning plans using generative AI models requires both convenience and accuracy, making it a highly advanced technology. Therefore, there is a need for the development of a system that can efficiently provide learning support tailored to each student.

[1957] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1958] In this invention, the server includes means for collecting and storing information on students' learning styles and learning histories, means for generating an optimal learning plan based on the collected learning styles and learning histories, and means for presenting learning content to students based on the generated learning plan. This makes it possible to provide customized learning plans suited to the learning characteristics of individual students in real time, thereby improving learning efficiency.

[1959] "Learning style" refers to the characteristics that indicate how a student learns best, such as retrieval, visual learning, or reading-based learning.

[1960] "Learning history" refers to information that records a student's past learning activities and achievements, such as past test scores, study time, and what they learned.

[1961] "Means of collection" refers to devices and software that acquire information from users and organize and store it. Examples include tablets, PCs, and dedicated input systems.

[1962] "Storage means" refers to methods or systems for recording and storing collected data in an appropriate format, such as database management systems or cloud storage.

[1963] "Generative means" refers to methods or systems that generate new information or plans based on given input data, specifically generative AI models and algorithms.

[1964] "Presenting means" refers to a method or system for visually or audibly presenting the generated information or plan to the user, including, for example, display on a display or audio announcement.

[1965] "Means of checking" refers to techniques or methods for evaluating submitted data or responses to determine their accuracy. Examples include automated software evaluation and analysis using artificial intelligence.

[1966] "Means for providing explanations" refers to methods or systems for explaining the evaluation results or errors in a way that is easy for users to understand. For example, this could be a text-based explanation or a voice assistant explanation.

[1967] "Device" refers to the input / output device used by a user, including tablets, PCs, smartphones, etc.

[1968] "Server" refers to a central computer that stores and processes data via a network and manages and operates the entire system in cooperation with terminals.

[1969] "Database" refers to a digital information management system that systematically stores collected information and enables it to be quickly accessed and retrieved when needed.

[1970] A "generative AI model" refers to an artificial intelligence mechanism that automatically generates various outputs (such as learning plans) based on input data.

[1971] A "prompt sentence" refers to a text sentence that is input to give specific instructions to a generative AI model.

[1972] "REST API" refers to a set of definitions and protocols that allow different software applications to communicate with each other and exchange data.

[1973] "JSON" stands for JavaScript Object Notation and refers to a lightweight data interchange format for structuring, storing, and transferring data.

[1974] An "HTTP POST request" is an HTTP method for sending data to a web server, which can save or update data.

[1975] This invention is a learning system that utilizes generative AI models and image analysis technology to provide optimal learning plans based on each student's individual learning style and learning history. Specific embodiments of this system are described in detail below.

[1976] First, the user (student) uses a device (tablet or PC) to input information about their learning style and past learning history. For example, they provide information such as "I'm good at math," "I like repeated learning," and "I got 80 points on the last test." The device then organizes this information and sends it to the server.

[1977] The server uses a database management system (e.g., MySQL or PostgreSQL) to store the received information in a database. The server then extracts students' learning styles and past learning histories from the database and analyzes these data using data analysis tools (e.g., the Pandas module in Python).

[1978] Based on the analysis results, the server calls a generative AI model (e.g., GPT-3). It gives instructions to the AI ​​using prompts. Examples of specific prompts include:

[1979] "Generating appropriate math problem sets for users who are good at iterative learning"

[1980] etc.

[1981] The generative AI model generates an optimal study plan based on the provided prompts, which includes math problem sets, reading materials, etc. The generated study plan is sent back to the server in JSON format, which the server parses and sends to the device.

[1982] The device presents the user with learning content based on the received learning plan. Specifically, it uses HTML, CSS, and JavaScript to display various questions and learning materials on the screen. The user's answers to the questions are entered into the device or collected via a voice input device (e.g., audio glasses).

[1983] When a user submits an answer, the device sends it to the server. The server uses a generative AI model to check the user's answer and determine whether it is correct or incorrect. If the answer is incorrect, the server uses the generative AI model to generate an explanation for the mistake. This explanation is also sent to the device in JSON format, for example, and the device displays it to the user.

[1984] As a concrete example, consider a fifth-grade student solving a math problem. The user (student) inputs their learning style (good at repetitive learning) and past test scores (80 points) into the device. The device sends this information to the server, which stores it in a database. The server then sends a prompt to the generative AI model.

[1985] "Generating appropriate math problem sets for users who are good at iterative learning"

[1986] The generative AI model generates an optimal problem set and sends it back to the server. The server sends this to the device. The device then displays a question to the user, such as "What is 234 + 567?", to which the user answers, for example, "801." The user's answer is again sent to the server and checked by the generative AI model. If the answer is correct, a message indicating the correct answer is displayed, and if it is incorrect, an explanation is displayed.

[1987] This makes it possible to provide an optimal learning experience tailored to each student's individual characteristics.

[1988] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1989] Step 1: User Enters Information

[1990] Users use a device (tablet or PC) to input information such as their learning style and past grades. For example, they might say, "I'm good at math," "I'm good at repeated learning," or "I got 80 points on the last test." This information is collected as input data.

[1991] Step 2: Device collects and sends information

[1992] The terminal internally organizes the data entered by the user and converts it into JSON format, then uses a REST API to send this information to the server, which uses an HTTP POST request to transfer the data to the server. The server receives this data and stores it in a database.

[1993] Step 3: The server analyzes the information

[1994] The server extracts the user's learning style and learning history from the database. It then analyzes the data using Python's Pandas module, for example. For example, it calculates and visualizes the distribution of past test scores and learning style trends. The input here is the data extracted from the database, and the output is the analysis results.

[1995] Step 4: The server invokes the generative AI model

[1996] The server sends a prompt to the generative AI model based on the analysis results. A specific prompt might be, "Generate the optimal set of math problems for a user who is good at repeated learning." The input is the analysis results and the prompt, and the output is the generated learning plan.

[1997] Step 5: Generative AI model generates a learning plan

[1998] The generative AI model generates an optimal lesson plan based on the prompt it receives, including a set of math problems and reading material, and outputs a JSON-formatted lesson plan that is sent back to the server.

[1999] Step 6: The server organizes and sends the lesson plan

[2000] The server parses the JSON-formatted learning plan returned by the generative AI model, formats it as needed, and then sends the data to the device using a REST API. The output is a curated learning plan.

[2001] Step 7: Your device will display what it learned

[2002] The device uses HTML, CSS, and JavaScript to display the learning content based on the received lesson plan. For example, it displays math problems or reading material on the screen. In this case, the input is the lesson plan, and the output is the learning content on the user interface.

[2003] Step 8: Users engage in learning

[2004] The user progresses with their learning based on the displayed learning content, for example, by answering math questions. The input is the learning content, and the output is the user's answer.

[2005] Step 9: User submits answer

[2006] The user enters the answer to the problem into the terminal, for example, by typing "801" on a keyboard or by using voice input to answer "234 plus 567 is 801." The input is the user's answer, and the output is the data entered into the terminal.

[2007] Step 10: The device sends the answer to the server

[2008] The terminal organizes the user's answers and sends them to the server using an HTTP POST request. The input is the user's answer data, and the output is the data transferred to the server.

[2009] Step 11: Server checks the answer

[2010] The server calls the generative AI model and checks whether the user's answer is correct. For example, it evaluates whether "234 + 567 is 801." The input is the user's answer and the model's correct answer data, and the output is the correctness of the answer.

[2011] Step 12: Server generates commentary

[2012] If the answer is incorrect, the server uses a generative AI model to generate a detailed explanation for the mistake, for example, creating a step-by-step explanation for arriving at the correct answer. The input is the user's answer data and the prompt text, and the output is the explanation text.

[2013] Step 13: The server sends the results and explanations to the device.

[2014] The server sends the answer's correctness and explanation to the terminal. The output is the correctness and explanation data.

[2015] Step 14: Your device will display the results and explanations.

[2016] The device displays the received results and explanations to the user, for example, a message such as "Your answer is correct!" or "You made a mistake here. Please refer to the next step." The input is the result and explanation data, and the output is the display on the user interface.

[2017] (Application example 1)

[2018] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2019] Existing learning systems have limitations in providing optimal learning plans that are tailored to each student's individual learning style and learning history. They also lack the ability to provide real-time learning support and feedback, resulting in insufficient support for students to learn efficiently. Furthermore, the methods of presenting learning content and providing feedback are inconsistent, resulting in reduced learning effectiveness. Therefore, there is a need for an effective learning system that meets the individual needs of each student.

[2020] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[2021] In this invention, the server includes means for collecting and storing information about the student's learning style and learning history, means for generating an optimal learning plan based on the collected learning style and learning history, means for presenting learning content to the student based on the generated learning plan, means for displaying the learning content and feedback using a glasses-type device worn by the student, and means for checking answers submitted by the student and providing explanations as necessary. This allows for the provision of a learning plan optimal for each student's individual learning style in real time, enabling efficient learning support.

[2022] "Student" refers to a learner who uses the learning system.

[2023] "Learning style" refers to the individual characteristics that indicate how students learn most effectively.

[2024] "Learning history" refers to a record of what a student has learned in the past and their grades.

[2025] "Collection and storage means" refers to methods and devices for capturing and storing information about students' learning styles and learning histories.

[2026] A "learning plan" refers to a plan that organizes learning content and methods in a way that is optimal for each student.

[2027] "Generating means" refers to a method or system that creates a learning plan based on the collected information.

[2028] "Presenting means" refers to a method or device for displaying the generated lesson plan to students.

[2029] "Eyeglasses-type device" refers to a wearable device that students wear and use to provide visual information.

[2030] "Feedback" refers to the evaluation and explanation given to students in response to their learning results and answers.

[2031] "Means of checking" refers to the method or device that evaluates the answers submitted by students and determines whether they are correct or incorrect.

[2032] "Means for providing explanations as needed" refers to a method or system that provides guidance or explanations to the correct answer when the answer is incorrect.

[2033] "Generative AI" refers to a system that uses artificial intelligence to generate new data and plans.

[2034] A "prompt sentence" refers to an input sentence that gives specific generation instructions to the generation AI.

[2035] The following system components and process steps are used to implement the invention.

[2036] System Components

[2037] 1. Eyeglass-type device worn by students

[2038] Hardware: Eyeglasses (e.g. Google Glass)

[2039] Software: Application for viewing learning content and receiving feedback

[2040] 2. Server

[2041] Hardware: High-performance server

[2042] Software: Database management systems (e.g., MySQL), generative AI (e.g., OpenAI GPT models), data analysis tools (e.g., Python-based analysis systems)

[2043] Process Overview

[2044] The server generates an optimal learning plan based on the student's learning style and learning history, and displays the learning content and feedback in real time on the glasses-type device worn by the student. Every time the student takes an action, that information is sent to the server, which analyzes the data and generates and provides the next appropriate learning content.

[2045] Detailed processing contents

[2046] 1. Information gathering

[2047] User: Students provide their learning style and learning history by speaking into the glasses, scanning a QR code, or manually entering information.

[2048] Device: The glasses send the collected information to a smartphone via Bluetooth, which then transmits it to a server via Wi-Fi.

[2049] 2. Data storage and analysis

[2050] Server: The server stores the received information in a database (MySQL) and uses data analysis tools to analyze learning styles and history.

[2051] 3. Create a learning plan

[2052] Server: Based on the analysis results, the server sends prompts to a generative AI model (e.g., OpenAI's GPT model) to generate an optimal learning plan. Example prompts:

[2053] Generate math problem sets suitable for middle school students who enjoy repetition.

[2054] 4. Presentation of learning content

[2055] Server: Sends the generated learning plan to the eyeglasses device.

[2056] Device: The glasses-type device displays the learning content according to the received learning plan.

[2057] 5. Providing Feedback

[2058] User: The student engages with the learning content and provides answers via voice or manual input into the glasses.

[2059] Device: The answer is sent to the server, where the generation AI determines whether it is correct or incorrect and generates feedback.

[2060] 6. Representation

[2061] Server: Sends feedback to the glasses-type device and prepares the next appropriate learning content.

[2062] Device: The glasses display feedback and new learnings.

[2063] Specific examples

[2064] For example, here is a scenario where a middle school student is taking extra math classes:

[2065] 1. Information gathering: Students speak into the glasses-like device, saying, "I want to solve a math problem."

[2066] 2. Data storage and analysis: The server receives the voice data and stores it in a database. The server analyzes the student's learning history and determines whether the student is a "junior high school student who excels at repeated learning."

[2067] 3. Learning plan generation: The server sends the following prompt to the generative AI model:

[2068] Generate math problem sets suitable for middle school students who enjoy repetition.

[2069] The generated problem set (e.g., "What is 234 + 567?") is sent to the eyeglass-type device.

[2070] 4. Presentation of learning content: The eyeglasses display "What is 234 + 567?"

[2071] 5. Providing feedback: The student types "801" by voice. The server uses generative AI to determine whether the answer is correct or not, and provides feedback saying, "That's correct!"

[2072] 6. Re-presentation: The server generates the next question and sends it to the glasses-type device. The learning content is displayed again.

[2073] This allows us to provide a learning experience that best suits each student's individual learning style.

[2074] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[2075] Step 1:

[2076] Information collection and transmission

[2077] User: Students provide their learning style and learning history by speaking into the glasses, scanning a QR code, or manually entering their learning history.

[2078] Input: Learning style characteristics and learning history (e.g., you are good at repeating things, you got 80 points on the last test, etc.).

[2079] Output: Audio data and manual input data.

[2080] How it works: The glasses send the collected information to a smartphone via Bluetooth, and then to a server via Wi-Fi.

[2081] Step 2:

[2082] Data storage and analysis

[2083] Server: The server stores the received information in a database (e.g., MySQL) and uses a data analysis tool (e.g., a Python-based analysis system) to analyze learning styles and histories.

[2084] Input: Voice data and manual input data.

[2085] Output: Analysis results based on learning style and history.

[2086] Specific operation: After storing the information in the database, the server analyzes the learning style and history and generates prompts for the generative AI model.

[2087] Step 3:

[2088] Generate a learning plan

[2089] Server: Based on the analysis results, it sends prompts to a generative AI model (e.g., OpenAI's GPT model) to generate an optimal learning plan.

[2090] Input: Analysis results based on learning style and history, prompt statement.

[2091] Output: The generated learning plan.

[2092] Specific operation: The server sends the following prompt to the generative AI model: "Please generate a set of math problems suitable for junior high school students who are good at repetitive learning." The generative AI then generates a study plan and sends it back to the server.

[2093] Step 4:

[2094] Presentation of learning content

[2095] Server and device: The server sends the generated learning plan to the eyeglass device, which then displays the learning content to the student.

[2096] Input: The generated learning plan.

[2097] Output: Learning content displayed on the eyeglasses device.

[2098] Specific operation: The glasses-type device presents students with questions such as "What is 234 + 567?"

[2099] Step 5:

[2100] Enter and submit your answers

[2101] User: The student engages with the learning content and provides answers via voice or manual input into the glasses.

[2102] Input: Student's answer (e.g., "801").

[2103] Output: Audio data or manually input data.

[2104] How it works: The glasses send students' responses via Bluetooth to a smartphone, which then transmits them to a server via Wi-Fi.

[2105] Step 6:

[2106] Checking answers and generating feedback

[2107] Server: Checks the received answers using the generative AI model and generates feedback (explanation) if necessary.

[2108] Input: Student answers.

[2109] Output: Correct / incorrect judgment and feedback (explanation).

[2110] What it does: The server matches the student's answer with the correct answer and uses a generative AI model to generate feedback, such as "That's right!" or an explanation if the answer is incorrect.

[2111] Step 7:

[2112] Providing feedback and next steps

[2113] Server and device: The server sends the generated feedback to the eyeglasses device, which prepares the next appropriate learning content. The eyeglasses device displays the feedback and new learning content to the student.

[2114] Input: Feedback and next learning plan.

[2115] Output: Feedback and next steps displayed on the glasses.

[2116] What it does: The glasses provide feedback and next steps to students, providing continuous support for their learning.

[2117] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[2118] This invention is a learning system that recognizes a student's learning style, learning history, and emotions and provides an optimal learning plan. The program processing of this system will be specifically explained below.

[2119] Collecting student information

[2120] 1. User (student) enters information

[2121] The user inputs information about their learning style and learning history into the terminal, such as "I'm good at repeating my learning" or "I got 80 points on the last test."

[2122] 2. The device collects and transmits information

[2123] The device collects the information entered by the user and sends it to the server, including the student ID, grade, learning style, and learning history.

[2124] 3. The server stores the information

[2125] The server stores the received information in a database, assigning an ID to each student's information to uniquely identify them.

[2126] Emotion recognition

[2127] 1. Real-time video and audio input by users

[2128] While studying, users input video and audio into the device in real time using a webcam and microphone.

[2129] 2. The device performs emotion analysis

[2130] The device sends the collected video and audio data to an emotion engine that recognizes the student's emotions, such as "concentrating," "tired," or "enjoying" in real time.

[2131] 3. Sending Emotional Data

[2132] The analyzed emotional data is sent from the device to a server and used when generating a learning plan.

[2133] Suggestions for learning content and methods

[2134] 1. The server analyzes the information

[2135] The server extracts information from the database about students' learning styles and past learning histories, as well as emotional data collected in real time, and analyzes this information.

[2136] 2. The server calls the generated AI

[2137] The server calls the AI ​​generator and instructs it to generate an optimal learning plan based on the analysis results, for example, by issuing specific instructions such as "generate math problems for students who are good at repetitive learning and are in a focused state."

[2138] 3. Generative AI generates a learning plan

[2139] The AI ​​generates optimal learning plans based on a student's learning style and emotions. For example, if a student is concentrating, it will increase the difficulty of the questions slightly, and if it determines that the student is tired, it will generate a plan that includes instructions to take a break.

[2140] 4. The server organizes and sends the study plan

[2141] The server organizes the learning plan received from the generation AI and sends it to the device in the appropriate format.

[2142] Presentation of learning content

[2143] 1. The server sends the study plan to the device

[2144] The server then sends the generated study plan to the device, which includes questions, study materials, and a study schedule.

[2145] 2. Your device will display your study plan

[2146] The device then displays the received study plan to the user, for example, displaying math practice problems on the screen and providing dynamic instructions such as "concentrate" and "take breaks as you progress."

[2147] 3. Users engage in learning

[2148] The user studies based on the presented study plan, either by typing answers into the device or by submitting answers aloud via audio glasses.

[2149] Check answers and explain

[2150] 1. User submits answer

[2151] The user submits the answer to the problem by typing it into the terminal or by voice, and the answer is registered in the terminal.

[2152] 2. The device sends the answer to the server

[2153] The device sends the user's response to the server, along with their emotional data if necessary.

[2154] 3. The server checks the answer

[2155] The server calls the generation AI and checks the answer submitted by the user against the correct answer, for example, checking whether '801' is the correct answer to 234 + 567.

[2156] 4. The server generates the description

[2157] If the user's answer is incorrect, the server uses a generative AI to create an explanation including the reason and the correct solution. It also provides feedback according to the user's emotional state.

[2158] 5. The server sends the results and explanations to the device.

[2159] The server checks the answers and sends them to the device along with explanations, including reasons for the correct answers, additional practice questions, and even encouragement and advice based on the user's emotional state.

[2160] 6. The device displays the results and explanations

[2161] The device displays the results and explanations to the student. Specifically, it displays messages such as "That's right! Well done!" or "That's wrong. The correct answer is '801'" on the screen, and also provides feedback that takes emotions into account.

[2162] Specific examples

[2163] For example, the processing flow when Tanaka, a fifth-grader, solves a math problem is shown below.

[2164] Step 1:

[2165] Tanaka types into the terminal, "I'm good at repeated learning and got 80 points on the last test."

[2166] Step 2:

[2167] Video and audio data collected in real time was analyzed to determine that Tanaka was "concentrating."

[2168] Step 3:

[2169] The device collects Tanaka's information and emotional data and sends it to the server.

[2170] Step 4:

[2171] The server stores the information in a database and calls the generation AI, instructing it to generate a "math problem set suitable for Tanaka, a student who excels at repeated learning, and who is currently concentrating."

[2172] Step 5:

[2173] The generation AI generates a plan and sends it back to the server.

[2174] Step 6:

[2175] The server organizes the learning plan and sends it to the device, displaying a question such as "What is 234 + 567?"

[2176] Step 7:

[2177] Tanaka responds by entering "801." The device then sends the response and emotion data to the server.

[2178] Step 8:

[2179] The server checks the answer using the generation AI and determines it to be correct.

[2180] Step 9:

[2181] The server sends the result to the device, which then displays to Tanaka, "That's correct! Maintain your current concentration."

[2182] This allows the learning system to provide optimal learning plans and feedback tailored to each student's learning needs, promoting efficient and effective learning.

[2183] The processing flow will be explained below.

[2184] Program processing flow

[2185] Collecting student information

[2186] Step 1:

[2187] The user (student) inputs information about their learning style and learning history into the terminal. Specifically, they input information such as "I'm good at repeating my learning" or "I got 80 points on the last test."

[2188] Step 2:

[2189] The device collects the information entered by the user and sends it to the server, including the student ID, grade, learning style, and learning history.

[2190] Step 3:

[2191] The server stores the received information in a database, assigning an ID to each student's information to uniquely identify them.

[2192] Emotion recognition

[2193] Step 1:

[2194] While learning, users input video and audio into the device in real time, and the device uses a webcam and microphone to collect the user's facial expressions and tone of voice.

[2195] Step 2:

[2196] The device sends the collected video and audio data to the emotion engine, which analyzes this data and recognizes the user's emotions, such as "concentrated," "tired," or "enjoyed."

[2197] Step 3:

[2198] The emotion data analyzed by the emotion engine is sent to the server along with the user's learning style and learning history.

[2199] Suggestions for learning content and methods

[2200] Step 1:

[2201] The server extracts the student's learning style and past learning history from the database, as well as emotional data collected in real time, and analyzes this information.

[2202] Step 2:

[2203] The server calls the AI ​​and instructs it to generate an optimal learning plan based on the extracted data (learning style, learning history, emotional data).For example, it gives specific instructions such as, "Generate math problems for students who are good at repetitive learning and are in a focused state."

[2204] Step 3:

[2205] The generative AI generates optimal study plans based on a student's learning style and emotions. For example, if a student is concentrating, it might increase the difficulty of the questions slightly, or if it detects fatigue, it might provide a plan that includes instructions to take a break.

[2206] Step 4:

[2207] The server receives the generated lesson plan, organizes it, and sends it to the device in the appropriate format.

[2208] Presentation of learning content

[2209] Step 1:

[2210] The server then sends the generated study plan to the device, which includes questions, study materials, and a study schedule.

[2211] Step 2:

[2212] The device then displays the received study plan to the user, for example, displaying math exercises on the screen and providing dynamic instructions such as "stay focused" or "you look tired, take a break."

[2213] Step 3:

[2214] The user studies based on the presented study plan, either by typing answers into the device or by submitting answers aloud via audio glasses.

[2215] Check answers and explain

[2216] Step 1:

[2217] After solving the problem, the user submits the answer by typing it into the terminal or by voice, and the answer is registered in the terminal.

[2218] Step 2:

[2219] The device sends the user's response to the server, along with their emotional data if necessary.

[2220] Step 3:

[2221] The server calls the generation AI and checks the answer submitted by the user against the correct answer, for example, checking whether '801' is the correct answer to 234 + 567.

[2222] Step 4:

[2223] The server determines whether the answer is correct, and if it is incorrect, the generative AI creates an explanation including the reason and the correct solution. It also provides feedback based on the user's emotional state.

[2224] Step 5:

[2225] The server checks the answers and sends them to the device along with explanations, including reasons for the correct answers, additional practice questions, and even encouragement and advice based on the user's emotional state.

[2226] Step 6:

[2227] The device displays the results and explanations to the student, such as "That's right! Well done!" or "That's wrong. The correct answer is '801'," and also provides emotional feedback.

[2228] Specific examples

[2229] For example, the processing flow when Tanaka, a fifth-grader, solves a math problem is shown below.

[2230] Step 1:

[2231] Tanaka types into the terminal, "I'm good at repeated learning and got 80 points on the last test."

[2232] Step 2:

[2233] Video and audio data collected in real time was analyzed to determine that Tanaka was "concentrating."

[2234] Step 3:

[2235] The device collects Tanaka's information and emotional data and sends it to the server.

[2236] Step 4:

[2237] The server stores the information in a database and calls the generation AI, instructing it to generate a "math problem set suitable for Tanaka, a student who excels at repeated learning, and who is currently concentrating."

[2238] Step 5:

[2239] The generation AI generates a plan and sends it back to the server.

[2240] Step 6:

[2241] The server organizes the study plan and sends it to the device, which then displays the question "What is 234 + 567?" to Tanaka.

[2242] Step 7:

[2243] Tanaka responds by entering "801." The device then sends the response and emotion data to the server.

[2244] Step 8:

[2245] The server checks the answer using the generation AI, and if it is determined to be correct, it sends the result to the device.

[2246] Step 9:

[2247] The device displays to Tanaka, "That's correct! Maintain your current concentration."

[2248] This allows the learning system to provide optimal learning plans and feedback tailored to each student's learning needs, promoting efficient and effective learning.

[2249] Example 2

[2250] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2251] Modern learning systems are expected to provide individualized learning plans based on students' learning styles and learning histories. However, traditional systems only use static data and are unable to adjust learning plans to take students' real-time emotions into account. This makes it difficult to provide dynamic learning plans that optimally maintain students' focus and motivation.

[2252] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[2253] In this invention, the server includes means for collecting and storing information on students' learning styles and learning histories, means for generating optimal learning plans based on the collected learning styles and learning histories, means for collecting real-time video and audio data, performing emotion analysis, and storing the data, and means for adjusting and generating learning plans based on the analyzed emotion data, thereby making it possible to provide learning plans that take into account the dynamic emotional states of students.

[2254] "Student learning style" refers to the individual ...

Claims

1. A means of collecting and storing information about students' learning styles and learning histories; A means for generating an optimal learning plan based on the collected learning style and learning history; A means for presenting learning content to students based on the generated learning plan; A means of checking answers submitted by students and providing explanations where necessary; A learning system including.

2. 2. The learning system according to claim 1, further comprising means for generating a learning plan by calling a generation AI for generating an optimal learning plan.

3. 2. The learning system according to claim 1, further comprising means for analyzing answers using image information to check answers to questions presented to students.

Images