System

Abstract

A system is provided.SOLUTION: A system comprising: means for collecting student test results and lesson outputs in order to generate student specific learning data; means for analyzing the learning data and grasping the degree of understanding and learning progress for each student; means for generating learning contents optimal for the student based on the generated learning data; means for providing the learning contents to the student in real time; and means for the teacher to confirm the progress and degree of understanding of the student.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In traditional educational settings, teachers have to teach large numbers of students at the same time, making it difficult to respond to each student's level of understanding and learning progress individually. As a result, some students are unable to keep up with the lessons, while others become bored because the material is too easy, making it difficult to provide an appropriate education to all students. In addition, there is also the issue that understanding students' learning situations and levels of understanding places a heavy burden on teachers, making it difficult to provide efficient instruction. [Means for solving the problem]

[0005] To solve the above problems, the present invention provides the following means: A means for collecting students' test results and lesson output is provided to generate individual student learning data. A means for analyzing the learning data and grasping each student's level of understanding and learning progress is then provided. A means for generating optimal learning content for each student from the generated learning data based on the results of this analysis is provided. Furthermore, a means for providing the learning content to students in real time and a means for responding to questions and requests from students in real time are provided, and a means for teachers to check students' progress and level of understanding is added. This configuration provides an optimal learning environment for each student, reducing the burden on teachers and enabling effective instruction.

[0006] "Individual student learning data" is data generated based on each student's level of understanding and learning progress, and forms a unique learning profile for each student.

[0007] "Means for collecting learning data" refers to a device or system that digitally collects the results and content of students' tests and class output when they submit them.

[0008] "Data analysis means" means an algorithm or system that analyzes a student's level of understanding and learning progress based on collected learning data and generates an individual learning profile.

[0009] "Learning content generation means" refers to a system or module for designing and generating optimal learning content for students based on analyzed learning data.

[0010] "Real-time delivery means" means a communication or display device that instantly delivers generated learning content and feedback to students.

[0011] "Real-time feedback means" refers to a system or module that instantly analyzes questions or requests from students and generates appropriate answers or additional assignments in response.

[0012] "Teacher confirmation means" refers to a dashboard or display device that allows teachers to efficiently check and understand students' learning progress and level of understanding. [Brief explanation of the drawings]

[0013] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14]FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0014] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0015] First, the terms used in the following description will be explained.

[0016] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0017] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0018] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0019] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0020] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0021] [First embodiment]

[0022] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0023] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0024] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0025] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0026] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0027] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0028] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0029] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0030] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0031] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0032] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0033] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0034] This invention relates to a learning system that uses generative AI to provide optimal learning content for each student. Specifically, it describes a method that incorporates students' test results and class output, generates and analyzes personal learning data, and provides learning content in real time based on that data.

[0035] Data collection and storage

[0036] Devices: When students take tests or submit assignments during class, their devices (PCs, tablets, etc.) collect this data and send it to the server.

[0037] Server: Stores received test results and lesson output in a database. For example, it stores students' test scores and answers, as well as images and text data of submitted reports and assignments.

[0038] Data analysis

[0039] Server: The analysis module is launched periodically or when new data is added. The server retrieves test results and lesson outputs for each student in the database and analyzes them.

[0040] Example: Analyze the accuracy rate, answer time, and patterns of incorrect answers of math problems solved by Student A to generate a learning profile. For example, if it turns out that the accuracy rate for a specific type of problem is low, provide learning content that reinforces that type of problem.

[0041] Generating learning content

[0042] Server: Based on the generated learning profile, the server uses generative AI to generate optimal learning content for each student. The content can come in a variety of formats, including text, images, videos, and interactive questions.

[0043] Example: For a type of math problem that Student A struggles with, we generate related explanatory videos and practice problems and provide them as a single learning package.

[0044] Real-time delivery and feedback

[0045] Server: The generated content is sent to students' devices in real time, allowing them to view the content on their own devices and progress with their studies.

[0046] Device: If a student has a question while studying, they can immediately input it into the AI. For example, they can ask, "Why is the answer to this problem like this?"

[0047] Server: Receives questions, analyzes them in real time, generates appropriate answers and additional questions, and sends them to students' devices. Students receive these and continue their learning.

[0048] Teacher Support and Monitoring

[0049] User (Teacher): Teachers access the server using a dedicated device or management app.

[0050] Server: Provides a dashboard for teachers, displaying each student's learning progress and level of understanding, allowing teachers to see at a glance how each student is learning.

[0051] Example: A teacher determines that Student A's learning progress is falling behind and plans additional follow-up lessons. If necessary, individual supplementary learning materials can be prepared and provided to the student via the server.

[0052] This will provide each student with the optimal learning environment, enabling high-quality education while reducing the workload of teachers.

[0053] The processing flow will be explained below.

[0054] Step 1:

[0055] User (Teacher): Uploads tests and supplementary materials to be used in class to the server.

[0056] Server: Organizes the uploaded data and links it to the corresponding lesson plans.

[0057] Device: Prepares students to receive tests and supplementary materials.

[0058] Step 2:

[0059] User (student): Takes a test or submits an assignment during class.

[0060] Device: Collects student answers and submissions and sends them to the server in real time. Records student answers digitally.

[0061] Server: Receives the transmitted data and immediately stores it in the database. Backs up the stored data.

[0062] Step 3:

[0063] Server: Launches the data analysis module periodically or when new student data is added.

[0064] Server: Retrieves student test results and lesson output from the database.

[0065] Server: Analyzes the acquired data and calculates each student's level of understanding, learning progress, and patterns of incorrect answers.

[0066] Server: Based on the analysis results, personal learning data is generated and the learning profile for each student is updated.

[0067] Step 4:

[0068] Server: Based on the updated personal learning data, the generative AI module is launched.

[0069] Server: Selects and combines resources (text, images, videos, etc.) containing learning content appropriate for each student.

[0070] Server: Sends the generated learning content to students' devices in real time.

[0071] Device: Display received learning content and make it accessible to students.

[0072] Step 5:

[0073] User (student): If a question arises during learning, the user sends the question to the AI ​​from their device.

[0074] Terminal: Sends questions from students to the server in real time.

[0075] Server: Receives the question and starts the process of analyzing it and generating the best answer.

[0076] Server: Sends generated answers and additional assignments to student devices.

[0077] Device: Show students answers and assignments so they can continue learning.

[0078] Step 6:

[0079] User (Teacher): Logs in to a dedicated device or management app and accesses the server.

[0080] Server: Authenticates the teacher access request and grants access to the Teacher Dashboard.

[0081] Server: Retrieves student learning data, progress, and understanding from the database and displays them on a dashboard.

[0082] User (Teacher): Check the dashboard to understand each student's understanding and progress.

[0083] User (Teacher): Sees that Student A is falling behind and acts on the data to plan make-up lessons and additional assignments.

[0084] Example 1

[0085] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0086] With conventional learning systems, it was difficult to provide individualized learning content that corresponded to each student's level of understanding and learning progress. It also required a great deal of effort for teachers to grasp each student's progress and level of understanding, making it difficult to provide appropriate guidance and feedback to each student in real time. Furthermore, there was a lack of a mechanism for immediate response when students had questions during their studies.

[0087] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0088] In this invention, the server includes means for collecting assessment results and lesson outcomes to generate individual student learning data, means for analyzing the learning data and grasping each student's level of understanding and learning progress, means for generating optimal learning content for each student based on the generated learning data using a generative AI model, means for providing the learning content to students in real time, means for responding to questions and requests from students in real time, means for teachers to check students' progress and level of understanding, and means for teachers to plan additional learning support and provide supplementary materials to students. This makes it possible to provide learning content appropriate for each student, resolve questions in real time, and provide effective educational support while reducing the burden on teachers.

[0089] "Individual student learning data" refers to digital data generated by each student through lessons and assessment activities, and is information that indicates the student's level of understanding and progress.

[0090] "Evaluation results" refers to the scores and evaluation content obtained through learning assessment methods such as tests and assignments.

[0091] "Class outcomes" refers to the results of learning activities that students undertake during class or extracurricular activities, and include a variety of formats such as text, images, and videos.

[0092] "Analysis" refers to the process of evaluating students' learning patterns and comprehension based on collected data and creating individual profiles.

[0093] "Understanding" refers to an indicator of how well a student understands a particular learning content.

[0094] "Learning progress" refers to an indicator that shows how far a student is progressing along the learning curriculum.

[0095] A "generative AI model" refers to an algorithm or model that uses AI technology to automatically generate specific learning content based on input data.

[0096] "Learning content" refers to educational content provided for students to learn, and includes various formats such as text, images, videos, and interactive questions.

[0097] "Real-time" refers to a state in which data is exchanged and processed immediately and is available without waiting.

[0098] "Teacher burden" refers to the time and effort required for teachers to check students' progress, provide feedback, and provide individual instruction.

[0099] "Instructional aids" refer to activities and tools that support the learning process and enable students to learn effectively.

[0100] "Supplemental materials" refers to additional learning materials or assignments provided to deepen students' understanding.

[0101] This invention relates to a learning system that uses a generative AI model to provide optimal learning content for each student. Specifically, it shows a method for incorporating student assessment results and class performance, generating and analyzing personalized learning data, and providing learning content in real time based on that data.

[0102] Data collection and storage

[0103] Device:

[0104] When students take tests or submit assignments during class, devices such as computers and tablets collect this data and send it to a server. For example, students answer test questions and save the results on their devices. The collected data is then sent to a server.

[0105] server:

[0106] The received evaluation results and lesson outcomes are stored in a database. Specifically, data such as student IDs, test scores, answers, and submitted reports are stored. After the data is stored in the database, the conditions for launching the analysis module are checked.

[0107] Data analysis

[0108] server:

[0109] When new data is added, the analysis module is triggered. The server collects data for each student in the database and passes it to the analysis engine. For example, analyzing math test results, it finds out that Student A has a low success rate in certain question types and generates a learning profile.

[0110] Generating learning content

[0111] server:

[0112] Based on the generated learning profile, specific learning content is input as a prompt to the generative AI model. For example, a prompt such as "Please generate explanatory videos and practice problems for math problem type X for Student A" is provided to the generative AI model. The text, images, videos, problem sets, etc. obtained from the generative AI model are compiled into a single learning package, including related explanatory videos and explanations of the answers to the problems.

[0113] As a specific example, if Student A has difficulty with a particular type of math problem, the generative AI model will generate explanatory videos and practice problems based on the prompt text and provide them as a learning package.

[0114] Example prompt sentence:

[0115] Please generate explanatory videos and practice questions for math problem type X based on the test results of Student A. I would like the format to be a combination of text, images, and videos.

[0116] Real-time delivery and feedback

[0117] server:

[0118] The generated learning content is sent to students' devices in real time. Data communication protocols are used to ensure reliable data delivery. Students can then check the content on their own devices and continue their learning.

[0119] Device:

[0120] If a student has a question while studying, they can immediately input it into the AI, for example, "Why is the answer to this problem like this?" The question is then sent to the server.

[0121] server:

[0122] Analyze the question and generate appropriate answers or provide additional practice questions, possibly using a generative AI model, and send the answers or additional questions to the student's device.

[0123] Teacher Support and Monitoring

[0124] User (Teacher):

[0125] Teachers access the server using dedicated devices or a management app. They can use a dashboard to visually monitor each student's progress and understanding. Teachers can use this information to plan additional lessons or provide supplementary materials for specific students.

[0126] server:

[0127] The teacher dashboard displays learning data for each student and makes it easily accessible to teachers. For example, areas of low understanding are visually highlighted. Based on this information, teachers can input follow-up instructions into the server and send them to students' devices.

[0128] This system provides each student with an optimal learning environment, enables real-time question resolution, and provides effective educational support while reducing the burden on teachers.

[0129] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0130] Step 1: Collect data

[0131] Input: Students use their devices to take tests or submit assignments.

[0132] Specific operation: Students answer test questions and save their answers and assignment data on their devices. This includes text data, image data, video data, etc.

[0133] Output: The device sends the collected data to the server.

[0134] Step 2: Receiving and storing data

[0135] Input: Data such as test results and class results sent from the device.

[0136] Specific operation: The server receives the student's evaluation results and lesson results and stores them in a database, including student IDs, test scores, answers, reports, etc.

[0137] Output: Data per student stored in a database.

[0138] Step 3: Analyze the data

[0139] Input: New student data stored in the database.

[0140] How it works: When the server detects new data, it launches the analysis module. The analysis engine collects data for each student in the database and evaluates the student's understanding and learning progress.

[0141] Output: The resulting learning profile.

[0142] Step 4: Generate a learning profile

[0143] Input: Analysis results for each student from the analysis module.

[0144] Specific operation: The server generates a learning profile and creates detailed learning data for each student, including their individual weaknesses and strengths, such as the percentage of correct answers to math problems, the time it takes to answer them, and error patterns.

[0145] Output: The generated learning profile.

[0146] Step 5: Generate learning content

[0147] Input: Learning profile.

[0148] Specific operation: The server inputs a prompt based on the learning profile into the generative AI model. For example, "Please generate an explanatory video and practice questions for math problem type X for Student A." The generative AI model analyzes the prompt and automatically generates appropriate learning content (text, images, videos, interactive questions, etc.).

[0149] Output: The generated learning content.

[0150] Step 6: Package your learning content

[0151] Input: Learning content from a generative AI model.

[0152] What it does: The server compiles the generated content into a learning package, including related instructional videos and explanations of the answers to the questions.

[0153] Output: Packaged learning content.

[0154] Step 7: Real-time serving

[0155] Input: Packaged learning content.

[0156] Specific operation: The server transmits the learning package to the student's device in real time, using a data communication protocol to ensure the data is delivered.

[0157] Output: The learning content displayed on the student's device.

[0158] Step 8: Answering student questions

[0159] Input: Questions from students during learning.

[0160] Specific operation: When a student has a question while studying, they type the question on their device and send it to the server. For example, "Why is the answer to this problem like this?"

[0161] Output: The question sent to the server.

[0162] Step 9: Parsing the question and generating an answer

[0163] Input: The student question sent to the server.

[0164] What it does: The server analyzes the question and generates appropriate answers or provides additional practice questions, possibly using a generative AI model.

[0165] Output: Generated answers and additional practice questions.

[0166] Step 10: Provide your answers

[0167] Input: Generated answers and additional practice questions.

[0168] Specific operation: The server sends answers and additional practice questions to the student's device, where the student receives them and uses them to help with their studies.

[0169] Output: Answers and exercises displayed on student devices.

[0170] Step 11: Teacher Support and Monitoring

[0171] Input: Access from a dedicated device or management app.

[0172] How it works: Teachers access the server and use a dedicated dashboard to check students' progress and understanding, which displays each student's learning profile and progress.

[0173] Output: Teacher progress review and lesson plan.

[0174] Step 12: Plan additional follow-up lessons

[0175] Input: Student progress data viewed through the dashboard.

[0176] What it does: Teachers plan additional lessons and supplementary materials for specific students through the dashboard, and follow-up instructions are entered into the server and sent to student devices.

[0177] Output: Follow-up instructions are sent to the student.

[0178] Through this series of steps, learning content tailored to each student is provided, questions are answered in real time, and teachers can provide effective educational support.

[0179] (Application example 1)

[0180] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0181] Conventional learning systems have difficulty providing optimal learning content in real time based on each student's level of understanding and learning progress. They also lack a system that can provide immediate answers when students have questions. Furthermore, there are insufficient means for teachers to grasp students' learning progress in detail and provide appropriate follow-up. As a result, an environment that maximizes the learning effect of each student in personalized education has not been established.

[0182] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0183] In this invention, the server includes: means for collecting students' test results and lesson output; means for analyzing the learning data and grasping each student's level of understanding and learning progress; means for generating learning content optimized for each student based on the generated learning data; means for providing the learning content to students in real time; means for responding to students' questions and requests in real time; means for teachers to check students' progress and understanding; means for connecting to smart devices used in brick-and-mortar educational facilities and providing learning content and feedback in real time; means for generating learning content using a generative AI model based on a specific learning profile; and means for generating prompt sentences to generate appropriate feedback on topics that the student struggles with. This enables the provision of optimal learning content for each student. Furthermore, real-time responses can immediately resolve students' questions and improve learning efficiency. Furthermore, providing teachers with a means for accurately grasping students' learning progress and providing effective follow-up can improve the quality of personalized education.

[0184] "Individual student learning data" refers to data on individual students' test results and class output.

[0185] "Learning data analysis" refers to the process of processing and analyzing collected data to understand each student's level of understanding and learning progress.

[0186] "Learning Content" refers to optimal learning materials provided to students using generative AI models, and may include formats such as text, images, and videos.

[0187] "Real-time delivery" refers to the instant delivery of generated learning content to students.

[0188] "Student Q&A" refers to the process of providing immediate and appropriate answers or additional learning content in response to student questions or requests.

[0189] "Teacher Monitoring Tools" refers to tools specifically designed for teachers to monitor and manage student progress and understanding.

[0190] "Smart device connectivity" refers to linking the system with devices such as smartphones, tablets, and smart glasses used in brick-and-mortar educational facilities.

[0191] A "generative AI model" refers to an artificial intelligence model that automatically generates optimal learning content and feedback based on input data.

[0192] A "prompt" is a sentence that constructs instructions or questions for a generative AI model based on a specific learning profile.

[0193]

[0194] This invention is a smart education support system that utilizes generative AI models to provide optimal learning content for individual students in brick-and-mortar educational facilities. A specific embodiment of this system is described below.

[0195] Hardware Configuration

[0196] It uses a server and smart devices such as smartphones, tablets, and smart glasses. The server has the central function of collecting, analyzing, generating, and distributing data. The smart devices are used by students and teachers as an interface to receive learning content and feedback in real time.

[0197] Software Configuration

[0198] It uses the Python programming language, the generative AI model API provided by OpenAI (registered trademark), and a database management system.

[0199] System action

[0200] The server collects test results and class output from students' devices and stores them in a database. The collected data is periodically analyzed on the server. As a result of the analysis, each student's level of understanding and learning progress is identified and an individual learning profile is generated.

[0201] Based on this learning profile, the server uses a generative AI model to generate optimal learning content. At this time, a prompt sentence is input to the AI ​​model. For example, the following prompt sentence is generated:

[0202] Example prompt sentence:

[0203] Based on the math test results, Student A's most common mistakes are solving equations and basic use of trigonometric functions. Create learning content to deepen their understanding of these topics.

[0204] The generated learning content is delivered to students' devices in real time. Students can view this content and continue their studies via smartphones, tablets, or smart glasses. If students have questions while studying, they can input them in real time, and the generative AI model will instantly generate an answer, which will then be delivered to the student's device.

[0205] Teachers can also use the system to access a dedicated dashboard that allows them to see students' progress and understanding in real time, allowing them to plan individual follow-ups and provide additional materials or assignments if necessary.

[0206] For example, if test analysis reveals that Student A is struggling with fraction calculations, the teacher can provide supplementary materials on fraction calculations. This system provides an optimal learning environment for each student, improving the quality of personalized education.

[0207] As described above, by implementing the present invention, it becomes possible to realize real-time individualized learning and teacher support, which was difficult with conventional systems.

[0208] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0209]

[0210] Step 1:

[0211] The server collects test results and class output (text data, image data, video data) from students' devices. This allows the collection of student learning data. The test results and class output data taken by students are used as input. As output, the data is stored in a database on the server.

[0212] Step 2:

[0213] The server periodically launches the data analysis module to retrieve the students' learning data stored in the database. The data analysis module analyzes this learning data to understand each student's level of understanding and learning progress. The collected learning data is used as input for this step. The output is a learning profile for each student.

[0214] Step 3:

[0215] The server creates a prompt to generate specific learning content based on the generated learning profile. For example, it generates a prompt such as, "Based on the results of the math test, the questions that Student A often gets wrong are: solving equations, basic use of trigonometric functions. Please generate learning content to deepen his understanding of these topics." The student's learning profile is used as input. The generated prompt is used as output.

[0216] Step 4:

[0217] The server sends the generated prompt sentences to a generative AI model, which then generates optimal learning content based on the prompt sentences. For example, videos and supplementary learning materials are generated to deepen understanding. The prompt sentences are used as input, and the learning content is generated as output.

[0218] Step 5:

[0219] The generated learning content is sent from the server to the student's device in real time. The student checks this content on their device and proceeds with their learning. The generated learning content is used as input for this step. The output is the content displayed on the student's device.

[0220] Step 6:

[0221] When a device receives a question or request from a student, the question or request is sent to the server. The server uses a generative AI model to generate an appropriate answer for the received question and sends it back to the student's device. The question or request from the student is used as input. The answer is displayed on the student's device as output.

[0222] Step 7:

[0223] Teachers access a dedicated dashboard to check students' learning progress and comprehension data provided by the server. If necessary, teachers create additional learning materials and assignments and provide them to students' devices via the server. Student learning profiles and progress data are used as input. Additional learning materials and assignments are provided to students as output.

[0224] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0225] This invention relates to a learning system that uses generative AI and an emotion engine to provide optimal learning content for each student. Specifically, it describes a method for incorporating students' test results, class output, and emotional data, generating and analyzing personalized learning data, and providing learning content in real time based on that data.

[0226] Data collection and storage

[0227] Devices: Students take tests and submit assignments during class. The devices (computers, tablets, etc.) collect this data and send it to the server.

[0228] Server: Stores received test results and lesson output in a database. For example, it stores students' test scores and answers, as well as images and text data of submitted reports and assignments.

[0229] Collecting Emotional Data

[0230] Device: The emotion engine analyzes the student's facial expression, tone of voice, and input content to generate emotion data. For example, it senses changes in facial expression and voice when a student is facing a problem.

[0231] Server: Receives emotion data and stores it in a database. Emotion data includes information such as "interested," "understanding," and "confused."

[0232] Data analysis

[0233] Server: Launches modules that periodically analyze test results, lesson output, and sentiment data.

[0234] Server: Retrieves necessary learning data from the database and analyzes each student's level of understanding, learning progress, and emotional patterns.

[0235] Server: Based on the analysis results, generate personal learning data and update each student's learning profile.

[0236] Generating learning content

[0237] Server: Based on the generated personal learning data, the generative AI module is launched.

[0238] Server: Selects appropriate learning content for each student and customizes it taking into account emotional data.

[0239] Server: Sends the generated learning content to students' devices in real time.

[0240] Device: Display received learning content and make it accessible to students.

[0241] Real-time delivery and feedback

[0242] User (student): If a question arises during learning, the user sends the question to the AI ​​from their device.

[0243] Terminal: Sends questions from students to the server in real time.

[0244] Server: Receives the question and starts the process of analyzing it and generating the best answer.

[0245] Server: Sends generated answers and additional assignments to student devices.

[0246] Device: Show students answers and assignments so they can continue learning.

[0247] Teacher Support and Monitoring

[0248] User (Teacher): Accesses the server using a dedicated device or management app.

[0249] Server: Authenticates the teacher access request and grants access to the Teacher Dashboard.

[0250] Server: Retrieves student learning data, progress, comprehension, and emotional data from the database and displays them on a dashboard.

[0251] Users (Teachers): Check the dashboard to understand individual students' understanding, progress, and emotional changes.

[0252] User (Teacher): If the teacher determines that a student's progress is falling behind, he / she plans supplementary lessons and additional assignments. If necessary, he / she prepares individual supplementary materials and provides them to the student via the server.

[0253] Specific examples

[0254] Example 1: A student takes a test

[0255] 1. User (student): Take a test during class.

[0256] 2. Terminal: Records students' answers and transmits them to the server in real time.

[0257] 3. Server: Receives and stores data.

[0258] 4. Server: The emotion engine analyzes the student's facial expressions and tone of voice to generate emotion data during the test.

[0259] 5. Server: Analyzes the test results and emotion data and updates the personal training data.

[0260] Example 2: When a student asks an AI a question

[0261] 1. User (student): If a question arises during learning, the user sends the question to the AI ​​from their device.

[0262] 2. Terminal: Sends questions from students to the server in real time.

[0263] 3. Server: Receives the question and starts the process of analyzing it and generating the best answer.

[0264] 4. Server: Sends the generated answers to the students' devices.

[0265] 5. Device: Show students their answers and continue learning.

[0266] Teachers checking student progress

[0267] 1. User (teacher): Log in to a dedicated device or the management app and access the server.

[0268] 2. Server: Authenticates the teacher access request and grants access to the Teacher Dashboard.

[0269] 3. Server: Retrieves students' learning data, progress, and emotional data from the database and displays them on the dashboard.

[0270] 4. Users (Teachers): Check the dashboard to understand individual students' understanding, progress, and emotional changes.

[0271] 5. User (Teacher): If you notice that a student is falling behind, plan a make-up lesson or additional assignment.

[0272] The processing flow will be explained below.

[0273] Step 1:

[0274] User (Teacher): Uploads tests and supplementary materials to be used in class to the server. Materials include workbooks, videos, slides, etc.

[0275] Server: Organizes uploaded data by format and links it to the corresponding lesson plans.

[0276] Devices: Student devices are prepared and configured to access materials.

[0277] Step 2:

[0278] User (student): Takes tests or submits assignments during class. Study the designated material on the device.

[0279] Terminal: Collects student answers and submissions (text, images, audio, etc.) and sends them to the server in real time.

[0280] Server: Receives the transmitted data and immediately stores it in a database, along with recording the timestamp information.

[0281] Step 3:

[0282] Devices: Data is collected using sensors that detect students' facial expressions and tone of voice, for example, cameras and microphones that analyze real-time video and audio.

[0283] Server: Analyzes facial expressions, tone of voice, and input content collected by the emotion engine to generate emotional data.

[0284] Server: Receives emotional data and stores it in a database. The emotional data includes students' reactions (happiness, confusion, etc.).

[0285] Step 4:

[0286] Server: Launches the data analysis module periodically or when new data is added.

[0287] Server: Retrieves students' test results, lesson output, and emotion data from the database.

[0288] Server: Comprehensively analyzes the acquired data and calculates each student's level of understanding, learning progress, and emotional patterns. For example, it evaluates the student's level of concentration when working on a specific task.

[0289] Server: Based on the analysis results, generate personal learning data and update each student's learning profile.

[0290] Step 5:

[0291] Server: Based on the updated personal learning data, the generative AI module is launched.

[0292] Server: Selects appropriate learning content for each student and customizes it taking into account emotional data, for example, including explanatory videos or additional practice questions for difficult assignments.

[0293] Server: Sends the generated learning content to students' devices in real time.

[0294] Device: Display received learning content and make it accessible to students.

[0295] Step 6:

[0296] User (student): If a question arises during learning, the user sends the question to the AI ​​from their device.

[0297] Terminal: Sends questions from students to the server in real time.

[0298] Server: Receives the question and starts the process of analyzing it and generating the best answer.

[0299] Server: Sends generated answers and additional assignments to student devices.

[0300] Device: Show students their answers and assignments and keep them learning.

[0301] Step 7:

[0302] User (Teacher): Accesses the server using a dedicated device or management app.

[0303] Server: Authenticates the teacher access request and grants access to the Teacher Dashboard.

[0304] Server: Retrieves student learning data, progress, comprehension, and emotion data from the database and displays them on a dashboard.

[0305] Users (Teachers): Check the dashboard to understand individual students' understanding, progress, and emotional changes.

[0306] User (Teacher): If it is determined that Student A's progress is falling behind, the teacher plans supplementary lessons and additional assignments. If necessary, the teacher conducts individual follow-up based on the emotional data.

[0307] This configuration allows a learning system incorporating an emotion engine to comprehensively evaluate not only a student's level of understanding and progress, but also their emotional changes, and provide optimal learning content in real time. This also makes it easier for teachers to grasp the psychological state of their students, enabling more detailed instruction.

[0308] Example 2

[0309] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0310] Conventional learning systems have the problem that it is difficult to accurately grasp each student's level of understanding and learning progress, and they can only provide uniform learning content. Furthermore, they are unable to provide learning content that takes into account students' emotional data, which affects students' motivation and learning efficiency. As a result, they are unable to provide an optimal learning experience for each student, and there are issues with not being able to provide effective learning support.

[0311] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0312] In this invention, the server includes means for collecting students' test results and lesson output to generate individual student learning data, means for analyzing the learning data and students' emotional data to grasp each student's level of understanding, learning progress, and emotional patterns, and means for generating optimal learning content for each student based on the generated learning data and emotional data. This makes it possible to provide optimal learning content based not only on each student's level of understanding and learning progress, but also on their emotional patterns.

[0313] "Individual student learning data" refers to data that records in detail each student's learning situation, achievements, and level of understanding.

[0314] "Test results" are data that indicate the scores and answers given by students on tests they have taken.

[0315] "Class output" refers to data that includes assignments, reports, and notes that students submit during and after class.

[0316] "Emotional data" is data about emotions obtained by analyzing students' facial expressions, tone of voice, content of speech, etc.

[0317] A "learning profile" is individual data that compiles each student's level of understanding, learning progress, emotional patterns, etc.

[0318] A "generative AI module" is a program or system that uses artificial intelligence technology to generate optimal learning content for students.

[0319] "Learning content" refers to the learning materials and assignments provided to students for learning, and includes various formats such as text, images, and videos.

[0320] The "Teacher Dashboard" is a management screen that allows teachers to check students' learning status, level of understanding, progress, and emotional patterns in real time.

[0321] "Providing in real time" means providing learning content and feedback instantly in line with the current learning environment and situation.

[0322] "Feedback" is information that includes evaluations of students' learning activities, advice, and areas for improvement.

[0323] "Additional teaching materials and assignments" refers to new learning materials and assignments that teachers add to reinforce what students have learned.

[0324] MODE FOR CARRYING OUT THE INVENTION

[0325] This invention is a learning system that provides optimal learning content to students, utilizing a generative AI model and an emotion engine. The system collects students' test results, class output, and emotion data, generates and analyzes individual learning data, and provides learning content in real time based on that data.

[0326] Hardware and software used

[0327] Device:

[0328] Student computers or tablets

[0329] Camera and microphone (for collecting emotional data)

[0330] server:

[0331] Database management system (e.g., MySQL (registered trademark), PostgreSQL)

[0332] Sentiment analysis API (e.g., Microsoft® Emotion API, Google® Cloud Vision)

[0333] Generative AI modules (e.g., OpenAI GPT-4 (registered trademark), Google BERT)

[0334] System Description

[0335] Data collection and storage

[0336] Device: Students take tests and submit assignments during class. The device collects this data (test answers and assignment images / text data) and sends it to the server.

[0337] Server: Stores the received data in a database. Collected data includes students' test scores, answers, submitted papers, and images and text of assignments.

[0338] Specific examples of behavior:

[0339] Students take math tests and upload the results from their devices to the server.

[0340] The server stores the received data in a database for subsequent analysis.

[0341] Collecting Emotional Data

[0342] Device: The emotion engine analyzes the student's facial expressions and tone of voice. For example, emotion data is generated from changes in the student's facial expressions and tone of voice.

[0343] Server: Receives the analyzed emotion data and stores it in a database. The emotion data includes information such as "interested," "understanding," and "confused."

[0344] Specific examples of behavior:

[0345] If a student shows a confused expression during a test, the emotion engine analyzes the facial expression data and generates emotion data for "confusion."

[0346] The generated emotion data is sent to a server and stored in a database.

[0347] Data analysis

[0348] Server: Launches a module that periodically analyzes test results, lesson output, and emotional data. It retrieves the necessary data from the database and analyzes each student's level of understanding, learning progress, and emotional patterns.

[0349] Server: Based on the analysis results, generate personal learning data and update the learning profile.

[0350] Specific examples of behavior:

[0351] Student A's level of understanding is evaluated based on his / her test results, and his / her learning profile is updated by integrating it with lesson output and emotional data.

[0352] Generating learning content

[0353] Server: Based on the generated personal learning data, a generation AI module is used to generate optimal learning content for students.

[0354] Server: Selects appropriate learning content for each student and customizes it taking into account emotional data.

[0355] Server: Sends the generated learning content to students' devices in real time, and the devices display the content.

[0356] Specific examples of behavior:

[0357] Prompt the generative AI model with learning content that includes gentle explanations on topics that Student A has low understanding of.

[0358] The generated content is sent to Student A's device and displayed on the screen.

[0359] Real-time delivery and feedback

[0360] User (student): If a question arises during learning, the user sends the question to the AI ​​from their device.

[0361] Terminal: Sends questions from students to the server in real time.

[0362] Server: Receives the question, analyzes it, and generates the best answer.

[0363] Server: Sends the generated answers and additional assignments to the student's device, which displays them.

[0364] Examples of prompts:

[0365] Question: "Why does water boil at 100 degrees?"

[0366] Answer: "When water reaches 100°C at standard atmospheric pressure, its vapor pressure becomes equal to atmospheric pressure and it begins to boil."

[0367] Teacher Support and Monitoring

[0368] User (Teacher): Accesses the server by logging in to a dedicated device or the management app.

[0369] Server: Authenticates the teacher access request and grants access to the Teacher Dashboard.

[0370] Server: Retrieves students' learning data, progress, and emotional data from the database and displays them on a dashboard.

[0371] Users (Teachers): Check the dashboard to understand each student's level of understanding, progress, and emotional changes, and plan supplementary lessons or additional assignments as needed.

[0372] Example prompt

[0373] Examples of prompt sentences are shown below.

[0374] Sample prompts: Examples of inputs to the AI ​​model that analyzes the content of students' answers

[0375] text

[0376] Analyze the students' test answers below and evaluate their level of understanding and emotional response.

[0377] Test Answers:

[0378] 1. Q1: The answer is "A"

[0379] 2. Q2: The answer is "B"

[0380] ...

[0381] Emotional Data:

[0382] When answering Q1: Facial expression "confused"

[0383] When answering Q2: Facial expression "understood"

[0384] By implementing the system based on this flow of specific data collection, analysis, generation and provision of learning content, and feedback, it is possible to maximize the learning efficiency of each student.

[0385] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0386] Program processing flow

[0387] Step 1: Collect and send data

[0388] Terminal: During class, students take tests and submit assignments. The terminal collects this input data (test answers and assignment images and text data).

[0389] Specific behavior:

[0390] When students answer a test, they enter their answers into the device.

[0391] When submitting an assignment, the text data and images of the assignment are uploaded to the device.

[0392] Input: Test answers, assignment text data, assignment images

[0393] Output: Data transmission request from the device to the server

[0394] Terminal: Sends collected data to the server in real time.

[0395] Specific behavior:

[0396] Generate an API request to the server based on the collected data and POST the data.

[0397] Input: Collected answer data, assignment data

[0398] Output: API request to the server

[0399] Step 2: Receiving and storing data

[0400] Server: Stores the received data in a database, including students' test scores, answers, and images and text of submitted reports and assignments.

[0401] Specific behavior:

[0402] The server executes queries to extract data from incoming API requests and store it in a database.

[0403] Input: Answer data and assignment data sent from the device

[0404] Output: Test results and lesson output stored in the database

[0405] Step 3: Collect and analyze emotion data

[0406] Device: The emotion engine analyzes the student's facial expressions and tone of voice. For example, emotion data is generated from changes in the student's facial expressions and tone of voice.

[0407] Specific behavior:

[0408] The device captures students' facial expressions and voices using the camera and microphone, and calls the emotion engine API to perform emotion analysis.

[0409] Input: Student facial and voice data from camera and microphone

[0410] Output: Parsed emotion data

[0411] Terminal: Transmits the analyzed emotion data to the server.

[0412] Specific behavior:

[0413] Emotion data is sent to the server as a POST request.

[0414] Input: Parsed emotion data

[0415] Output: API request to the server

[0416] Step 4: Receiving and storing emotion data

[0417] Server: Receives emotion data and stores it in a database. Emotion data includes information such as "interested," "understanding," and "confused."

[0418] Specific behavior:

[0419] The server executes a query to store the received emotion data in a database.

[0420] Input: Emotion data sent from the device

[0421] Output: Emotion data stored in a database

[0422] Step 5: Data analysis

[0423] Server: Launches a module that periodically analyzes test results, lesson output, and emotional data. It retrieves the necessary data from the database and analyzes each student's level of understanding, learning progress, and emotional patterns.

[0424] Specific behavior:

[0425] Batch processing is performed to retrieve the necessary data from the database and execute the analysis algorithm.

[0426] Input: Test results, lesson output, and emotional data stored in the database

[0427] Output: Analysis results (personal learning data, learning profile)

[0428] Server: Generates personal learning data based on the analysis results and updates the learning profile.

[0429] Specific behavior:

[0430] Based on the analysis results, a query is executed to generate personal learning data and store it in the database.

[0431] Input: Analysis results

[0432] Output: Updated personal learning data, learning profile

[0433] Step 6: Generate learning content

[0434] Server: Based on the generated personal learning data, a generation AI module is used to generate optimal learning content for students.

[0435] Specific behavior:

[0436] The necessary information is extracted from the student's learning profile and used to create prompts, which are then fed into a generative AI model.

[0437] Input: personal training data, prompt sentence

[0438] Output: Learning content output from the generative AI model

[0439] Server: Selects appropriate learning content for each student and customizes it taking into account emotional data.

[0440] Specific behavior:

[0441] Adjust the generated learning content (e.g., change the difficulty level of the content) based on emotional data.

[0442] Input: Generated learning content, emotion data

[0443] Output: Customized learning content

[0444] Server: Sends the generated learning content to students' devices in real time.

[0445] Specific behavior:

[0446] Send an API response containing customized learning content to the device.

[0447] Input: Customized learning content

[0448] Output: Learning content sent to your device

[0449] Device: Display received learning content and make it accessible to students.

[0450] Specific behavior:

[0451] Display the received learning content on the screen.

[0452] Input: Learning content sent to your device

[0453] Output: Learning content displayed on the screen

[0454] Step 7: Real-time delivery and feedback

[0455] User (student): If a question arises during learning, the user sends the question to the AI ​​from their device.

[0456] Specific behavior:

[0457] Enter your question in the form and press the submit button.

[0458] Input: Question

[0459] Output: Sending query data from the terminal to the server

[0460] Terminal: Sends questions from students to the server in real time.

[0461] Specific behavior:

[0462] Sends the question data to the server as a POST request.

[0463] Input: Question

[0464] Output: API request to the server

[0465] Server: Receives the question, analyzes it, and generates the best answer.

[0466] Specific behavior:

[0467] The question content is analyzed, a corresponding prompt sentence is generated, and input into the AI ​​model.

[0468] Receive and format answers from the AI ​​model.

[0469] Input: Question

[0470] Output: The generated answer

[0471] Server: Sends generated answers and additional assignments to student devices.

[0472] Specific behavior:

[0473] Sends an API response containing the answer to the device.

[0474] Input: Generated Answer

[0475] Output: Answer sent to terminal

[0476] Device: Show students their answers and assignments and keep them learning.

[0477] Specific behavior:

[0478] The received response is displayed on the screen.

[0479] Input: Answer sent to the terminal

[0480] Output: Answers displayed on the screen

[0481] Step 8: Teacher support and monitoring

[0482] User (Teacher): Logs in to a dedicated device or management app and accesses the server.

[0483] Specific behavior:

[0484] Enter your authentication information into the login form and submit it.

[0485] Input: Credentials

[0486] Output: Login request

[0487] Server: Authenticates the teacher access request and grants access to the Teacher Dashboard.

[0488] Specific behavior:

[0489] Generate an authentication token to grant access to the dashboard.

[0490] Input: Login request

[0491] Output: Authentication token

[0492] Server: Retrieves student learning data, progress, and emotional data from the database and displays them on a dashboard.

[0493] Specific behavior:

[0494] Execute SQL queries to retrieve data and display it in a dashboard in real time.

[0495] Input: Authenticated teacher request

[0496] Output: Training data, progress, and sentiment data on a dashboard

[0497] Users (Teachers): Check the dashboard to understand individual students' understanding, progress, and emotional changes.

[0498] Specific behavior:

[0499] Check out the graphs and statistics on the dashboard.

[0500] Input: Data on the dashboard

[0501] Output: Understanding level of understanding, progress, and emotional changes

[0502] User (Teacher): If you notice that a student is falling behind, plan a make-up lesson or additional assignment.

[0503] Specific behavior:

[0504] Enter and set up supplementary lesson plans and additional assignments into the system.

[0505] Input: Supplementary lesson plan details, additional assignment details

[0506] Output: Set supplementary lessons and additional assignments

[0507] User (Teacher): Prepares individual supplementary teaching materials as needed and provides them to students via the server.

[0508] Specific behavior:

[0509] Upload supplementary materials and send them to students' devices.

[0510] Input: Supplementary materials

[0511] Output: Supplementary materials sent to the device

[0512] This enables the system of the present invention to provide optimal learning content according to each student's level of understanding, learning progress, and emotional pattern, and also to support teachers.

[0513] (Application example 2)

[0514] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0515] It is difficult for stores to provide services and product suggestions that meet the needs of each individual customer in real time. Using conventional methods, it is not realistic for store staff to provide optimal service to every customer, and there is a lack of means to quickly and accurately analyze large amounts of customer data. Furthermore, there is a growing need for systems that can grasp customer emotions and behavior in real time and provide services based on that information.

[0516] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for collecting behavioral data and emotional data of customers, means for analyzing the behavioral data and emotional data and providing optimal product proposals and services to individual customers, and means for generating optimal product information and campaign information for each customer using a generative AI model. This makes it possible to propose optimal services and products to each customer individually.

[0517] "Learning data" refers to various data related to student learning, such as student test results, class output, and emotional data.

[0518] "Test results" refers to data such as the scores and answers students receive on tests and exams.

[0519] "Class output" refers to data such as reports, assignments, and presentations submitted by students during class.

[0520] "Emotional data" refers to data on emotions such as interest, understanding, and confusion that are analyzed from students' facial expressions and tone of voice.

[0521] "Level of understanding" is an indicator of how well a student understands the learning content.

[0522] "Learning progress" is an indicator of how far a student has progressed with their learning plan and curriculum.

[0523] "Learning content" refers to content such as teaching materials, assignments, and tests provided to support students' learning.

[0524] "Real time" refers to the time characteristic in which processing is carried out at the exact moment an event occurs.

[0525] "Questions and requests" are actions or requests that indicate a student's doubts or requests for assistance during their studies.

[0526] "Progress" is the progress that indicates how far a student is progressing with a learning activity or assignment.

[0527] "Behavioral data" refers to data on the actions customers take in the store, including their location, route of movement, and direction of gaze.

[0528] "Product proposal" is the act of selecting and recommending an appropriate product to a specific customer.

[0529] "Service" means any support or assistance provided to enhance Customer satisfaction.

[0530] A "generative AI model" is an artificial intelligence model that generates unique information or answers based on input data.

[0531] A "prompt" is an instruction or introductory sentence entered into a generative AI model to prompt it to generate specific information or an answer.

[0532] "Campaign Information" refers to marketing information such as discounts, special offers, and events offered for a specific period or under specific conditions.

[0533] This invention implements a "customer experience improvement system" that responds to the needs of each individual customer in a store.

[0534] 1. Data Collection and Storage

[0535] Devices: Video and audio are collected using smartphones or smart glasses used by customers in the store. Smart glasses have built-in cameras and microphones and collect data in real time. For example, video and audio data is collected when a customer uses smart glasses to browse the shelves.

[0536] Server: The collected data is sent to a cloud server and stored in a database. The server can use AWS (registered trademark) cloud services or Google Cloud.

[0537] 2. Collecting and analyzing emotion data

[0538] Device: Using an emotion engine (such as Amazon Rekognition or Google Cloud Vision), the device analyzes the customer's facial expressions and tone of voice to generate emotion data. For example, if a customer is interested in a particular product but confused, emotion data representing interest and confusion will be generated from their facial expressions.

[0539] Server: Emotion data is sent to the cloud server in real time and stored in a database.

[0540] 3. Data Analysis

[0541] Server: The cloud-based analysis module periodically retrieves necessary data from the database and analyzes customer behavior patterns and sentiment data. This analysis uses machine learning algorithms using programming languages ​​such as Python and R.

[0542] Server: Based on the analysis results, the customer's purchasing profile is updated. This profile includes the customer's past purchasing history, behavioral data, and emotional data.

[0543] 4. Creation of Learning Content Services

[0544] Server: A generative AI model (such as GPT-4) generates optimal product information and campaign information based on the prompt text.

[0545] Server: The generated information is sent to your device in real time.

[0546] Terminal: Displays the received information and provides it in a form that can be viewed and used by the customer.

[0547] 5. Real-time delivery and feedback

[0548] Customers: When customers select a product or have questions, they send their feedback or questions to the system via their smartphone or smart glasses.

[0549] Device and Server: Feedback and questions are sent in real time to the cloud server for analysis. Based on the analysis results, the generative AI model generates the optimal answer and delivers it to the customer's device.

[0550] Specific examples

[0551] Example 1: Customer browsing the store on their smartphone

[0552] 1. A customer takes a photo of a product shelf using their smartphone camera.

[0553] 2. The device transmits video and audio in real time to the cloud server.

[0554] 3. The server analyzes the data using an emotion engine and generates emotion data such as interest or confusion.

[0555] 4. The server launches a generative AI model based on the analysis results to create optimal product recommendations.

[0556] 5. The device will display suggested product information and campaigns.

[0557] Example 2: Using smart glasses

[0558] 1. A customer puts on smart glasses and walks through the store.

[0559] 2. The device collects video and audio using the camera and microphone and sends them to a cloud server.

[0560] 3. The server uses an emotion engine to analyze the customer's facial expressions and tone of voice, generating emotion data in real time.

[0561] 4. The server uses the generative AI model to generate optimal product suggestions and campaign information.

[0562] 5. The device displays the suggested information on the smart glasses display.

[0563] Examples of prompt statements

[0564] "When a customer is looking at the shelf facing the camera, the prompt reads:

[0565] "This customer seems interested in this product, but also a little confused. Please provide the following information: product features, customer testimonials, other customer reviews, and recommended outfits."

[0566] "When a customer asks a question, the prompt is:

[0567] "Customers have asked the following question: 'What other colors does this product come in and what are its benefits?'"

[0568] This enables the system to provide individualized information and services to each customer in real time.

[0569] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0570] Step 1:

[0571] User: A customer uses a smartphone or smart glasses in a store to browse the shelves.

[0572] Input: Video and audio data collected by smartphones and smart glasses.

[0573] How it works: The camera captures images of the shelves in the store, and the microphone collects audio. The collected data is then saved in the device in real time.

[0574] Step 2:

[0575] Terminal: Sends collected video and audio data to the cloud server.

[0576] Input: Video and audio data stored on a smartphone or smart glasses.

[0577] Output: Video and audio data sent to the cloud server.

[0578] Specific operation: The terminal uses the HTTPS protocol to send data to the cloud server over the network.

[0579] Step 3:

[0580] Server: The cloud server generates customer emotion data using the emotion engine.

[0581] Input: Video and audio data stored on a cloud server.

[0582] Output: Analyzed emotion data (emotional information such as interest, confusion, etc.).

[0583] What it does: It uses emotion analysis services such as Amazon Rekognition and Google Cloud Vision to analyze the customer's facial expressions and tone of voice from video and audio to generate emotion data.

[0584] Step 4:

[0585] Server: The cloud server integrates and analyzes the behavioral and emotional data to update the customer's purchasing profile.

[0586] Input: Previously generated emotion data and previously stored behavioral data.

[0587] Output: Updated customer purchasing profile.

[0588] What it does: Collates databases, integrates and analyzes historical behavioral and sentiment data, and uses machine learning algorithms in Python and R to analyze customer buying patterns and interests.

[0589] Step 5:

[0590] Server: Generates optimal product suggestions and campaign information using generative AI models.

[0591] Input: Updated customer purchasing profile and prompt text.

[0592] Output: Generated product suggestions and campaign information.

[0593] How it works: A generative AI model (such as GPT-4) generates optimal product suggestions and campaign information based on a prompt such as, "This customer seems interested in this product, but also seems a little confused. Please provide the following information: this product's features, customer reviews, other customer reviews, and recommended outfits."

[0594] Step 6:

[0595] Server: Generated product proposals and campaign information are sent to the customer's device in real time.

[0596] Input: Product suggestions and campaign information generated by the AI ​​model.

[0597] Output: Product offers and campaign information sent to your device.

[0598] What it does: The generated information is sent in real time to your smartphone or smart glasses using the HTTPS protocol.

[0599] Step 7:

[0600] Device: Check and use product information and campaign information suggested by customers.

[0601] Input: Product offers and campaign information sent to your device.

[0602] Output: Display of product information and campaign information that customers can check.

[0603] Specific operation: Product information and campaign information generated by the generative AI model is displayed on the device display, and provided in a form that can be viewed and used by the customer.

[0604] Step 8:

[0605] Users and devices: To help you select products and provide feedback if you have questions.

[0606] Input: Feedback of your questions and suggestions.

[0607] Specific operation: Using a smartphone or smart glasses, customers enter feedback or questions and send them to a cloud server.

[0608] Step 9:

[0609] Server: Generates optimal answers based on feedback and questions, and delivers them to your device in real time.

[0610] Input: Feedback and questions submitted by customers.

[0611] Output: Optimized answer and / or additional information.

[0612] What it does: Uses a generative AI model to generate optimal answers based on customer questions and feedback, and sends them to the device in real time. For example, it uses a prompt like, "A customer asked the following question: 'What other colors does this product come in and what are its benefits?'"

[0613] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0614] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0615] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0616] [Second embodiment]

[0617] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0618] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0619] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0620] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0621] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0622] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0623] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0624] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0625] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0626] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0627] In the smart glasses 214, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0628] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0629] This invention relates to a learning system that uses generative AI to provide optimal learning content for each student. Specifically, it describes a method that incorporates students' test results and class output, generates and analyzes personal learning data, and provides learning content in real time based on that data.

[0630] Data collection and storage

[0631] Devices: When students take tests or submit assignments during class, their devices (PCs, tablets, etc.) collect this data and send it to the server.

[0632] Server: Stores received test results and lesson output in a database. For example, it stores students' test scores and answers, as well as images and text data of submitted reports and assignments.

[0633] Data analysis

[0634] Server: The analysis module is launched periodically or when new data is added. The server retrieves test results and lesson outputs for each student in the database and analyzes them.

[0635] Example: Analyze the accuracy rate, answer time, and patterns of incorrect answers of math problems solved by Student A to generate a learning profile. For example, if it turns out that the accuracy rate for a specific type of problem is low, provide learning content that reinforces that type of problem.

[0636] Generating learning content

[0637] Server: Based on the generated learning profile, the server uses generative AI to generate optimal learning content for each student. The content can come in a variety of formats, including text, images, videos, and interactive questions.

[0638] Example: For a type of math problem that Student A struggles with, we generate related explanatory videos and practice problems and provide them as a single learning package.

[0639] Real-time delivery and feedback

[0640] Server: The generated content is sent to students' devices in real time, allowing them to view the content on their own devices and progress with their studies.

[0641] Device: If a student has a question while studying, they can immediately input it into the AI. For example, they can ask, "Why is the answer to this problem like this?"

[0642] Server: Receives questions, analyzes them in real time, generates appropriate answers and additional questions, and sends them to students' devices. Students receive these and continue their learning.

[0643] Teacher Support and Monitoring

[0644] User (Teacher): Teachers access the server using a dedicated device or management app.

[0645] Server: Provides a dashboard for teachers, displaying each student's learning progress and level of understanding, allowing teachers to see at a glance how each student is learning.

[0646] Example: A teacher determines that Student A's learning progress is falling behind and plans additional follow-up lessons. If necessary, individual supplementary learning materials can be prepared and provided to the student via the server.

[0647] This will provide each student with the optimal learning environment, enabling high-quality education while reducing the workload of teachers.

[0648] The processing flow will be explained below.

[0649] Step 1:

[0650] User (Teacher): Uploads tests and supplementary materials to be used in class to the server.

[0651] Server: Organizes the uploaded data and links it to the corresponding lesson plans.

[0652] Device: Prepares students to receive tests and supplementary materials.

[0653] Step 2:

[0654] User (student): Takes a test or submits an assignment during class.

[0655] Device: Collects student answers and submissions and sends them to the server in real time. Records student answers digitally.

[0656] Server: Receives the transmitted data and immediately stores it in the database. Backs up the stored data.

[0657] Step 3:

[0658] Server: Launches the data analysis module periodically or when new student data is added.

[0659] Server: Retrieves student test results and lesson output from the database.

[0660] Server: Analyzes the acquired data and calculates each student's level of understanding, learning progress, and patterns of incorrect answers.

[0661] Server: Based on the analysis results, personal learning data is generated and the learning profile for each student is updated.

[0662] Step 4:

[0663] Server: Based on the updated personal learning data, the generative AI module is launched.

[0664] Server: Selects and combines resources (text, images, videos, etc.) containing learning content appropriate for each student.

[0665] Server: Sends the generated learning content to students' devices in real time.

[0666] Device: Display received learning content and make it accessible to students.

[0667] Step 5:

[0668] User (student): If a question arises during learning, the user sends the question to the AI ​​from their device.

[0669] Terminal: Sends questions from students to the server in real time.

[0670] Server: Receives the question and starts the process of analyzing it and generating the best answer.

[0671] Server: Sends generated answers and additional assignments to student devices.

[0672] Device: Show students answers and assignments so they can continue learning.

[0673] Step 6:

[0674] User (Teacher): Logs in to a dedicated device or management app and accesses the server.

[0675] Server: Authenticates the teacher access request and grants access to the Teacher Dashboard.

[0676] Server: Retrieves student learning data, progress, and understanding from the database and displays them on a dashboard.

[0677] User (Teacher): Check the dashboard to understand each student's understanding and progress.

[0678] User (Teacher): Sees that Student A is falling behind and acts on the data to plan make-up lessons and additional assignments.

[0679] Example 1

[0680] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0681] With conventional learning systems, it was difficult to provide individualized learning content that corresponded to each student's level of understanding and learning progress. It also required a great deal of effort for teachers to grasp each student's progress and level of understanding, making it difficult to provide appropriate guidance and feedback to each student in real time. Furthermore, there was a lack of a mechanism for immediate response when students had questions during their studies.

[0682] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0683] In this invention, the server includes means for collecting assessment results and lesson outcomes to generate individual student learning data, means for analyzing the learning data and grasping each student's level of understanding and learning progress, means for generating optimal learning content for each student based on the generated learning data using a generative AI model, means for providing the learning content to students in real time, means for responding to questions and requests from students in real time, means for teachers to check students' progress and level of understanding, and means for teachers to plan additional learning support and provide supplementary materials to students. This makes it possible to provide learning content appropriate for each student, resolve questions in real time, and provide effective educational support while reducing the burden on teachers.

[0684] "Individual student learning data" refers to digital data generated by each student through lessons and assessment activities, and is information that indicates the student's level of understanding and progress.

[0685] "Evaluation results" refers to the scores and evaluation content obtained through learning assessment methods such as tests and assignments.

[0686] "Class outcomes" refers to the results of learning activities that students undertake during class or extracurricular activities, and include a variety of formats such as text, images, and videos.

[0687] "Analysis" refers to the process of evaluating students' learning patterns and comprehension based on collected data and creating individual profiles.

[0688] "Understanding" refers to an indicator of how well a student understands a particular learning content.

[0689] "Learning progress" refers to an indicator that shows how far a student is progressing along the learning curriculum.

[0690] A "generative AI model" refers to an algorithm or model that uses AI technology to automatically generate specific learning content based on input data.

[0691] "Learning content" refers to educational content provided for students to learn, and includes various formats such as text, images, videos, and interactive questions.

[0692] "Real-time" refers to a state in which data is exchanged and processed immediately and is available without waiting.

[0693] "Teacher burden" refers to the time and effort required for teachers to check students' progress, provide feedback, and provide individual instruction.

[0694] "Instructional aids" refer to activities and tools that support the learning process and enable students to learn effectively.

[0695] "Supplemental materials" refers to additional learning materials or assignments provided to deepen students' understanding.

[0696] This invention relates to a learning system that uses a generative AI model to provide optimal learning content for each student. Specifically, it shows a method for incorporating student assessment results and class performance, generating and analyzing personalized learning data, and providing learning content in real time based on that data.

[0697] Data collection and storage

[0698] Device:

[0699] When students take tests or submit assignments during class, devices such as computers and tablets collect this data and send it to a server. For example, students answer test questions and save the results on their devices. The collected data is then sent to a server.

[0700] server:

[0701] The received evaluation results and lesson outcomes are stored in a database. Specifically, data such as student IDs, test scores, answers, and submitted reports are stored. After the data is stored in the database, the conditions for launching the analysis module are checked.

[0702] Data analysis

[0703] server:

[0704] When new data is added, the analysis module is triggered. The server collects data for each student in the database and passes it to the analysis engine. For example, analyzing math test results, it finds out that Student A has a low success rate in certain question types and generates a learning profile.

[0705] Generating learning content

[0706] server:

[0707] Based on the generated learning profile, specific learning content is input as a prompt to the generative AI model. For example, a prompt such as "Please generate explanatory videos and practice problems for math problem type X for Student A" is provided to the generative AI model. The text, images, videos, problem sets, etc. obtained from the generative AI model are compiled into a single learning package, including related explanatory videos and explanations of the answers to the problems.

[0708] As a specific example, if Student A has difficulty with a particular type of math problem, the generative AI model will generate explanatory videos and practice problems based on the prompt text and provide them as a learning package.

[0709] Example prompt sentence:

[0710] Please generate explanatory videos and practice questions for math problem type X based on the test results of Student A. I would like the format to be a combination of text, images, and videos.

[0711] Real-time delivery and feedback

[0712] server:

[0713] The generated learning content is sent to students' devices in real time. Data communication protocols are used to ensure reliable data delivery. Students can then check the content on their own devices and continue their learning.

[0714] Device:

[0715] If a student has a question while studying, they can immediately input it into the AI, for example, "Why is the answer to this problem like this?" The question is then sent to the server.

[0716] server:

[0717] Analyze the question and generate appropriate answers or provide additional practice questions, possibly using a generative AI model, and send the answers or additional questions to the student's device.

[0718] Teacher Support and Monitoring

[0719] User (Teacher):

[0720] Teachers access the server using dedicated devices or a management app. They can use a dashboard to visually monitor each student's progress and understanding. Teachers can use this information to plan additional lessons or provide supplementary materials for specific students.

[0721] server:

[0722] The teacher dashboard displays learning data for each student and makes it easily accessible to teachers. For example, areas of low understanding are visually highlighted. Based on this information, teachers can input follow-up instructions into the server and send them to students' devices.

[0723] This system provides each student with an optimal learning environment, enables real-time question resolution, and provides effective educational support while reducing the burden on teachers.

[0724] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0725] Step 1: Collect data

[0726] Input: Students use their devices to take tests or submit assignments.

[0727] Specific operation: Students answer test questions and save their answers and assignment data on their devices. This includes text data, image data, video data, etc.

[0728] Output: The device sends the collected data to the server.

[0729] Step 2: Receiving and storing data

[0730] Input: Data such as test results and class results sent from the device.

[0731] Specific operation: The server receives the student's evaluation results and lesson results and stores them in a database, including student IDs, test scores, answers, reports, etc.

[0732] Output: Data per student stored in a database.

[0733] Step 3: Analyze the data

[0734] Input: New student data stored in the database.

[0735] How it works: When the server detects new data, it launches the analysis module. The analysis engine collects data for each student in the database and evaluates the student's understanding and learning progress.

[0736] Output: The resulting learning profile.

[0737] Step 4: Generate a learning profile

[0738] Input: Analysis results for each student from the analysis module.

[0739] Specific operation: The server generates a learning profile and creates detailed learning data for each student, including their individual weaknesses and strengths, such as the percentage of correct answers to math problems, the time it takes to answer them, and error patterns.

[0740] Output: The generated learning profile.

[0741] Step 5: Generate learning content

[0742] Input: Learning profile.

[0743] Specific operation: The server inputs a prompt based on the learning profile into the generative AI model. For example, "Please generate an explanatory video and practice questions for math problem type X for Student A." The generative AI model analyzes the prompt and automatically generates appropriate learning content (text, images, videos, interactive questions, etc.).

[0744] Output: The generated learning content.

[0745] Step 6: Package your learning content

[0746] Input: Learning content from a generative AI model.

[0747] What it does: The server compiles the generated content into a learning package, including related instructional videos and explanations of the answers to the questions.

[0748] Output: Packaged learning content.

[0749] Step 7: Real-time serving

[0750] Input: Packaged learning content.

[0751] Specific operation: The server transmits the learning package to the student's device in real time, using a data communication protocol to ensure the data is delivered.

[0752] Output: The learning content displayed on the student's device.

[0753] Step 8: Answering student questions

[0754] Input: Questions from students during learning.

[0755] Specific operation: When a student has a question while studying, they type the question on their device and send it to the server. For example, "Why is the answer to this problem like this?"

[0756] Output: The question sent to the server.

[0757] Step 9: Parsing the question and generating an answer

[0758] Input: The student question sent to the server.

[0759] What it does: The server analyzes the question and generates appropriate answers or provides additional practice questions, possibly using a generative AI model.

[0760] Output: Generated answers and additional practice questions.

[0761] Step 10: Provide your answers

[0762] Input: Generated answers and additional practice questions.

[0763] Specific operation: The server sends answers and additional practice questions to the student's device, where the student receives them and uses them to help with their studies.

[0764] Output: Answers and exercises displayed on student devices.

[0765] Step 11: Teacher Support and Monitoring

[0766] Input: Access from a dedicated device or management app.

[0767] How it works: Teachers access the server and use a dedicated dashboard to check students' progress and understanding, which displays each student's learning profile and progress.

[0768] Output: Teacher progress review and lesson plan.

[0769] Step 12: Plan additional follow-up lessons

[0770] Input: Student progress data viewed through the dashboard.

[0771] What it does: Teachers plan additional lessons and supplementary materials for specific students through the dashboard, and follow-up instructions are entered into the server and sent to student devices.

[0772] Output: Follow-up instructions are sent to the student.

[0773] Through this series of steps, learning content tailored to each student is provided, questions are answered in real time, and teachers can provide effective educational support.

[0774] (Application example 1)

[0775] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0776] Conventional learning systems have difficulty providing optimal learning content in real time based on each student's level of understanding and learning progress. They also lack a system that can provide immediate answers when students have questions. Furthermore, there are insufficient means for teachers to grasp students' learning progress in detail and provide appropriate follow-up. As a result, an environment that maximizes the learning effect of each student in personalized education has not been established.

[0777] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0778] In this invention, the server includes: means for collecting students' test results and lesson output; means for analyzing the learning data and grasping each student's level of understanding and learning progress; means for generating learning content optimized for each student based on the generated learning data; means for providing the learning content to students in real time; means for responding to students' questions and requests in real time; means for teachers to check students' progress and understanding; means for connecting to smart devices used in brick-and-mortar educational facilities and providing learning content and feedback in real time; means for generating learning content using a generative AI model based on a specific learning profile; and means for generating prompt sentences to generate appropriate feedback on topics that the student struggles with. This enables the provision of optimal learning content for each student. Furthermore, real-time responses can immediately resolve students' questions and improve learning efficiency. Furthermore, providing teachers with a means for accurately grasping students' learning progress and providing effective follow-up can improve the quality of personalized education.

[0779] "Individual student learning data" refers to data on individual students' test results and class output.

[0780] "Learning data analysis" refers to the process of processing and analyzing collected data to understand each student's level of understanding and learning progress.

[0781] "Learning Content" refers to optimal learning materials provided to students using generative AI models, and may include formats such as text, images, and videos.

[0782] "Real-time delivery" refers to the instant delivery of generated learning content to students.

[0783] "Student Q&A" refers to the process of providing immediate and appropriate answers or additional learning content in response to student questions or requests.

[0784] "Teacher Monitoring Tools" refers to tools specifically designed for teachers to monitor and manage student progress and understanding.

[0785] "Smart device connectivity" refers to linking the system with devices such as smartphones, tablets, and smart glasses used in brick-and-mortar educational facilities.

[0786] A "generative AI model" refers to an artificial intelligence model that automatically generates optimal learning content and feedback based on input data.

[0787] A "prompt" is a sentence that constructs instructions or questions for a generative AI model based on a specific learning profile.

[0788]

[0789] This invention is a smart education support system that utilizes generative AI models to provide optimal learning content for individual students in brick-and-mortar educational facilities. A specific embodiment of this system is described below.

[0790] Hardware Configuration

[0791] It uses a server and smart devices such as smartphones, tablets, and smart glasses. The server has the central function of collecting, analyzing, generating, and distributing data. The smart devices are used by students and teachers as an interface to receive learning content and feedback in real time.

[0792] Software Configuration

[0793] It uses the Python programming language, the generative AI model API provided by OpenAI, and a database management system.

[0794] System action

[0795] The server collects test results and class output from students' devices and stores them in a database. The collected data is periodically analyzed on the server. As a result of the analysis, each student's level of understanding and learning progress is identified and an individual learning profile is generated.

[0796] Based on this learning profile, the server uses a generative AI model to generate optimal learning content. At this time, a prompt sentence is input to the AI ​​model. For example, the following prompt sentence is generated:

[0797] Example prompt sentence:

[0798] Based on the math test results, Student A's most common mistakes are solving equations and basic use of trigonometric functions. Create learning content to deepen their understanding of these topics.

[0799] The generated learning content is delivered to students' devices in real time. Students can view this content and continue their studies via smartphones, tablets, or smart glasses. If students have questions while studying, they can input them in real time, and the generative AI model will instantly generate an answer, which will then be delivered to the student's device.

[0800] Teachers can also use the system to access a dedicated dashboard that allows them to see students' progress and understanding in real time, allowing them to plan individual follow-ups and provide additional materials or assignments if necessary.

[0801] For example, if test analysis reveals that Student A is struggling with fraction calculations, the teacher can provide supplementary materials on fraction calculations. This system provides an optimal learning environment for each student, improving the quality of personalized education.

[0802] As described above, by implementing the present invention, it becomes possible to realize real-time individualized learning and teacher support, which was difficult with conventional systems.

[0803] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0804]

[0805] Step 1:

[0806] The server collects test results and class output (text data, image data, video data) from students' devices. This allows the collection of student learning data. The test results and class output data taken by students are used as input. As output, the data is stored in a database on the server.

[0807] Step 2:

[0808] The server periodically launches the data analysis module to retrieve the students' learning data stored in the database. The data analysis module analyzes this learning data to understand each student's level of understanding and learning progress. The collected learning data is used as input for this step. The output is a learning profile for each student.

[0809] Step 3:

[0810] The server creates a prompt to generate specific learning content based on the generated learning profile. For example, it generates a prompt such as, "Based on the results of the math test, the questions that Student A often gets wrong are: solving equations, basic use of trigonometric functions. Please generate learning content to deepen his understanding of these topics." The student's learning profile is used as input. The generated prompt is used as output.

[0811] Step 4:

[0812] The server sends the generated prompt sentences to a generative AI model, which then generates optimal learning content based on the prompt sentences. For example, videos and supplementary learning materials are generated to deepen understanding. The prompt sentences are used as input, and the learning content is generated as output.

[0813] Step 5:

[0814] The generated learning content is sent from the server to the student's device in real time. The student checks this content on their device and proceeds with their learning. The generated learning content is used as input for this step. The output is the content displayed on the student's device.

[0815] Step 6:

[0816] When a device receives a question or request from a student, the question or request is sent to the server. The server uses a generative AI model to generate an appropriate answer for the received question and sends it back to the student's device. The question or request from the student is used as input. The answer is displayed on the student's device as output.

[0817] Step 7:

[0818] Teachers access a dedicated dashboard to check students' learning progress and comprehension data provided by the server. If necessary, teachers create additional learning materials and assignments and provide them to students' devices via the server. Student learning profiles and progress data are used as input. Additional learning materials and assignments are provided to students as output.

[0819] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0820] This invention relates to a learning system that uses generative AI and an emotion engine to provide optimal learning content for each student. Specifically, it describes a method for incorporating students' test results, class output, and emotional data, generating and analyzing personalized learning data, and providing learning content in real time based on that data.

[0821] Data collection and storage

[0822] Devices: Students take tests and submit assignments during class. The devices (computers, tablets, etc.) collect this data and send it to the server.

[0823] Server: Stores received test results and lesson output in a database. For example, it stores students' test scores and answers, as well as images and text data of submitted reports and assignments.

[0824] Collecting Emotional Data

[0825] Device: The emotion engine analyzes the student's facial expression, tone of voice, and input content to generate emotion data. For example, it senses changes in facial expression and voice when a student is facing a problem.

[0826] Server: Receives emotion data and stores it in a database. Emotion data includes information such as "interested," "understanding," and "confused."

[0827] Data analysis

[0828] Server: Launches modules that periodically analyze test results, lesson output, and sentiment data.

[0829] Server: Retrieves necessary learning data from the database and analyzes each student's level of understanding, learning progress, and emotional patterns.

[0830] Server: Based on the analysis results, generate personal learning data and update each student's learning profile.

[0831] Generating learning content

[0832] Server: Based on the generated personal learning data, the generative AI module is launched.

[0833] Server: Selects appropriate learning content for each student and customizes it taking into account emotional data.

[0834] Server: Sends the generated learning content to students' devices in real time.

[0835] Device: Display received learning content and make it accessible to students.

[0836] Real-time delivery and feedback

[0837] User (student): If a question arises during learning, the user sends the question to the AI ​​from their device.

[0838] Terminal: Sends questions from students to the server in real time.

[0839] Server: Receives the question and starts the process of analyzing it and generating the best answer.

[0840] Server: Sends generated answers and additional assignments to student devices.

[0841] Device: Show students answers and assignments so they can continue learning.

[0842] Teacher Support and Monitoring

[0843] User (Teacher): Accesses the server using a dedicated device or management app.

[0844] Server: Authenticates the teacher access request and grants access to the Teacher Dashboard.

[0845] Server: Retrieves student learning data, progress, comprehension, and emotional data from the database and displays them on a dashboard.

[0846] Users (Teachers): Check the dashboard to understand individual students' understanding, progress, and emotional changes.

[0847] User (Teacher): If the teacher determines that a student's progress is falling behind, he / she plans supplementary lessons and additional assignments. If necessary, he / she prepares individual supplementary materials and provides them to the student via the server.

[0848] Specific examples

[0849] Example 1: A student takes a test

[0850] 1. User (student): Take a test during class.

[0851] 2. Terminal: Records students' answers and transmits them to the server in real time.

[0852] 3. Server: Receives and stores data.

[0853] 4. Server: The emotion engine analyzes the student's facial expressions and tone of voice to generate emotion data during the test.

[0854] 5. Server: Analyzes the test results and emotion data and updates the personal training data.

[0855] Example 2: When a student asks an AI a question

[0856] 1. User (student): If a question arises during learning, the user sends the question to the AI ​​from their device.

[0857] 2. Terminal: Sends questions from students to the server in real time.

[0858] 3. Server: Receives the question and starts the process of analyzing it and generating the best answer.

[0859] 4. Server: Sends the generated answers to the students' devices.

[0860] 5. Device: Show students their answers and continue learning.

[0861] Teachers checking student progress

[0862] 1. User (teacher): Log in to a dedicated device or the management app and access the server.

[0863] 2. Server: Authenticates the teacher access request and grants access to the Teacher Dashboard.

[0864] 3. Server: Retrieves students' learning data, progress, and emotional data from the database and displays them on the dashboard.

[0865] 4. Users (Teachers): Check the dashboard to understand individual students' understanding, progress, and emotional changes.

[0866] 5. User (Teacher): If you notice that a student is falling behind, plan a make-up lesson or additional assignment.

[0867] The processing flow will be explained below.

[0868] Step 1:

[0869] User (Teacher): Uploads tests and supplementary materials to be used in class to the server. Materials include workbooks, videos, slides, etc.

[0870] Server: Organizes uploaded data by format and links it to the corresponding lesson plans.

[0871] Devices: Student devices are prepared and configured to access materials.

[0872] Step 2:

[0873] User (student): Takes tests or submits assignments during class. Study the designated material on the device.

[0874] Terminal: Collects student answers and submissions (text, images, audio, etc.) and sends them to the server in real time.

[0875] Server: Receives the transmitted data and immediately stores it in a database, along with recording the timestamp information.

[0876] Step 3:

[0877] Devices: Data is collected using sensors that detect students' facial expressions and tone of voice, for example, cameras and microphones that analyze real-time video and audio.

[0878] Server: Analyzes facial expressions, tone of voice, and input content collected by the emotion engine to generate emotional data.

[0879] Server: Receives emotional data and stores it in a database. The emotional data includes students' reactions (happiness, confusion, etc.).

[0880] Step 4:

[0881] Server: Launches the data analysis module periodically or when new data is added.

[0882] Server: Retrieves students' test results, lesson output, and emotion data from the database.

[0883] Server: Comprehensively analyzes the acquired data and calculates each student's level of understanding, learning progress, and emotional patterns. For example, it evaluates the student's level of concentration when working on a specific task.

[0884] Server: Based on the analysis results, generate personal learning data and update each student's learning profile.

[0885] Step 5:

[0886] Server: Based on the updated personal learning data, the generative AI module is launched.

[0887] Server: Selects appropriate learning content for each student and customizes it taking into account emotional data, for example, including explanatory videos or additional practice questions for difficult assignments.

[0888] Server: Sends the generated learning content to students' devices in real time.

[0889] Device: Display received learning content and make it accessible to students.

[0890] Step 6:

[0891] User (student): If a question arises during learning, the user sends the question to the AI ​​from their device.

[0892] Terminal: Sends questions from students to the server in real time.

[0893] Server: Receives the question and starts the process of analyzing it and generating the best answer.

[0894] Server: Sends generated answers and additional assignments to student devices.

[0895] Device: Show students their answers and assignments and keep them learning.

[0896] Step 7:

[0897] User (Teacher): Accesses the server using a dedicated device or management app.

[0898] Server: Authenticates the teacher access request and grants access to the Teacher Dashboard.

[0899] Server: Retrieves student learning data, progress, comprehension, and emotion data from the database and displays them on a dashboard.

[0900] Users (Teachers): Check the dashboard to understand individual students' understanding, progress, and emotional changes.

[0901] User (Teacher): If it is determined that Student A's progress is falling behind, the teacher plans supplementary lessons and additional assignments. If necessary, the teacher conducts individual follow-up based on the emotional data.

[0902] This configuration allows a learning system incorporating an emotion engine to comprehensively evaluate not only a student's level of understanding and progress, but also their emotional changes, and provide optimal learning content in real time. This also makes it easier for teachers to grasp the psychological state of their students, enabling more detailed instruction.

[0903] Example 2

[0904] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0905] Conventional learning systems have the problem that it is difficult to accurately grasp each student's level of understanding and learning progress, and they can only provide uniform learning content. Furthermore, they are unable to provide learning content that takes into account students' emotional data, which affects students' motivation and learning efficiency. As a result, they are unable to provide an optimal learning experience for each student, and there are issues with not being able to provide effective learning support.

[0906] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0907] In this invention, the server includes means for collecting students' test results and lesson output to generate individual student learning data, means for analyzing the learning data and students' emotional data to grasp each student's level of understanding, learning progress, and emotional patterns, and means for generating optimal learning content for each student based on the generated learning data and emotional data. This makes it possible to provide optimal learning content based not only on each student's level of understanding and learning progress, but also on their emotional patterns.

[0908] "Individual student learning data" refers to data that records in detail each student's learning situation, achievements, and level of understanding.

[0909] "Test results" are data that indicate the scores and answers given by students on tests they have taken.

[0910] "Class output" refers to data that includes assignments, reports, and notes that students submit during and after class.

[0911] "Emotional data" is data about emotions obtained by analyzing students' facial expressions, tone of voice, content of speech, etc.

[0912] A "learning profile" is individual data that compiles each student's level of understanding, learning progress, emotional patterns, etc.

[0913] A "generative AI module" is a program or system that uses artificial intelligence technology to generate optimal learning content for students.

[0914] "Learning content" refers to the learning materials and assignments provided to students for learning, and includes various formats such as text, images, and videos.

[0915] The "Teacher Dashboard" is a management screen that allows teachers to check students' learning status, level of understanding, progress, and emotional patterns in real time.

[0916] "Providing in real time" means providing learning content and feedback instantly in line with the current learning environment and situation.

[0917] "Feedback" is information that includes evaluations of students' learning activities, advice, and areas for improvement.

[0918] "Additional teaching materials and assignments" refers to new learning materials and assignments that teachers add to reinforce what students have learned.

[0919] MODE FOR CARRYING OUT THE INVENTION

[0920] This invention is a learning system that provides optimal learning content to students, utilizing a generative AI model and an emotion engine. The system collects students' test results, class output, and emotion data, generates and analyzes individual learning data, and provides learning content in real time based on that data.

[0921] Hardware and software used

[0922] Device:

[0923] Student computers or tablets

[0924] Camera and microphone (for collecting emotional data)

[0925] server:

[0926] Database management systems (e.g., MySQL, PostgreSQL)

[0927] Emotion analysis API (e.g. Microsoft Emotion API, Google Cloud Vision)

[0928] Generative AI modules (e.g., OpenAI GPT-4, Google BERT)

[0929] System Description

[0930] Data collection and storage

[0931] Device: Students take tests and submit assignments during class. The device collects this data (test answers and assignment images / text data) and sends it to the server.

[0932] Server: Stores the received data in a database. Collected data includes students' test scores, answers, submitted papers, and images and text of assignments.

[0933] Specific examples of behavior:

[0934] Students take math tests and upload the results from their devices to the server.

[0935] The server stores the received data in a database for subsequent analysis.

[0936] Collecting Emotional Data

[0937] Device: The emotion engine analyzes the student's facial expressions and tone of voice. For example, emotion data is generated from changes in the student's facial expressions and tone of voice.

[0938] Server: Receives the analyzed emotion data and stores it in a database. The emotion data includes information such as "interested," "understanding," and "confused."

[0939] Specific examples of behavior:

[0940] If a student shows a confused expression during a test, the emotion engine analyzes the facial expression data and generates emotion data for "confusion."

[0941] The generated emotion data is sent to a server and stored in a database.

[0942] Data analysis

[0943] Server: Launches a module that periodically analyzes test results, lesson output, and emotional data. It retrieves the necessary data from the database and analyzes each student's level of understanding, learning progress, and emotional patterns.

[0944] Server: Based on the analysis results, generate personal learning data and update the learning profile.

[0945] Specific examples of behavior:

[0946] Student A's level of understanding is evaluated based on his / her test results, and his / her learning profile is updated by integrating it with lesson output and emotional data.

[0947] Generating learning content

[0948] Server: Based on the generated personal learning data, a generation AI module is used to generate optimal learning content for students.

[0949] Server: Selects appropriate learning content for each student and customizes it taking into account emotional data.

[0950] Server: Sends the generated learning content to students' devices in real time, and the devices display the content.

[0951] Specific examples of behavior:

[0952] Prompt the generative AI model with learning content that includes gentle explanations on topics that Student A has low understanding of.

[0953] The generated content is sent to Student A's device and displayed on the screen.

[0954] Real-time delivery and feedback

[0955] User (student): If a question arises during learning, the user sends the question to the AI ​​from their device.

[0956] Terminal: Sends questions from students to the server in real time.

[0957] Server: Receives the question, analyzes it, and generates the best answer.

[0958] Server: Sends the generated answers and additional assignments to the student's device, which displays them.

[0959] Examples of prompts:

[0960] Question: "Why does water boil at 100 degrees?"

[0961] Answer: "When water reaches 100°C at standard atmospheric pressure, its vapor pressure becomes equal to atmospheric pressure and it begins to boil."

[0962] Teacher Support and Monitoring

[0963] User (Teacher): Accesses the server by logging in to a dedicated device or the management app.

[0964] Server: Authenticates the teacher access request and grants access to the Teacher Dashboard.

[0965] Server: Retrieves students' learning data, progress, and emotional data from the database and displays them on a dashboard.

[0966] Users (Teachers): Check the dashboard to understand each student's level of understanding, progress, and emotional changes, and plan supplementary lessons or additional assignments as needed.

[0967] Example prompt

[0968] Examples of prompt sentences are shown below.

[0969] Sample prompts: Examples of inputs to the AI ​​model that analyzes the content of students' answers

[0970] text

[0971] Analyze the students' test answers below and evaluate their level of understanding and emotional response.

[0972] Test Answers:

[0973] 1. Q1: The answer is "A"

[0974] 2. Q2: The answer is "B"

[0975] ...

[0976] Emotional Data:

[0977] When answering Q1: Facial expression "confused"

[0978] When answering Q2: Facial expression "understood"

[0979] By implementing the system based on this flow of specific data collection, analysis, generation and provision of learning content, and feedback, it is possible to maximize the learning efficiency of each student.

[0980] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0981] Program processing flow

[0982] Step 1: Collect and send data

[0983] Terminal: During class, students take tests and submit assignments. The terminal collects this input data (test answers and assignment images and text data).

[0984] Specific behavior:

[0985] When students answer a test, they enter their answers into the device.

[0986] When submitting an assignment, the text data and images of the assignment are uploaded to the device.

[0987] Input: Test answers, assignment text data, assignment images

[0988] Output: Data transmission request from the device to the server

[0989] Terminal: Sends collected data to the server in real time.

[0990] Specific behavior:

[0991] Generate an API request to the server based on the collected data and POST the data.

[0992] Input: Collected answer data, assignment data

[0993] Output: API request to the server

[0994] Step 2: Receiving and storing data

[0995] Server: Stores the received data in a database, including students' test scores, answers, and images and text of submitted reports and assignments.

[0996] Specific behavior:

[0997] The server executes queries to extract data from incoming API requests and store it in a database.

[0998] Input: Answer data and assignment data sent from the device

[0999] Output: Test results and lesson output stored in the database

[1000] Step 3: Collect and analyze emotion data

[1001] Device: The emotion engine analyzes the student's facial expressions and tone of voice. For example, emotion data is generated from changes in the student's facial expressions and tone of voice.

[1002] Specific behavior:

[1003] The device captures students' facial expressions and voices using the camera and microphone, and calls the emotion engine API to perform emotion analysis.

[1004] Input: Student facial and voice data from camera and microphone

[1005] Output: Parsed emotion data

[1006] Terminal: Transmits the analyzed emotion data to the server.

[1007] Specific behavior:

[1008] Emotion data is sent to the server as a POST request.

[1009] Input: Parsed emotion data

[1010] Output: API request to the server

[1011] Step 4: Receiving and storing emotion data

[1012] Server: Receives emotion data and stores it in a database. Emotion data includes information such as "interested," "understanding," and "confused."

[1013] Specific behavior:

[1014] The server executes a query to store the received emotion data in a database.

[1015] Input: Emotion data sent from the device

[1016] Output: Emotion data stored in a database

[1017] Step 5: Data analysis

[1018] Server: Launches a module that periodically analyzes test results, lesson output, and emotional data. It retrieves the necessary data from the database and analyzes each student's level of understanding, learning progress, and emotional patterns.

[1019] Specific behavior:

[1020] Batch processing is performed to retrieve the necessary data from the database and execute the analysis algorithm.

[1021] Input: Test results, lesson output, and emotional data stored in the database

[1022] Output: Analysis results (personal learning data, learning profile)

[1023] Server: Generates personal learning data based on the analysis results and updates the learning profile.

[1024] Specific behavior:

[1025] Based on the analysis results, a query is executed to generate personal learning data and store it in the database.

[1026] Input: Analysis results

[1027] Output: Updated personal learning data, learning profile

[1028] Step 6: Generate learning content

[1029] Server: Based on the generated personal learning data, a generation AI module is used to generate optimal learning content for students.

[1030] Specific behavior:

[1031] The necessary information is extracted from the student's learning profile and used to create prompts, which are then fed into a generative AI model.

[1032] Input: personal training data, prompt sentence

[1033] Output: Learning content output from the generative AI model

[1034] Server: Selects appropriate learning content for each student and customizes it taking into account emotional data.

[1035] Specific behavior:

[1036] Adjust the generated learning content (e.g., change the difficulty level of the content) based on emotional data.

[1037] Input: Generated learning content, emotion data

[1038] Output: Customized learning content

[1039] Server: Sends the generated learning content to students' devices in real time.

[1040] Specific behavior:

[1041] Send an API response containing customized learning content to the device.

[1042] Input: Customized learning content

[1043] Output: Learning content sent to your device

[1044] Device: Display received learning content and make it accessible to students.

[1045] Specific behavior:

[1046] Display the received learning content on the screen.

[1047] Input: Learning content sent to your device

[1048] Output: Learning content displayed on the screen

[1049] Step 7: Real-time delivery and feedback

[1050] User (student): If a question arises during learning, the user sends the question to the AI ​​from their device.

[1051] Specific behavior:

[1052] Enter your question in the form and press the submit button.

[1053] Input: Question

[1054] Output: Sending query data from the terminal to the server

[1055] Terminal: Sends questions from students to the server in real time.

[1056] Specific behavior:

[1057] Sends the question data to the server as a POST request.

[1058] Input: Question

[1059] Output: API request to the server

[1060] Server: Receives the question, analyzes it, and generates the best answer.

[1061] Specific behavior:

[1062] The question content is analyzed, a corresponding prompt sentence is generated, and input into the AI ​​model.

[1063] Receive and format answers from the AI ​​model.

[1064] Input: Question

[1065] Output: The generated answer

[1066] Server: Sends generated answers and additional assignments to student devices.

[1067] Specific behavior:

[1068] Sends an API response containing the answer to the device.

[1069] Input: Generated Answer

[1070] Output: Answer sent to terminal

[1071] Device: Show students their answers and assignments and keep them learning.

[1072] Specific behavior:

[1073] The received response is displayed on the screen.

[1074] Input: Answer sent to the terminal

[1075] Output: Answers displayed on the screen

[1076] Step 8: Teacher support and monitoring

[1077] User (Teacher): Logs in to a dedicated device or management app and accesses the server.

[1078] Specific behavior:

[1079] Enter your authentication information into the login form and submit it.

[1080] Input: Credentials

[1081] Output: Login request

[1082] Server: Authenticates the teacher access request and grants access to the Teacher Dashboard.

[1083] Specific behavior:

[1084] Generate an authentication token to grant access to the dashboard.

[1085] Input: Login request

[1086] Output: Authentication token

[1087] Server: Retrieves student learning data, progress, and emotional data from the database and displays them on a dashboard.

[1088] Specific behavior:

[1089] Execute SQL queries to retrieve data and display it in a dashboard in real time.

[1090] Input: Authenticated teacher request

[1091] Output: Training data, progress, and sentiment data on a dashboard

[1092] Users (Teachers): Check the dashboard to understand individual students' understanding, progress, and emotional changes.

[1093] Specific behavior:

[1094] Check out the graphs and statistics on the dashboard.

[1095] Input: Data on the dashboard

[1096] Output: Understanding level of understanding, progress, and emotional changes

[1097] User (Teacher): If you notice that a student is falling behind, plan a make-up lesson or additional assignment.

[1098] Specific behavior:

[1099] Enter and set up supplementary lesson plans and additional assignments into the system.

[1100] Input: Supplementary lesson plan details, additional assignment details

[1101] Output: Set supplementary lessons and additional assignments

[1102] User (Teacher): Prepares individual supplementary teaching materials as needed and provides them to students via the server.

[1103] Specific behavior:

[1104] Upload supplementary materials and send them to students' devices.

[1105] Input: Supplementary materials

[1106] Output: Supplementary materials sent to the device

[1107] This enables the system of the present invention to provide optimal learning content according to each student's level of understanding, learning progress, and emotional pattern, and also to support teachers.

[1108] (Application example 2)

[1109] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1110] It is difficult for stores to provide services and product suggestions that meet the needs of each individual customer in real time. Using conventional methods, it is not realistic for store staff to provide optimal service to every customer, and there is a lack of means to quickly and accurately analyze large amounts of customer data. Furthermore, there is a growing need for systems that can grasp customer emotions and behavior in real time and provide services based on that information.

[1111] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for collecting behavioral data and emotional data of customers, means for analyzing the behavioral data and emotional data and providing optimal product proposals and services to individual customers, and means for generating optimal product information and campaign information for each customer using a generative AI model. This makes it possible to propose optimal services and products to each customer individually.

[1112] "Learning data" refers to various data related to student learning, such as student test results, class output, and emotional data.

[1113] "Test results" refers to data such as the scores and answers students receive on tests and exams.

[1114] "Class output" refers to data such as reports, assignments, and presentations submitted by students during class.

[1115] "Emotional data" refers to data on emotions such as interest, understanding, and confusion that are analyzed from students' facial expressions and tone of voice.

[1116] "Level of understanding" is an indicator of how well a student understands the learning content.

[1117] "Learning progress" is an indicator of how far a student has progressed with their learning plan and curriculum.

[1118] "Learning content" refers to content such as teaching materials, assignments, and tests provided to support students' learning.

[1119] "Real time" refers to the time characteristic in which processing is carried out at the exact moment an event occurs.

[1120] "Questions and requests" are actions or requests that indicate a student's doubts or requests for assistance during their studies.

[1121] "Progress" is the progress that indicates how far a student is progressing with a learning activity or assignment.

[1122] "Behavioral data" refers to data on the actions customers take in the store, including their location, route of movement, and direction of gaze.

[1123] "Product proposal" is the act of selecting and recommending an appropriate product to a specific customer.

[1124] "Service" means any support or assistance provided to enhance Customer satisfaction.

[1125] A "generative AI model" is an artificial intelligence model that generates unique information or answers based on input data.

[1126] A "prompt" is an instruction or introductory sentence entered into a generative AI model to prompt it to generate specific information or an answer.

[1127] "Campaign Information" refers to marketing information such as discounts, special offers, and events offered for a specific period or under specific conditions.

[1128] This invention implements a "customer experience improvement system" that responds to the needs of each individual customer in a store.

[1129] 1. Data Collection and Storage

[1130] Devices: Video and audio are collected using smartphones or smart glasses used by customers in the store. Smart glasses have built-in cameras and microphones and collect data in real time. For example, video and audio data is collected when a customer uses smart glasses to browse the shelves.

[1131] Server: The collected data is sent to a cloud server and stored in a database. The server can use AWS cloud services or Google Cloud.

[1132] 2. Collecting and analyzing emotion data

[1133] Device: Using an emotion engine (such as Amazon Rekognition or Google Cloud Vision), the device analyzes the customer's facial expressions and tone of voice to generate emotion data. For example, if a customer is interested in a particular product but confused, emotion data representing interest and confusion will be generated from their facial expressions.

[1134] Server: Emotion data is sent to the cloud server in real time and stored in a database.

[1135] 3. Data Analysis

[1136] Server: The cloud-based analysis module periodically retrieves necessary data from the database and analyzes customer behavior patterns and sentiment data. This analysis uses machine learning algorithms using programming languages ​​such as Python and R.

[1137] Server: Based on the analysis results, the customer's purchasing profile is updated. This profile includes the customer's past purchasing history, behavioral data, and emotional data.

[1138] 4. Creation of Learning Content Services

[1139] Server: A generative AI model (such as GPT-4) generates optimal product information and campaign information based on the prompt text.

[1140] Server: The generated information is sent to your device in real time.

[1141] Terminal: Displays the received information and provides it in a form that can be viewed and used by the customer.

[1142] 5. Real-time delivery and feedback

[1143] Customers: When customers select a product or have questions, they send their feedback or questions to the system via their smartphone or smart glasses.

[1144] Device and Server: Feedback and questions are sent in real time to the cloud server for analysis. Based on the analysis results, the generative AI model generates the optimal answer and delivers it to the customer's device.

[1145] Specific examples

[1146] Example 1: Customer browsing the store on their smartphone

[1147] 1. A customer takes a photo of a product shelf using their smartphone camera.

[1148] 2. The device transmits video and audio in real time to the cloud server.

[1149] 3. The server analyzes the data using an emotion engine and generates emotion data such as interest or confusion.

[1150] 4. The server launches a generative AI model based on the analysis results to create optimal product recommendations.

[1151] 5. The device will display suggested product information and campaigns.

[1152] Example 2: Using smart glasses

[1153] 1. A customer puts on smart glasses and walks through the store.

[1154] 2. The device collects video and audio using the camera and microphone and sends them to a cloud server.

[1155] 3. The server uses an emotion engine to analyze the customer's facial expressions and tone of voice, generating emotion data in real time.

[1156] 4. The server uses the generative AI model to generate optimal product suggestions and campaign information.

[1157] 5. The device displays the suggested information on the smart glasses display.

[1158] Examples of prompt statements

[1159] "When a customer is looking at the shelf facing the camera, the prompt reads:

[1160] "This customer seems interested in this product, but also a little confused. Please provide the following information: product features, customer testimonials, other customer reviews, and recommended outfits."

[1161] "When a customer asks a question, the prompt is:

[1162] "Customers have asked the following question: 'What other colors does this product come in and what are its benefits?'"

[1163] This enables the system to provide individualized information and services to each customer in real time.

[1164] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1165] Step 1:

[1166] User: A customer uses a smartphone or smart glasses in a store to browse the shelves.

[1167] Input: Video and audio data collected by smartphones and smart glasses.

[1168] How it works: The camera captures images of the shelves in the store, and the microphone collects audio. The collected data is then saved in the device in real time.

[1169] Step 2:

[1170] Terminal: Sends collected video and audio data to the cloud server.

[1171] Input: Video and audio data stored on a smartphone or smart glasses.

[1172] Output: Video and audio data sent to the cloud server.

[1173] Specific operation: The terminal uses the HTTPS protocol to send data to the cloud server over the network.

[1174] Step 3:

[1175] Server: The cloud server generates customer emotion data using the emotion engine.

[1176] Input: Video and audio data stored on a cloud server.

[1177] Output: Analyzed emotion data (emotional information such as interest, confusion, etc.).

[1178] What it does: It uses emotion analysis services such as Amazon Rekognition and Google Cloud Vision to analyze the customer's facial expressions and tone of voice from video and audio to generate emotion data.

[1179] Step 4:

[1180] Server: The cloud server integrates and analyzes the behavioral and emotional data to update the customer's purchasing profile.

[1181] Input: Previously generated emotion data and previously stored behavioral data.

[1182] Output: Updated customer purchasing profile.

[1183] What it does: Collates databases, integrates and analyzes historical behavioral and sentiment data, and uses machine learning algorithms in Python and R to analyze customer buying patterns and interests.

[1184] Step 5:

[1185] Server: Generates optimal product suggestions and campaign information using generative AI models.

[1186] Input: Updated customer purchasing profile and prompt text.

[1187] Output: Generated product suggestions and campaign information.

[1188] How it works: A generative AI model (such as GPT-4) generates optimal product suggestions and campaign information based on a prompt such as, "This customer seems interested in this product, but also seems a little confused. Please provide the following information: this product's features, customer reviews, other customer reviews, and recommended outfits."

[1189] Step 6:

[1190] Server: Generated product proposals and campaign information are sent to the customer's device in real time.

[1191] Input: Product suggestions and campaign information generated by the AI ​​model.

[1192] Output: Product offers and campaign information sent to your device.

[1193] What it does: The generated information is sent in real time to your smartphone or smart glasses using the HTTPS protocol.

[1194] Step 7:

[1195] Device: Check and use product information and campaign information suggested by customers.

[1196] Input: Product offers and campaign information sent to your device.

[1197] Output: Display of product information and campaign information that customers can check.

[1198] Specific operation: Product information and campaign information generated by the generative AI model is displayed on the device display, and provided in a form that can be viewed and used by the customer.

[1199] Step 8:

[1200] Users and devices: To help you select products and provide feedback if you have questions.

[1201] Input: Feedback of your questions and suggestions.

[1202] Specific operation: Using a smartphone or smart glasses, customers enter feedback or questions and send them to a cloud server.

[1203] Step 9:

[1204] Server: Generates optimal answers based on feedback and questions, and delivers them to your device in real time.

[1205] Input: Feedback and questions submitted by customers.

[1206] Output: Optimized answer and / or additional information.

[1207] What it does: Uses a generative AI model to generate optimal answers based on customer questions and feedback, and sends them to the device in real time. For example, it uses a prompt like, "A customer asked the following question: 'What other colors does this product come in and what are its benefits?'"

[1208] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1209] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1210] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1211] [Third embodiment]

[1212] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1213] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[1214] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1215] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1216] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1217] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1218] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1219] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1220] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1221] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1222] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1223] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1224] This invention relates to a learning system that uses generative AI to provide optimal learning content for each student. Specifically, it describes a method that incorporates students' test results and class output, generates and analyzes personal learning data, and provides learning content in real time based on that data.

[1225] Data collection and storage

[1226] Devices: When students take tests or submit assignments during class, their devices (PCs, tablets, etc.) collect this data and send it to the server.

[1227] Server: Stores received test results and lesson output in a database. For example, it stores students' test scores and answers, as well as images and text data of submitted reports and assignments.

[1228] Data analysis

[1229] Server: The analysis module is launched periodically or when new data is added. The server retrieves test results and lesson outputs for each student in the database and analyzes them.

[1230] Example: Analyze the accuracy rate, answer time, and patterns of incorrect answers of math problems solved by Student A to generate a learning profile. For example, if it turns out that the accuracy rate for a specific type of problem is low, provide learning content that reinforces that type of problem.

[1231] Generating learning content

[1232] Server: Based on the generated learning profile, the server uses generative AI to generate optimal learning content for each student. The content can come in a variety of formats, including text, images, videos, and interactive questions.

[1233] Example: For a type of math problem that Student A struggles with, we generate related explanatory videos and practice problems and provide them as a single learning package.

[1234] Real-time delivery and feedback

[1235] Server: The generated content is sent to students' devices in real time, allowing them to view the content on their own devices and progress with their studies.

[1236] Device: If a student has a question while studying, they can immediately input it into the AI. For example, they can ask, "Why is the answer to this problem like this?"

[1237] Server: Receives questions, analyzes them in real time, generates appropriate answers and additional questions, and sends them to students' devices. Students receive these and continue their learning.

[1238] Teacher Support and Monitoring

[1239] User (Teacher): Teachers access the server using a dedicated device or management app.

[1240] Server: Provides a dashboard for teachers, displaying each student's learning progress and level of understanding, allowing teachers to see at a glance how each student is learning.

[1241] Example: A teacher determines that Student A's learning progress is falling behind and plans additional follow-up lessons. If necessary, individual supplementary learning materials can be prepared and provided to the student via the server.

[1242] This will provide each student with the optimal learning environment, enabling high-quality education while reducing the workload of teachers.

[1243] The processing flow will be explained below.

[1244] Step 1:

[1245] User (Teacher): Uploads tests and supplementary materials to be used in class to the server.

[1246] Server: Organizes the uploaded data and links it to the corresponding lesson plans.

[1247] Device: Prepares students to receive tests and supplementary materials.

[1248] Step 2:

[1249] User (student): Takes a test or submits an assignment during class.

[1250] Device: Collects student answers and submissions and sends them to the server in real time. Records student answers digitally.

[1251] Server: Receives the transmitted data and immediately stores it in the database. Backs up the stored data.

[1252] Step 3:

[1253] Server: Launches the data analysis module periodically or when new student data is added.

[1254] Server: Retrieves student test results and lesson output from the database.

[1255] Server: Analyzes the acquired data and calculates each student's level of understanding, learning progress, and patterns of incorrect answers.

[1256] Server: Based on the analysis results, personal learning data is generated and the learning profile for each student is updated.

[1257] Step 4:

[1258] Server: Based on the updated personal learning data, the generative AI module is launched.

[1259] Server: Selects and combines resources (text, images, videos, etc.) containing learning content appropriate for each student.

[1260] Server: Sends the generated learning content to students' devices in real time.

[1261] Device: Display received learning content and make it accessible to students.

[1262] Step 5:

[1263] User (student): If a question arises during learning, the user sends the question to the AI ​​from their device.

[1264] Terminal: Sends questions from students to the server in real time.

[1265] Server: Receives the question and starts the process of analyzing it and generating the best answer.

[1266] Server: Sends generated answers and additional assignments to student devices.

[1267] Device: Show students answers and assignments so they can continue learning.

[1268] Step 6:

[1269] User (Teacher): Logs in to a dedicated device or management app and accesses the server.

[1270] Server: Authenticates the teacher access request and grants access to the Teacher Dashboard.

[1271] Server: Retrieves student learning data, progress, and understanding from the database and displays them on a dashboard.

[1272] User (Teacher): Check the dashboard to understand each student's understanding and progress.

[1273] User (Teacher): Sees that Student A is falling behind and acts on the data to plan make-up lessons and additional assignments.

[1274] Example 1

[1275] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1276] With conventional learning systems, it was difficult to provide individualized learning content that corresponded to each student's level of understanding and learning progress. It also required a great deal of effort for teachers to grasp each student's progress and level of understanding, making it difficult to provide appropriate guidance and feedback to each student in real time. Furthermore, there was a lack of a mechanism for immediate response when students had questions during their studies.

[1277] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1278] In this invention, the server includes means for collecting assessment results and lesson outcomes to generate individual student learning data, means for analyzing the learning data and grasping each student's level of understanding and learning progress, means for generating optimal learning content for each student based on the generated learning data using a generative AI model, means for providing the learning content to students in real time, means for responding to questions and requests from students in real time, means for teachers to check students' progress and level of understanding, and means for teachers to plan additional learning support and provide supplementary materials to students. This makes it possible to provide learning content appropriate for each student, resolve questions in real time, and provide effective educational support while reducing the burden on teachers.

[1279] "Individual student learning data" refers to digital data generated by each student through lessons and assessment activities, and is information that indicates the student's level of understanding and progress.

[1280] "Evaluation results" refers to the scores and evaluation content obtained through learning assessment methods such as tests and assignments.

[1281] "Class outcomes" refers to the results of learning activities that students undertake during class or extracurricular activities, and include a variety of formats such as text, images, and videos.

[1282] "Analysis" refers to the process of evaluating students' learning patterns and comprehension based on collected data and creating individual profiles.

[1283] "Understanding" refers to an indicator of how well a student understands a particular learning content.

[1284] "Learning progress" refers to an indicator that shows how far a student is progressing along the learning curriculum.

[1285] A "generative AI model" refers to an algorithm or model that uses AI technology to automatically generate specific learning content based on input data.

[1286] "Learning content" refers to educational content provided for students to learn, and includes various formats such as text, images, videos, and interactive questions.

[1287] "Real-time" refers to a state in which data is exchanged and processed immediately and is available without waiting.

[1288] "Teacher burden" refers to the time and effort required for teachers to check students' progress, provide feedback, and provide individual instruction.

[1289] "Instructional aids" refer to activities and tools that support the learning process and enable students to learn effectively.

[1290] "Supplemental materials" refers to additional learning materials or assignments provided to deepen students' understanding.

[1291] This invention relates to a learning system that uses a generative AI model to provide optimal learning content for each student. Specifically, it shows a method for incorporating student assessment results and class performance, generating and analyzing personalized learning data, and providing learning content in real time based on that data.

[1292] Data collection and storage

[1293] Device:

[1294] When students take tests or submit assignments during class, devices such as computers and tablets collect this data and send it to a server. For example, students answer test questions and save the results on their devices. The collected data is then sent to a server.

[1295] server:

[1296] The received evaluation results and lesson outcomes are stored in a database. Specifically, data such as student IDs, test scores, answers, and submitted reports are stored. After the data is stored in the database, the conditions for launching the analysis module are checked.

[1297] Data analysis

[1298] server:

[1299] When new data is added, the analysis module is triggered. The server collects data for each student in the database and passes it to the analysis engine. For example, analyzing math test results, it finds out that Student A has a low success rate in certain question types and generates a learning profile.

[1300] Generating learning content

[1301] server:

[1302] Based on the generated learning profile, specific learning content is input as a prompt to the generative AI model. For example, a prompt such as "Please generate explanatory videos and practice problems for math problem type X for Student A" is provided to the generative AI model. The text, images, videos, problem sets, etc. obtained from the generative AI model are compiled into a single learning package, including related explanatory videos and explanations of the answers to the problems.

[1303] As a specific example, if Student A has difficulty with a particular type of math problem, the generative AI model will generate explanatory videos and practice problems based on the prompt text and provide them as a learning package.

[1304] Example prompt sentence:

[1305] Please generate explanatory videos and practice questions for math problem type X based on the test results of Student A. I would like the format to be a combination of text, images, and videos.

[1306] Real-time delivery and feedback

[1307] server:

[1308] The generated learning content is sent to students' devices in real time. Data communication protocols are used to ensure reliable data delivery. Students can then check the content on their own devices and continue their learning.

[1309] Device:

[1310] If a student has a question while studying, they can immediately input it into the AI, for example, "Why is the answer to this problem like this?" The question is then sent to the server.

[1311] server:

[1312] Analyze the question and generate appropriate answers or provide additional practice questions, possibly using a generative AI model, and send the answers or additional questions to the student's device.

[1313] Teacher Support and Monitoring

[1314] User (Teacher):

[1315] Teachers access the server using dedicated devices or a management app. They can use a dashboard to visually monitor each student's progress and understanding. Teachers can use this information to plan additional lessons or provide supplementary materials for specific students.

[1316] server:

[1317] The teacher dashboard displays learning data for each student and makes it easily accessible to teachers. For example, areas of low understanding are visually highlighted. Based on this information, teachers can input follow-up instructions into the server and send them to students' devices.

[1318] This system provides each student with an optimal learning environment, enables real-time question resolution, and provides effective educational support while reducing the burden on teachers.

[1319] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1320] Step 1: Collect data

[1321] Input: Students use their devices to take tests or submit assignments.

[1322] Specific operation: Students answer test questions and save their answers and assignment data on their devices. This includes text data, image data, video data, etc.

[1323] Output: The device sends the collected data to the server.

[1324] Step 2: Receiving and storing data

[1325] Input: Data such as test results and class results sent from the device.

[1326] Specific operation: The server receives the student's evaluation results and lesson results and stores them in a database, including student IDs, test scores, answers, reports, etc.

[1327] Output: Data per student stored in a database.

[1328] Step 3: Analyze the data

[1329] Input: New student data stored in the database.

[1330] How it works: When the server detects new data, it launches the analysis module. The analysis engine collects data for each student in the database and evaluates the student's understanding and learning progress.

[1331] Output: The resulting learning profile.

[1332] Step 4: Generate a learning profile

[1333] Input: Analysis results for each student from the analysis module.

[1334] Specific operation: The server generates a learning profile and creates detailed learning data for each student, including their individual weaknesses and strengths, such as the percentage of correct answers to math problems, the time it takes to answer them, and error patterns.

[1335] Output: The generated learning profile.

[1336] Step 5: Generate learning content

[1337] Input: Learning profile.

[1338] Specific operation: The server inputs a prompt based on the learning profile into the generative AI model. For example, "Please generate an explanatory video and practice questions for math problem type X for Student A." The generative AI model analyzes the prompt and automatically generates appropriate learning content (text, images, videos, interactive questions, etc.).

[1339] Output: The generated learning content.

[1340] Step 6: Package your learning content

[1341] Input: Learning content from a generative AI model.

[1342] What it does: The server compiles the generated content into a learning package, including related instructional videos and explanations of the answers to the questions.

[1343] Output: Packaged learning content.

[1344] Step 7: Real-time serving

[1345] Input: Packaged learning content.

[1346] Specific operation: The server transmits the learning package to the student's device in real time, using a data communication protocol to ensure the data is delivered.

[1347] Output: The learning content displayed on the student's device.

[1348] Step 8: Answering student questions

[1349] Input: Questions from students during learning.

[1350] Specific operation: When a student has a question while studying, they type the question on their device and send it to the server. For example, "Why is the answer to this problem like this?"

[1351] Output: The question sent to the server.

[1352] Step 9: Parsing the question and generating an answer

[1353] Input: The student question sent to the server.

[1354] What it does: The server analyzes the question and generates appropriate answers or provides additional practice questions, possibly using a generative AI model.

[1355] Output: Generated answers and additional practice questions.

[1356] Step 10: Provide your answers

[1357] Input: Generated answers and additional practice questions.

[1358] Specific operation: The server sends answers and additional practice questions to the student's device, where the student receives them and uses them to help with their studies.

[1359] Output: Answers and exercises displayed on student devices.

[1360] Step 11: Teacher Support and Monitoring

[1361] Input: Access from a dedicated device or management app.

[1362] How it works: Teachers access the server and use a dedicated dashboard to check students' progress and understanding, which displays each student's learning profile and progress.

[1363] Output: Teacher progress review and lesson plan.

[1364] Step 12: Plan additional follow-up lessons

[1365] Input: Student progress data viewed through the dashboard.

[1366] What it does: Teachers plan additional lessons and supplementary materials for specific students through the dashboard, and follow-up instructions are entered into the server and sent to student devices.

[1367] Output: Follow-up instructions are sent to the student.

[1368] Through this series of steps, learning content tailored to each student is provided, questions are answered in real time, and teachers can provide effective educational support.

[1369] (Application example 1)

[1370] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1371] Conventional learning systems have difficulty providing optimal learning content in real time based on each student's level of understanding and learning progress. They also lack a system that can provide immediate answers when students have questions. Furthermore, there are insufficient means for teachers to grasp students' learning progress in detail and provide appropriate follow-up. As a result, an environment that maximizes the learning effect of each student in personalized education has not been established.

[1372] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1373] In this invention, the server includes: means for collecting students' test results and lesson output; means for analyzing the learning data and grasping each student's level of understanding and learning progress; means for generating learning content optimized for each student based on the generated learning data; means for providing the learning content to students in real time; means for responding to students' questions and requests in real time; means for teachers to check students' progress and understanding; means for connecting to smart devices used in brick-and-mortar educational facilities and providing learning content and feedback in real time; means for generating learning content using a generative AI model based on a specific learning profile; and means for generating prompt sentences to generate appropriate feedback on topics that the student struggles with. This enables the provision of optimal learning content for each student. Furthermore, real-time responses can immediately resolve students' questions and improve learning efficiency. Furthermore, providing teachers with a means for accurately grasping students' learning progress and providing effective follow-up can improve the quality of personalized education.

[1374] "Individual student learning data" refers to data on individual students' test results and class output.

[1375] "Learning data analysis" refers to the process of processing and analyzing collected data to understand each student's level of understanding and learning progress.

[1376] "Learning Content" refers to optimal learning materials provided to students using generative AI models, and may include formats such as text, images, and videos.

[1377] "Real-time delivery" refers to the instant delivery of generated learning content to students.

[1378] "Student Q&A" refers to the process of providing immediate and appropriate answers or additional learning content in response to student questions or requests.

[1379] "Teacher Monitoring Tools" refers to tools specifically designed for teachers to monitor and manage student progress and understanding.

[1380] "Smart device connectivity" refers to linking the system with devices such as smartphones, tablets, and smart glasses used in brick-and-mortar educational facilities.

[1381] A "generative AI model" refers to an artificial intelligence model that automatically generates optimal learning content and feedback based on input data.

[1382] A "prompt" is a sentence that constructs instructions or questions for a generative AI model based on a specific learning profile.

[1383]

[1384] This invention is a smart education support system that utilizes generative AI models to provide optimal learning content for individual students in brick-and-mortar educational facilities. A specific embodiment of this system is described below.

[1385] Hardware Configuration

[1386] It uses a server and smart devices such as smartphones, tablets, and smart glasses. The server has the central function of collecting, analyzing, generating, and distributing data. The smart devices are used by students and teachers as an interface to receive learning content and feedback in real time.

[1387] Software Configuration

[1388] It uses the Python programming language, the generative AI model API provided by OpenAI, and a database management system.

[1389] System action

[1390] The server collects test results and class output from students' devices and stores them in a database. The collected data is periodically analyzed on the server. As a result of the analysis, each student's level of understanding and learning progress is identified and an individual learning profile is generated.

[1391] Based on this learning profile, the server uses a generative AI model to generate optimal learning content. At this time, a prompt sentence is input to the AI ​​model. For example, the following prompt sentence is generated:

[1392] Example prompt sentence:

[1393] Based on the math test results, Student A's most common mistakes are solving equations and basic use of trigonometric functions. Create learning content to deepen their understanding of these topics.

[1394] The generated learning content is delivered to students' devices in real time. Students can view this content and continue their studies via smartphones, tablets, or smart glasses. If students have questions while studying, they can input them in real time, and the generative AI model will instantly generate an answer, which will then be delivered to the student's device.

[1395] Teachers can also use the system to access a dedicated dashboard that allows them to see students' progress and understanding in real time, allowing them to plan individual follow-ups and provide additional materials or assignments if necessary.

[1396] For example, if test analysis reveals that Student A is struggling with fraction calculations, the teacher can provide supplementary materials on fraction calculations. This system provides an optimal learning environment for each student, improving the quality of personalized education.

[1397] As described above, by implementing the present invention, it becomes possible to realize real-time individualized learning and teacher support, which was difficult with conventional systems.

[1398] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1399]

[1400] Step 1:

[1401] The server collects test results and class output (text data, image data, video data) from students' devices. This allows the collection of student learning data. The test results and class output data taken by students are used as input. As output, the data is stored in a database on the server.

[1402] Step 2:

[1403] The server periodically launches the data analysis module to retrieve the students' learning data stored in the database. The data analysis module analyzes this learning data to understand each student's level of understanding and learning progress. The collected learning data is used as input for this step. The output is a learning profile for each student.

[1404] Step 3:

[1405] The server creates a prompt to generate specific learning content based on the generated learning profile. For example, it generates a prompt such as, "Based on the results of the math test, the questions that Student A often gets wrong are: solving equations, basic use of trigonometric functions. Please generate learning content to deepen his understanding of these topics." The student's learning profile is used as input. The generated prompt is used as output.

[1406] Step 4:

[1407] The server sends the generated prompt sentences to a generative AI model, which then generates optimal learning content based on the prompt sentences. For example, videos and supplementary learning materials are generated to deepen understanding. The prompt sentences are used as input, and the learning content is generated as output.

[1408] Step 5:

[1409] The generated learning content is sent from the server to the student's device in real time. The student checks this content on their device and proceeds with their learning. The generated learning content is used as input for this step. The output is the content displayed on the student's device.

[1410] Step 6:

[1411] When a device receives a question or request from a student, the question or request is sent to the server. The server uses a generative AI model to generate an appropriate answer for the received question and sends it back to the student's device. The question or request from the student is used as input. The answer is displayed on the student's device as output.

[1412] Step 7:

[1413] Teachers access a dedicated dashboard to check students' learning progress and comprehension data provided by the server. If necessary, teachers create additional learning materials and assignments and provide them to students' devices via the server. Student learning profiles and progress data are used as input. Additional learning materials and assignments are provided to students as output.

[1414] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1415] This invention relates to a learning system that uses generative AI and an emotion engine to provide optimal learning content for each student. Specifically, it describes a method for incorporating students' test results, class output, and emotional data, generating and analyzing personalized learning data, and providing learning content in real time based on that data.

[1416] Data collection and storage

[1417] Devices: Students take tests and submit assignments during class. The devices (computers, tablets, etc.) collect this data and send it to the server.

[1418] Server: Stores received test results and lesson output in a database. For example, it stores students' test scores and answers, as well as images and text data of submitted reports and assignments.

[1419] Collecting Emotional Data

[1420] Device: The emotion engine analyzes the student's facial expression, tone of voice, and input content to generate emotion data. For example, it senses changes in facial expression and voice when a student is facing a problem.

[1421] Server: Receives emotion data and stores it in a database. Emotion data includes information such as "interested," "understanding," and "confused."

[1422] Data analysis

[1423] Server: Launches modules that periodically analyze test results, lesson output, and sentiment data.

[1424] Server: Retrieves necessary learning data from the database and analyzes each student's level of understanding, learning progress, and emotional patterns.

[1425] Server: Based on the analysis results, generate personal learning data and update each student's learning profile.

[1426] Generating learning content

[1427] Server: Based on the generated personal learning data, the generative AI module is launched.

[1428] Server: Selects appropriate learning content for each student and customizes it taking into account emotional data.

[1429] Server: Sends the generated learning content to students' devices in real time.

[1430] Device: Display received learning content and make it accessible to students.

[1431] Real-time delivery and feedback

[1432] User (student): If a question arises during learning, the user sends the question to the AI ​​from their device.

[1433] Terminal: Sends questions from students to the server in real time.

[1434] Server: Receives the question and starts the process of analyzing it and generating the best answer.

[1435] Server: Sends generated answers and additional assignments to student devices.

[1436] Device: Show students answers and assignments so they can continue learning.

[1437] Teacher Support and Monitoring

[1438] User (Teacher): Accesses the server using a dedicated device or management app.

[1439] Server: Authenticates the teacher access request and grants access to the Teacher Dashboard.

[1440] Server: Retrieves student learning data, progress, comprehension, and emotional data from the database and displays them on a dashboard.

[1441] Users (Teachers): Check the dashboard to understand individual students' understanding, progress, and emotional changes.

[1442] User (Teacher): If the teacher determines that a student's progress is falling behind, he / she plans supplementary lessons and additional assignments. If necessary, he / she prepares individual supplementary materials and provides them to the student via the server.

[1443] Specific examples

[1444] Example 1: A student takes a test

[1445] 1. User (student): Take a test during class.

[1446] 2. Terminal: Records students' answers and transmits them to the server in real time.

[1447] 3. Server: Receives and stores data.

[1448] 4. Server: The emotion engine analyzes the student's facial expressions and tone of voice to generate emotion data during the test.

[1449] 5. Server: Analyzes the test results and emotion data and updates the personal training data.

[1450] Example 2: When a student asks an AI a question

[1451] 1. User (student): If a question arises during learning, the user sends the question to the AI ​​from their device.

[1452] 2. Terminal: Sends questions from students to the server in real time.

[1453] 3. Server: Receives the question and starts the process of analyzing it and generating the best answer.

[1454] 4. Server: Sends the generated answers to the students' devices.

[1455] 5. Device: Show students their answers and continue learning.

[1456] Teachers checking student progress

[1457] 1. User (teacher): Log in to a dedicated device or the management app and access the server.

[1458] 2. Server: Authenticates the teacher access request and grants access to the Teacher Dashboard.

[1459] 3. Server: Retrieves students' learning data, progress, and emotional data from the database and displays them on the dashboard.

[1460] 4. Users (Teachers): Check the dashboard to understand individual students' understanding, progress, and emotional changes.

[1461] 5. User (Teacher): If you notice that a student is falling behind, plan a make-up lesson or additional assignment.

[1462] The processing flow will be explained below.

[1463] Step 1:

[1464] User (Teacher): Uploads tests and supplementary materials to be used in class to the server. Materials include workbooks, videos, slides, etc.

[1465] Server: Organizes uploaded data by format and links it to the corresponding lesson plans.

[1466] Devices: Student devices are prepared and configured to access materials.

[1467] Step 2:

[1468] User (student): Takes tests or submits assignments during class. Study the designated material on the device.

[1469] Terminal: Collects student answers and submissions (text, images, audio, etc.) and sends them to the server in real time.

[1470] Server: Receives the transmitted data and immediately stores it in a database, along with recording the timestamp information.

[1471] Step 3:

[1472] Devices: Data is collected using sensors that detect students' facial expressions and tone of voice, for example, cameras and microphones that analyze real-time video and audio.

[1473] Server: Analyzes facial expressions, tone of voice, and input content collected by the emotion engine to generate emotional data.

[1474] Server: Receives emotional data and stores it in a database. The emotional data includes students' reactions (happiness, confusion, etc.).

[1475] Step 4:

[1476] Server: Launches the data analysis module periodically or when new data is added.

[1477] Server: Retrieves students' test results, lesson output, and emotion data from the database.

[1478] Server: Comprehensively analyzes the acquired data and calculates each student's level of understanding, learning progress, and emotional patterns. For example, it evaluates the student's level of concentration when working on a specific task.

[1479] Server: Based on the analysis results, generate personal learning data and update each student's learning profile.

[1480] Step 5:

[1481] Server: Based on the updated personal learning data, the generative AI module is launched.

[1482] Server: Selects appropriate learning content for each student and customizes it taking into account emotional data, for example, including explanatory videos or additional practice questions for difficult assignments.

[1483] Server: Sends the generated learning content to students' devices in real time.

[1484] Device: Display received learning content and make it accessible to students.

[1485] Step 6:

[1486] User (student): If a question arises during learning, the user sends the question to the AI ​​from their device.

[1487] Terminal: Sends questions from students to the server in real time.

[1488] Server: Receives the question and starts the process of analyzing it and generating the best answer.

[1489] Server: Sends generated answers and additional assignments to student devices.

[1490] Device: Show students their answers and assignments and keep them learning.

[1491] Step 7:

[1492] User (Teacher): Accesses the server using a dedicated device or management app.

[1493] Server: Authenticates the teacher access request and grants access to the Teacher Dashboard.

[1494] Server: Retrieves student learning data, progress, comprehension, and emotion data from the database and displays them on a dashboard.

[1495] Users (Teachers): Check the dashboard to understand individual students' understanding, progress, and emotional changes.

[1496] User (Teacher): If it is determined that Student A's progress is falling behind, the teacher plans supplementary lessons and additional assignments. If necessary, the teacher conducts individual follow-up based on the emotional data.

[1497] This configuration allows a learning system incorporating an emotion engine to comprehensively evaluate not only a student's level of understanding and progress, but also their emotional changes, and provide optimal learning content in real time. This also makes it easier for teachers to grasp the psychological state of their students, enabling more detailed instruction.

[1498] Example 2

[1499] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1500] Conventional learning systems have the problem that it is difficult to accurately grasp each student's level of understanding and learning progress, and they can only provide uniform learning content. Furthermore, they are unable to provide learning content that takes into account students' emotional data, which affects students' motivation and learning efficiency. As a result, they are unable to provide an optimal learning experience for each student, and there are issues with not being able to provide effective learning support.

[1501] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1502] In this invention, the server includes means for collecting students' test results and lesson output to generate individual student learning data, means for analyzing the learning data and students' emotional data to grasp each student's level of understanding, learning progress, and emotional patterns, and means for generating optimal learning content for each student based on the generated learning data and emotional data. This makes it possible to provide optimal learning content based not only on each student's level of understanding and learning progress, but also on their emotional patterns.

[1503] "Individual student learning data" refers to data that records in detail each student's learning situation, achievements, and level of understanding.

[1504] "Test results" are data that indicate the scores and answers given by students on tests they have taken.

[1505] "Class output" refers to data that includes assignments, reports, and notes that students submit during and after class.

[1506] "Emotional data" is data about emotions obtained by analyzing students' facial expressions, tone of voice, content of speech, etc.

[1507] A "learning profile" is individual data that compiles each student's level of understanding, learning progress, emotional patterns, etc.

[1508] A "generative AI module" is a program or system that uses artificial intelligence technology to generate optimal learning content for students.

[1509] "Learning content" refers to the learning materials and assignments provided to students for learning, and includes various formats such as text, images, and videos.

[1510] The "Teacher Dashboard" is a management screen that allows teachers to check students' learning status, level of understanding, progress, and emotional patterns in real time.

[1511] "Providing in real time" means providing learning content and feedback instantly in line with the current learning environment and situation.

[1512] "Feedback" is information that includes evaluations of students' learning activities, advice, and areas for improvement.

[1513] "Additional teaching materials and assignments" refers to new learning materials and assignments that teachers add to reinforce what students have learned.

[1514] MODE FOR CARRYING OUT THE INVENTION

[1515] This invention is a learning system that provides optimal learning content to students, utilizing a generative AI model and an emotion engine. The system collects students' test results, class output, and emotion data, generates and analyzes individual learning data, and provides learning content in real time based on that data.

[1516] Hardware and software used

[1517] Device:

[1518] Student computers or tablets

[1519] Camera and microphone (for collecting emotional data)

[1520] server:

[1521] Database management systems (e.g., MySQL, PostgreSQL)

[1522] Emotion analysis API (e.g. Microsoft Emotion API, Google Cloud Vision)

[1523] Generative AI modules (e.g., OpenAI GPT-4, Google BERT)

[1524] System Description

[1525] Data collection and storage

[1526] Device: Students take tests and submit assignments during class. The device collects this data (test answers and assignment images / text data) and sends it to the server.

[1527] Server: Stores the received data in a database. Collected data includes students' test scores, answers, submitted papers, and images and text of assignments.

[1528] Specific examples of behavior:

[1529] Students take math tests and upload the results from their devices to the server.

[1530] The server stores the received data in a database for subsequent analysis.

[1531] Collecting Emotional Data

[1532] Device: The emotion engine analyzes the student's facial expressions and tone of voice. For example, emotion data is generated from changes in the student's facial expressions and tone of voice.

[1533] Server: Receives the analyzed emotion data and stores it in a database. The emotion data includes information such as "interested," "understanding," and "confused."

[1534] Specific examples of behavior:

[1535] If a student shows a confused expression during a test, the emotion engine analyzes the facial expression data and generates emotion data for "confusion."

[1536] The generated emotion data is sent to a server and stored in a database.

[1537] Data analysis

[1538] Server: Launches a module that periodically analyzes test results, lesson output, and emotional data. It retrieves the necessary data from the database and analyzes each student's level of understanding, learning progress, and emotional patterns.

[1539] Server: Based on the analysis results, generate personal learning data and update the learning profile.

[1540] Specific examples of behavior:

[1541] Student A's level of understanding is evaluated based on his / her test results, and his / her learning profile is updated by integrating it with lesson output and emotional data.

[1542] Generating learning content

[1543] Server: Based on the generated personal learning data, a generation AI module is used to generate optimal learning content for students.

[1544] Server: Selects appropriate learning content for each student and customizes it taking into account emotional data.

[1545] Server: Sends the generated learning content to students' devices in real time, and the devices display the content.

[1546] Specific examples of behavior:

[1547] Prompt the generative AI model with learning content that includes gentle explanations on topics that Student A has low understanding of.

[1548] The generated content is sent to Student A's device and displayed on the screen.

[1549] Real-time delivery and feedback

[1550] User (student): If a question arises during learning, the user sends the question to the AI ​​from their device.

[1551] Terminal: Sends questions from students to the server in real time.

[1552] Server: Receives the question, analyzes it, and generates the best answer.

[1553] Server: Sends the generated answers and additional assignments to the student's device, which displays them.

[1554] Examples of prompts:

[1555] Question: "Why does water boil at 100 degrees?"

[1556] Answer: "When water reaches 100°C at standard atmospheric pressure, its vapor pressure becomes equal to atmospheric pressure and it begins to boil."

[1557] Teacher Support and Monitoring

[1558] User (Teacher): Accesses the server by logging in to a dedicated device or the management app.

[1559] Server: Authenticates the teacher access request and grants access to the Teacher Dashboard.

[1560] Server: Retrieves students' learning data, progress, and emotional data from the database and displays them on a dashboard.

[1561] Users (Teachers): Check the dashboard to understand each student's level of understanding, progress, and emotional changes, and plan supplementary lessons or additional assignments as needed.

[1562] Example prompt

[1563] Examples of prompt sentences are shown below.

[1564] Sample prompts: Examples of inputs to the AI ​​model that analyzes the content of students' answers

[1565] text

[1566] Analyze the students' test answers below and evaluate their level of understanding and emotional response.

[1567] Test Answers:

[1568] 1. Q1: The answer is "A"

[1569] 2. Q2: The answer is "B"

[1570] ...

[1571] Emotional Data:

[1572] When answering Q1: Facial expression "confused"

[1573] When answering Q2: Facial expression "understood"

[1574] By implementing the system based on this flow of specific data collection, analysis, generation and provision of learning content, and feedback, it is possible to maximize the learning efficiency of each student.

[1575] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1576] Program processing flow

[1577] Step 1: Collect and send data

[1578] Terminal: During class, students take tests and submit assignments. The terminal collects this input data (test answers and assignment images and text data).

[1579] Specific behavior:

[1580] When students answer a test, they enter their answers into the device.

[1581] When submitting an assignment, the text data and images of the assignment are uploaded to the device.

[1582] Input: Test answers, assignment text data, assignment images

[1583] Output: Data transmission request from the device to the server

[1584] Terminal: Sends collected data to the server in real time.

[1585] Specific behavior:

[1586] Generate an API request to the server based on the collected data and POST the data.

[1587] Input: Collected answer data, assignment data

[1588] Output: API request to the server

[1589] Step 2: Receiving and storing data

[1590] Server: Stores the received data in a database, including students' test scores, answers, and images and text of submitted reports and assignments.

[1591] Specific behavior:

[1592] The server executes queries to extract data from incoming API requests and store it in a database.

[1593] Input: Answer data and assignment data sent from the device

[1594] Output: Test results and lesson output stored in the database

[1595] Step 3: Collect and analyze emotion data

[1596] Device: The emotion engine analyzes the student's facial expressions and tone of voice. For example, emotion data is generated from changes in the student's facial expressions and tone of voice.

[1597] Specific behavior:

[1598] The device captures students' facial expressions and voices using the camera and microphone, and calls the emotion engine API to perform emotion analysis.

[1599] Input: Student facial and voice data from camera and microphone

[1600] Output: Parsed emotion data

[1601] Terminal: Transmits the analyzed emotion data to the server.

[1602] Specific behavior:

[1603] Emotion data is sent to the server as a POST request.

[1604] Input: Parsed emotion data

[1605] Output: API request to the server

[1606] Step 4: Receiving and storing emotion data

[1607] Server: Receives emotion data and stores it in a database. Emotion data includes information such as "interested," "understanding," and "confused."

[1608] Specific behavior:

[1609] The server executes a query to store the received emotion data in a database.

[1610] Input: Emotion data sent from the device

[1611] Output: Emotion data stored in a database

[1612] Step 5: Data analysis

[1613] Server: Launches a module that periodically analyzes test results, lesson output, and emotional data. It retrieves the necessary data from the database and analyzes each student's level of understanding, learning progress, and emotional patterns.

[1614] Specific behavior:

[1615] Batch processing is performed to retrieve the necessary data from the database and execute the analysis algorithm.

[1616] Input: Test results, lesson output, and emotional data stored in the database

[1617] Output: Analysis results (personal learning data, learning profile)

[1618] Server: Generates personal learning data based on the analysis results and updates the learning profile.

[1619] Specific behavior:

[1620] Based on the analysis results, a query is executed to generate personal learning data and store it in the database.

[1621] Input: Analysis results

[1622] Output: Updated personal learning data, learning profile

[1623] Step 6: Generate learning content

[1624] Server: Based on the generated personal learning data, a generation AI module is used to generate optimal learning content for students.

[1625] Specific behavior:

[1626] The necessary information is extracted from the student's learning profile and used to create prompts, which are then fed into a generative AI model.

[1627] Input: personal training data, prompt sentence

[1628] Output: Learning content output from the generative AI model

[1629] Server: Selects appropriate learning content for each student and customizes it taking into account emotional data.

[1630] Specific behavior:

[1631] Adjust the generated learning content (e.g., change the difficulty level of the content) based on emotional data.

[1632] Input: Generated learning content, emotion data

[1633] Output: Customized learning content

[1634] Server: Sends the generated learning content to students' devices in real time.

[1635] Specific behavior:

[1636] Send an API response containing customized learning content to the device.

[1637] Input: Customized learning content

[1638] Output: Learning content sent to your device

[1639] Device: Display received learning content and make it accessible to students.

[1640] Specific behavior:

[1641] Display the received learning content on the screen.

[1642] Input: Learning content sent to your device

[1643] Output: Learning content displayed on the screen

[1644] Step 7: Real-time delivery and feedback

[1645] User (student): If a question arises during learning, the user sends the question to the AI ​​from their device.

[1646] Specific behavior:

[1647] Enter your question in the form and press the submit button.

[1648] Input: Question

[1649] Output: Sending query data from the terminal to the server

[1650] Terminal: Sends questions from students to the server in real time.

[1651] Specific behavior:

[1652] Sends the question data to the server as a POST request.

[1653] Input: Question

[1654] Output: API request to the server

[1655] Server: Receives the question, analyzes it, and generates the best answer.

[1656] Specific behavior:

[1657] The question content is analyzed, a corresponding prompt sentence is generated, and input into the AI ​​model.

[1658] Receive and format answers from the AI ​​model.

[1659] Input: Question

[1660] Output: The generated answer

[1661] Server: Sends generated answers and additional assignments to student devices.

[1662] Specific behavior:

[1663] Sends an API response containing the answer to the device.

[1664] Input: Generated Answer

[1665] Output: Answer sent to terminal

[1666] Device: Show students their answers and assignments and keep them learning.

[1667] Specific behavior:

[1668] The received response is displayed on the screen.

[1669] Input: Answer sent to the terminal

[1670] Output: Answers displayed on the screen

[1671] Step 8: Teacher support and monitoring

[1672] User (Teacher): Logs in to a dedicated device or management app and accesses the server.

[1673] Specific behavior:

[1674] Enter your authentication information into the login form and submit it.

[1675] Input: Credentials

[1676] Output: Login request

[1677] Server: Authenticates the teacher access request and grants access to the Teacher Dashboard.

[1678] Specific behavior:

[1679] Generate an authentication token to grant access to the dashboard.

[1680] Input: Login request

[1681] Output: Authentication token

[1682] Server: Retrieves student learning data, progress, and emotional data from the database and displays them on a dashboard.

[1683] Specific behavior:

[1684] Execute SQL queries to retrieve data and display it in a dashboard in real time.

[1685] Input: Authenticated teacher request

[1686] Output: Training data, progress, and sentiment data on a dashboard

[1687] Users (Teachers): Check the dashboard to understand individual students' understanding, progress, and emotional changes.

[1688] Specific behavior:

[1689] Check out the graphs and statistics on the dashboard.

[1690] Input: Data on the dashboard

[1691] Output: Understanding level of understanding, progress, and emotional changes

[1692] User (Teacher): If you notice that a student is falling behind, plan a make-up lesson or additional assignment.

[1693] Specific behavior:

[1694] Enter and set up supplementary lesson plans and additional assignments into the system.

[1695] Input: Supplementary lesson plan details, additional assignment details

[1696] Output: Set supplementary lessons and additional assignments

[1697] User (Teacher): Prepares individual supplementary teaching materials as needed and provides them to students via the server.

[1698] Specific behavior:

[1699] Upload supplementary materials and send them to students' devices.

[1700] Input: Supplementary materials

[1701] Output: Supplementary materials sent to the device

[1702] This enables the system of the present invention to provide optimal learning content according to each student's level of understanding, learning progress, and emotional pattern, and also to support teachers.

[1703] (Application example 2)

[1704] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1705] It is difficult for stores to provide services and product suggestions that meet the needs of each individual customer in real time. Using conventional methods, it is not realistic for store staff to provide optimal service to every customer, and there is a lack of means to quickly and accurately analyze large amounts of customer data. Furthermore, there is a growing need for systems that can grasp customer emotions and behavior in real time and provide services based on that information.

[1706] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for collecting behavioral data and emotional data of customers, means for analyzing the behavioral data and emotional data and providing optimal product proposals and services to individual customers, and means for generating optimal product information and campaign information for each customer using a generative AI model. This makes it possible to propose optimal services and products to each customer individually.

[1707] "Learning data" refers to various data related to student learning, such as student test results, class output, and emotional data.

[1708] "Test results" refers to data such as the scores and answers students receive on tests and exams.

[1709] "Class output" refers to data such as reports, assignments, and presentations submitted by students during class.

[1710] "Emotional data" refers to data on emotions such as interest, understanding, and confusion that are analyzed from students' facial expressions and tone of voice.

[1711] "Level of understanding" is an indicator of how well a student understands the learning content.

[1712] "Learning progress" is an indicator of how far a student has progressed with their learning plan and curriculum.

[1713] "Learning content" refers to content such as teaching materials, assignments, and tests provided to support students' learning.

[1714] "Real time" refers to the time characteristic in which processing is carried out at the exact moment an event occurs.

[1715] "Questions and requests" are actions or requests that indicate a student's doubts or requests for assistance during their studies.

[1716] "Progress" is the progress that indicates how far a student is progressing with a learning activity or assignment.

[1717] "Behavioral data" refers to data on the actions customers take in the store, including their location, route of movement, and direction of gaze.

[1718] "Product proposal" is the act of selecting and recommending an appropriate product to a specific customer.

[1719] "Service" means any support or assistance provided to enhance Customer satisfaction.

[1720] A "generative AI model" is an artificial intelligence model that generates unique information or answers based on input data.

[1721] A "prompt" is an instruction or introductory sentence entered into a generative AI model to prompt it to generate specific information or an answer.

[1722] "Campaign Information" refers to marketing information such as discounts, special offers, and events offered for a specific period or under specific conditions.

[1723] This invention implements a "customer experience improvement system" that responds to the needs of each individual customer in a store.

[1724] 1. Data Collection and Storage

[1725] Devices: Video and audio are collected using smartphones or smart glasses used by customers in the store. Smart glasses have built-in cameras and microphones and collect data in real time. For example, video and audio data is collected when a customer uses smart glasses to browse the shelves.

[1726] Server: The collected data is sent to a cloud server and stored in a database. The server can use AWS cloud services or Google Cloud.

[1727] 2. Collecting and analyzing emotion data

[1728] Device: Using an emotion engine (such as Amazon Rekognition or Google Cloud Vision), the device analyzes the customer's facial expressions and tone of voice to generate emotion data. For example, if a customer is interested in a particular product but confused, emotion data representing interest and confusion will be generated from their facial expressions.

[1729] Server: Emotion data is sent to the cloud server in real time and stored in a database.

[1730] 3. Data Analysis

[1731] Server: The cloud-based analysis module periodically retrieves necessary data from the database and analyzes customer behavior patterns and sentiment data. This analysis uses machine learning algorithms using programming languages ​​such as Python and R.

[1732] Server: Based on the analysis results, the customer's purchasing profile is updated. This profile includes the customer's past purchasing history, behavioral data, and emotional data.

[1733] 4. Creation of Learning Content Services

[1734] Server: A generative AI model (such as GPT-4) generates optimal product information and campaign information based on the prompt text.

[1735] Server: The generated information is sent to your device in real time.

[1736] Terminal: Displays the received information and provides it in a form that can be viewed and used by the customer.

[1737] 5. Real-time delivery and feedback

[1738] Customers: When customers select a product or have questions, they send their feedback or questions to the system via their smartphone or smart glasses.

[1739] Device and Server: Feedback and questions are sent in real time to the cloud server for analysis. Based on the analysis results, the generative AI model generates the optimal answer and delivers it to the customer's device.

[1740] Specific examples

[1741] Example 1: Customer browsing the store on their smartphone

[1742] 1. A customer takes a photo of a product shelf using their smartphone camera.

[1743] 2. The device transmits video and audio in real time to the cloud server.

[1744] 3. The server analyzes the data using an emotion engine and generates emotion data such as interest or confusion.

[1745] 4. The server launches a generative AI model based on the analysis results to create optimal product recommendations.

[1746] 5. The device will display suggested product information and campaigns.

[1747] Example 2: Using smart glasses

[1748] 1. A customer puts on smart glasses and walks through the store.

[1749] 2. The device collects video and audio using the camera and microphone and sends them to a cloud server.

[1750] 3. The server uses an emotion engine to analyze the customer's facial expressions and tone of voice, generating emotion data in real time.

[1751] 4. The server uses the generative AI model to generate optimal product suggestions and campaign information.

[1752] 5. The device displays the suggested information on the smart glasses display.

[1753] Examples of prompt statements

[1754] "When a customer is looking at the shelf facing the camera, the prompt reads:

[1755] "This customer seems interested in this product, but also a little confused. Please provide the following information: product features, customer testimonials, other customer reviews, and recommended outfits."

[1756] "When a customer asks a question, the prompt is:

[1757] "Customers have asked the following question: 'What other colors does this product come in and what are its benefits?'"

[1758] This enables the system to provide individualized information and services to each customer in real time.

[1759] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1760] Step 1:

[1761] User: A customer uses a smartphone or smart glasses in a store to browse the shelves.

[1762] Input: Video and audio data collected by smartphones and smart glasses.

[1763] How it works: The camera captures images of the shelves in the store, and the microphone collects audio. The collected data is then saved in the device in real time.

[1764] Step 2:

[1765] Terminal: Sends collected video and audio data to the cloud server.

[1766] Input: Video and audio data stored on a smartphone or smart glasses.

[1767] Output: Video and audio data sent to the cloud server.

[1768] Specific operation: The terminal uses the HTTPS protocol to send data to the cloud server over the network.

[1769] Step 3:

[1770] Server: The cloud server generates customer emotion data using the emotion engine.

[1771] Input: Video and audio data stored on a cloud server.

[1772] Output: Analyzed emotion data (emotional information such as interest, confusion, etc.).

[1773] What it does: It uses emotion analysis services such as Amazon Rekognition and Google Cloud Vision to analyze the customer's facial expressions and tone of voice from video and audio to generate emotion data.

[1774] Step 4:

[1775] Server: The cloud server integrates and analyzes the behavioral and emotional data to update the customer's purchasing profile.

[1776] Input: Previously generated emotion data and previously stored behavioral data.

[1777] Output: Updated customer purchasing profile.

[1778] What it does: Collates databases, integrates and analyzes historical behavioral and sentiment data, and uses machine learning algorithms in Python and R to analyze customer buying patterns and interests.

[1779] Step 5:

[1780] Server: Generates optimal product suggestions and campaign information using generative AI models.

[1781] Input: Updated customer purchasing profile and prompt text.

[1782] Output: Generated product suggestions and campaign information.

[1783] How it works: A generative AI model (such as GPT-4) generates optimal product suggestions and campaign information based on a prompt such as, "This customer seems interested in this product, but also seems a little confused. Please provide the following information: this product's features, customer reviews, other customer reviews, and recommended outfits."

[1784] Step 6:

[1785] Server: Generated product proposals and campaign information are sent to the customer's device in real time.

[1786] Input: Product suggestions and campaign information generated by the AI ​​model.

[1787] Output: Product offers and campaign information sent to your device.

[1788] What it does: The generated information is sent in real time to your smartphone or smart glasses using the HTTPS protocol.

[1789] Step 7:

[1790] Device: Check and use product information and campaign information suggested by customers.

[1791] Input: Product offers and campaign information sent to your device.

[1792] Output: Display of product information and campaign information that customers can check.

[1793] Specific operation: Product information and campaign information generated by the generative AI model is displayed on the device display, and provided in a form that can be viewed and used by the customer.

[1794] Step 8:

[1795] Users and devices: To help you select products and provide feedback if you have questions.

[1796] Input: Feedback of your questions and suggestions.

[1797] Specific operation: Using a smartphone or smart glasses, customers enter feedback or questions and send them to a cloud server.

[1798] Step 9:

[1799] Server: Generates optimal answers based on feedback and questions, and delivers them to your device in real time.

[1800] Input: Feedback and questions submitted by customers.

[1801] Output: Optimized answer and / or additional information.

[1802] What it does: Uses a generative AI model to generate optimal answers based on customer questions and feedback, and sends them to the device in real time. For example, it uses a prompt like, "A customer asked the following question: 'What other colors does this product come in and what are its benefits?'"

[1803] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1804] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1805] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1806] [Fourth embodiment]

[1807] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1808] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1809] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1810] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1811] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1812] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1813] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1814] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1815] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1816] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1817] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1818] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1819] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1820] This invention relates to a learning system that uses generative AI to provide optimal learning content for each student. Specifically, it describes a method that incorporates students' test results and class output, generates and analyzes personal learning data, and provides learning content in real time based on that data.

[1821] Data collection and storage

[1822] Devices: When students take tests or submit assignments during class, their devices (PCs, tablets, etc.) collect this data and send it to the server.

[1823] Server: Stores received test results and lesson output in a database. For example, it stores students' test scores and answers, as well as images and text data of submitted reports and assignments.

[1824] Data analysis

[1825] Server: The analysis module is launched periodically or when new data is added. The server retrieves test results and lesson outputs for each student in the database and analyzes them.

[1826] Example: Analyze the accuracy rate, answer time, and patterns of incorrect answers of math problems solved by Student A to generate a learning profile. For example, if it turns out that the accuracy rate for a specific type of problem is low, provide learning content that reinforces that type of problem.

[1827] Generating learning content

[1828] Server: Based on the generated learning profile, the server uses generative AI to generate optimal learning content for each student. The content can come in a variety of formats, including text, images, videos, and interactive questions.

[1829] Example: For a type of math problem that Student A struggles with, we generate related explanatory videos and practice problems and provide them as a single learning package.

[1830] Real-time delivery and feedback

[1831] Server: The generated content is sent to students' devices in real time, allowing them to view the content on their own devices and progress with their studies.

[1832] Device: If a student has a question while studying, they can immediately input it into the AI. For example, they can ask, "Why is the answer to this problem like this?"

[1833] Server: Receives questions, analyzes them in real time, generates appropriate answers and additional questions, and sends them to students' devices. Students receive these and continue their learning.

[1834] Teacher Support and Monitoring

[1835] User (Teacher): Teachers access the server using a dedicated device or management app.

[1836] Server: Provides a dashboard for teachers, displaying each student's learning progress and level of understanding, allowing teachers to see at a glance how each student is learning.

[1837] Example: A teacher determines that Student A's learning progress is falling behind and plans additional follow-up lessons. If necessary, individual supplementary learning materials can be prepared and provided to the student via the server.

[1838] This will provide each student with the optimal learning environment, enabling high-quality education while reducing the workload of teachers.

[1839] The processing flow will be explained below.

[1840] Step 1:

[1841] User (Teacher): Uploads tests and supplementary materials to be used in class to the server.

[1842] Server: Organizes the uploaded data and links it to the corresponding lesson plans.

[1843] Device: Prepares students to receive tests and supplementary materials.

[1844] Step 2:

[1845] User (student): Takes a test or submits an assignment during class.

[1846] Device: Collects student answers and submissions and sends them to the server in real time. Records student answers digitally.

[1847] Server: Receives the transmitted data and immediately stores it in the database. Backs up the stored data.

[1848] Step 3:

[1849] Server: Launches the data analysis module periodically or when new student data is added.

[1850] Server: Retrieves student test results and lesson output from the database.

[1851] Server: Analyzes the acquired data and calculates each student's level of understanding, learning progress, and patterns of incorrect answers.

[1852] Server: Based on the analysis results, personal learning data is generated and the learning profile for each student is updated.

[1853] Step 4:

[1854] Server: Based on the updated personal learning data, the generative AI module is launched.

[1855] Server: Selects and combines resources (text, images, videos, etc.) containing learning content appropriate for each student.

[1856] Server: Sends the generated learning content to students' devices in real time.

[1857] Device: Display received learning content and make it accessible to students.

[1858] Step 5:

[1859] User (student): If a question arises during learning, the user sends the question to the AI ​​from their device.

[1860] Terminal: Sends questions from students to the server in real time.

[1861] Server: Receives the question and starts the process of analyzing it and generating the best answer.

[1862] Server: Sends generated answers and additional assignments to student devices.

[1863] Device: Show students answers and assignments so they can continue learning.

[1864] Step 6:

[1865] User (Teacher): Logs in to a dedicated device or management app and accesses the server.

[1866] Server: Authenticates the teacher access request and grants access to the Teacher Dashboard.

[1867] Server: Retrieves student learning data, progress, and understanding from the database and displays them on a dashboard.

[1868] User (Teacher): Check the dashboard to understand each student's understanding and progress.

[1869] User (Teacher): Sees that Student A is falling behind and acts on the data to plan make-up lessons and additional assignments.

[1870] Example 1

[1871] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1872] With conventional learning systems, it was difficult to provide individualized learning content that corresponded to each student's level of understanding and learning progress. It also required a great deal of effort for teachers to grasp each student's progress and level of understanding, making it difficult to provide appropriate guidance and feedback to each student in real time. Furthermore, there was a lack of a mechanism for immediate response when students had questions during their studies.

[1873] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1874] In this invention, the server includes means for collecting assessment results and lesson outcomes to generate individual student learning data, means for analyzing the learning data and grasping each student's level of understanding and learning progress, means for generating optimal learning content for each student based on the generated learning data using a generative AI model, means for providing the learning content to students in real time, means for responding to questions and requests from students in real time, means for teachers to check students' progress and level of understanding, and means for teachers to plan additional learning support and provide supplementary materials to students. This makes it possible to provide learning content appropriate for each student, resolve questions in real time, and provide effective educational support while reducing the burden on teachers.

[1875] "Individual student learning data" refers to digital data generated by each student through lessons and assessment activities, and is information that indicates the student's level of understanding and progress.

[1876] "Evaluation results" refers to the scores and evaluation content obtained through learning assessment methods such as tests and assignments.

[1877] "Class outcomes" refers to the results of learning activities that students undertake during class or extracurricular activities, and include a variety of formats such as text, images, and videos.

[1878] "Analysis" refers to the process of evaluating students' learning patterns and comprehension based on collected data and creating individual profiles.

[1879] "Understanding" refers to an indicator of how well a student understands a particular learning content.

[1880] "Learning progress" refers to an indicator that shows how far a student is progressing along the learning curriculum.

[1881] A "generative AI model" refers to an algorithm or model that uses AI technology to automatically generate specific learning content based on input data.

[1882] "Learning content" refers to educational content provided for students to learn, and includes various formats such as text, images, videos, and interactive questions.

[1883] "Real-time" refers to a state in which data is exchanged and processed immediately and is available without waiting.

[1884] "Teacher burden" refers to the time and effort required for teachers to check students' progress, provide feedback, and provide individual instruction.

[1885] "Instructional aids" refer to activities and tools that support the learning process and enable students to learn effectively.

[1886] "Supplemental materials" refers to additional learning materials or assignments provided to deepen students' understanding.

[1887] This invention relates to a learning system that uses a generative AI model to provide optimal learning content for each student. Specifically, it shows a method for incorporating student assessment results and class performance, generating and analyzing personalized learning data, and providing learning content in real time based on that data.

[1888] Data collection and storage

[1889] Device:

[1890] When students take tests or submit assignments during class, devices such as computers and tablets collect this data and send it to a server. For example, students answer test questions and save the results on their devices. The collected data is then sent to a server.

[1891] server:

[1892] The received evaluation results and lesson outcomes are stored in a database. Specifically, data such as student IDs, test scores, answers, and submitted reports are stored. After the data is stored in the database, the conditions for launching the analysis module are checked.

[1893] Data analysis

[1894] server:

[1895] When new data is added, the analysis module is triggered. The server collects data for each student in the database and passes it to the analysis engine. For example, analyzing math test results, it finds out that Student A has a low success rate in certain question types and generates a learning profile.

[1896] Generating learning content

[1897] server:

[1898] Based on the generated learning profile, specific learning content is input as a prompt to the generative AI model. For example, a prompt such as "Please generate explanatory videos and practice problems for math problem type X for Student A" is provided to the generative AI model. The text, images, videos, problem sets, etc. obtained from the generative AI model are compiled into a single learning package, including related explanatory videos and explanations of the answers to the problems.

[1899] As a specific example, if Student A has difficulty with a particular type of math problem, the generative AI model will generate explanatory videos and practice problems based on the prompt text and provide them as a learning package.

[1900] Example prompt sentence:

[1901] Please generate explanatory videos and practice questions for math problem type X based on the test results of Student A. I would like the format to be a combination of text, images, and videos.

[1902] Real-time delivery and feedback

[1903] server:

[1904] The generated learning content is sent to students' devices in real time. Data communication protocols are used to ensure reliable data delivery. Students can then check the content on their own devices and continue their learning.

[1905] Device:

[1906] If a student has a question while studying, they can immediately input it into the AI, for example, "Why is the answer to this problem like this?" The question is then sent to the server.

[1907] server:

[1908] Analyze the question and generate appropriate answers or provide additional practice questions, possibly using a generative AI model, and send the answers or additional questions to the student's device.

[1909] Teacher Support and Monitoring

[1910] User (Teacher):

[1911] Teachers access the server using dedicated devices or a management app. They can use a dashboard to visually monitor each student's progress and understanding. Teachers can use this information to plan additional lessons or provide supplementary materials for specific students.

[1912] server:

[1913] The teacher dashboard displays learning data for each student and makes it easily accessible to teachers. For example, areas of low understanding are visually highlighted. Based on this information, teachers can input follow-up instructions into the server and send them to students' devices.

[1914] This system provides each student with an optimal learning environment, enables real-time question resolution, and provides effective educational support while reducing the burden on teachers.

[1915] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1916] Step 1: Collect data

[1917] Input: Students use their devices to take tests or submit assignments.

[1918] Specific operation: Students answer test questions and save their answers and assignment data on their devices. This includes text data, image data, video data, etc.

[1919] Output: The device sends the collected data to the server.

[1920] Step 2: Receiving and storing data

[1921] Input: Data such as test results and class results sent from the device.

[1922] Specific operation: The server receives the student's evaluation results and lesson results and stores them in a database, including student IDs, test scores, answers, reports, etc.

[1923] Output: Data per student stored in a database.

[1924] Step 3: Analyze the data

[1925] Input: New student data stored in the database.

[1926] How it works: When the server detects new data, it launches the analysis module. The analysis engine collects data for each student in the database and evaluates the student's understanding and learning progress.

[1927] Output: The resulting learning profile.

[1928] Step 4: Generate a learning profile

[1929] Input: Analysis results for each student from the analysis module.

[1930] Specific operation: The server generates a learning profile and creates detailed learning data for each student, including their individual weaknesses and strengths, such as the percentage of correct answers to math problems, the time it takes to answer them, and error patterns.

[1931] Output: The generated learning profile.

[1932] Step 5: Generate learning content

[1933] Input: Learning profile.

[1934] Specific operation: The server inputs a prompt based on the learning profile into the generative AI model. For example, "Please generate an explanatory video and practice questions for math problem type X for Student A." The generative AI model analyzes the prompt and automatically generates appropriate learning content (text, images, videos, interactive questions, etc.).

[1935] Output: The generated learning content.

[1936] Step 6: Package your learning content

[1937] Input: Learning content from a generative AI model.

[1938] What it does: The server compiles the generated content into a learning package, including related instructional videos and explanations of the answers to the questions.

[1939] Output: Packaged learning content.

[1940] Step 7: Real-time serving

[1941] Input: Packaged learning content.

[1942] Specific operation: The server transmits the learning package to the student's device in real time, using a data communication protocol to ensure the data is delivered.

[1943] Output: The learning content displayed on the student's device.

[1944] Step 8: Answering student questions

[1945] Input: Questions from students during learning.

[1946] Specific operation: When a student has a question while studying, they type the question on their device and send it to the server. For example, "Why is the answer to this problem like this?"

[1947] Output: The question sent to the server.

[1948] Step 9: Parsing the question and generating an answer

[1949] Input: The student question sent to the server.

[1950] What it does: The server analyzes the question and generates appropriate answers or provides additional practice questions, possibly using a generative AI model.

[1951] Output: Generated answers and additional practice questions.

[1952] Step 10: Provide your answers

[1953] Input: Generated answers and additional practice questions.

[1954] Specific operation: The server sends answers and additional practice questions to the student's device, where the student receives them and uses them to help with their studies.

[1955] Output: Answers and exercises displayed on student devices.

[1956] Step 11: Teacher Support and Monitoring

[1957] Input: Access from a dedicated device or management app.

[1958] How it works: Teachers access the server and use a dedicated dashboard to check students' progress and understanding, which displays each student's learning profile and progress.

[1959] Output: Teacher progress review and lesson plan.

[1960] Step 12: Plan additional follow-up lessons

[1961] Input: Student progress data viewed through the dashboard.

[1962] What it does: Teachers plan additional lessons and supplementary materials for specific students through the dashboard, and follow-up instructions are entered into the server and sent to student devices.

[1963] Output: Follow-up instructions are sent to the student.

[1964] Through this series of steps, learning content tailored to each student is provided, questions are answered in real time, and teachers can provide effective educational support.

[1965] (Application example 1)

[1966] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1967] Conventional learning systems have difficulty providing optimal learning content in real time based on each student's level of understanding and learning progress. They also lack a system that can provide immediate answers when students have questions. Furthermore, there are insufficient means for teachers to grasp students' learning progress in detail and provide appropriate follow-up. As a result, an environment that maximizes the learning effect of each student in personalized education has not been established.

[1968] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1969] In this invention, the server includes: means for collecting students' test results and lesson output; means for analyzing the learning data and grasping each student's level of understanding and learning progress; means for generating learning content optimized for each student based on the generated learning data; means for providing the learning content to students in real time; means for responding to students' questions and requests in real time; means for teachers to check students' progress and understanding; means for connecting to smart devices used in brick-and-mortar educational facilities and providing learning content and feedback in real time; means for generating learning content using a generative AI model based on a specific learning profile; and means for generating prompt sentences to generate appropriate feedback on topics that the student struggles with. This enables the provision of optimal learning content for each student. Furthermore, real-time responses can immediately resolve students' questions and improve learning efficiency. Furthermore, providing teachers with a means for accurately grasping students' learning progress and providing effective follow-up can improve the quality of personalized education.

[1970] "Individual student learning data" refers to data on individual students' test results and class output.

[1971] "Learning data analysis" refers to the process of processing and analyzing collected data to understand each student's level of understanding and learning progress.

[1972] "Learning Content" refers to optimal learning materials provided to students using generative AI models, and may include formats such as text, images, and videos.

[1973] "Real-time delivery" refers to the instant delivery of generated learning content to students.

[1974] "Student Q&A" refers to the process of providing immediate and appropriate answers or additional learning content in response to student questions or requests.

[1975] "Teacher Monitoring Tools" refers to tools specifically designed for teachers to monitor and manage student progress and understanding.

[1976] "Smart device connectivity" refers to linking the system with devices such as smartphones, tablets, and smart glasses used in brick-and-mortar educational facilities.

[1977] A "generative AI model" refers to an artificial intelligence model that automatically generates optimal learning content and feedback based on input data.

[1978] A "prompt" is a sentence that constructs instructions or questions for a generative AI model based on a specific learning profile.

[1979]

[1980] This invention is a smart education support system that utilizes generative AI models to provide optimal learning content for individual students in brick-and-mortar educational facilities. A specific embodiment of this system is described below.

[1981] Hardware Configuration

[1982] It uses a server and smart devices such as smartphones, tablets, and smart glasses. The server has the central function of collecting, analyzing, generating, and distributing data. The smart devices are used by students and teachers as an interface to receive learning content and feedback in real time.

[1983] Software Configuration

[1984] It uses the Python programming language, the generative AI model API provided by OpenAI, and a database management system.

[1985] System action

[1986] The server collects test results and class output from students' devices and stores them in a database. The collected data is periodically analyzed on the server. As a result of the analysis, each student's level of understanding and learning progress is identified and an individual learning profile is generated.

[1987] Based on this learning profile, the server uses a generative AI model to generate optimal learning content. At this time, a prompt sentence is input to the AI ​​model. For example, the following prompt sentence is generated:

[1988] Example prompt sentence:

[1989] Based on the math test results, Student A's most common mistakes are solving equations and basic use of trigonometric functions. Create learning content to deepen their understanding of these topics.

[1990] The generated learning content is delivered to students' devices in real time. Students can view this content and continue their studies via smartphones, tablets, or smart glasses. If students have questions while studying, they can input them in real time, and the generative AI model will instantly generate an answer, which will then be delivered to the student's device.

[1991] Teachers can also use the system to access a dedicated dashboard that allows them to see students' progress and understanding in real time, allowing them to plan individual follow-ups and provide additional materials or assignments if necessary.

[1992] For example, if test analysis reveals that Student A is struggling with fraction calculations, the teacher can provide supplementary materials on fraction calculations. This system provides an optimal learning environment for each student, improving the quality of personalized education.

[1993] As described above, by implementing the present invention, it becomes possible to realize real-time individualized learning and teacher support, which was difficult with conventional systems.

[1994] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1995]

[1996] Step 1:

[1997] The server collects test results and class output (text data, image data, video data) from students' devices. This allows the collection of student learning data. The test results and class output data taken by students are used as input. As output, the data is stored in a database on the server.

[1998] Step 2:

[1999] The server periodically launches the data analysis module to retrieve the students' learning data stored in the database. The data analysis module analyzes this learning data to understand each student's level of understanding and learning progress. The collected learning data is used as input for this step. The output is a learning profile for each student.

[2000] Step 3:

[2001] The server creates a prompt to generate specific learning content based on the generated learning profile. For example, it generates a prompt such as, "Based on the results of the math test, the questions that Student A often gets wrong are: solving equations, basic use of trigonometric functions. Please generate learning content to deepen his understanding of these topics." The student's learning profile is used as input. The generated prompt is used as output.

[2002] Step 4:

[2003] The server sends the generated prompt sentences to a generative AI model, which then generates optimal learning content based on the prompt sentences. For example, videos and supplementary learning materials are generated to deepen understanding. The prompt sentences are used as input, and the learning content is generated as output.

[2004] Step 5:

[2005] The generated learning content is sent from the server to the student's device in real time. The student checks this content on their device and proceeds with their learning. The generated learning content is used as input for this step. The output is the content displayed on the student's device.

[2006] Step 6:

[2007] When a device receives a question or request from a student, the question or request is sent to the server. The server uses a generative AI model to generate an appropriate answer for the received question and sends it back to the student's device. The question or request from the student is used as input. The answer is displayed on the student's device as output.

[2008] Step 7:

[2009] Teachers access a dedicated dashboard to check students' learning progress and comprehension data provided by the server. If necessary, teachers create additional learning materials and assignments and provide them to students' devices via the server. Student learning profiles and progress data are used as input. Additional learning materials and assignments are provided to students as output.

[2010] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[2011] This invention relates to a learning system that uses generative AI and an emotion engine to provide optimal learning content for each student. Specifically, it describes a method for incorporating students' test results, class output, and emotional data, generating and analyzing personalized learning data, and providing learning content in real time based on that data.

[2012] Data collection and storage

[2013] Devices: Students take tests and submit assignments during class. The devices (computers, tablets, etc.) collect this data and send it to the server.

[2014] Server: Stores received test results and lesson output in a database. For example, it stores students' test scores and answers, as well as images and text data of submitted reports and assignments.

[2015] Collecting Emotional Data

[2016] Device: The emotion engine analyzes the student's facial expression, tone of voice, and input content to generate emotion data. For example, it senses changes in facial expression and voice when a student is facing a problem.

[2017] Server: Receives emotion data and stores it in a database. Emotion data includes information such as "interested," "understanding," and "confused."

[2018] Data analysis

[2019] Server: Launches modules that periodically analyze test results, lesson output, and sentiment data.

[2020] Server: Retrieves necessary learning data from the database and analyzes each student's level of understanding, learning progress, and emotional patterns.

[2021] Server: Based on the analysis results, generate personal learning data and update each student's learning profile.

[2022] Generating learning content

[2023] Server: Based on the generated personal learning data, the generative AI module is launched.

[2024] Server: Selects appropriate learning content for each student and customizes it taking into account emotional data.

[2025] Server: Sends the generated learning content to students' devices in real time.

[2026] Device: Display received learning content and make it accessible to students.

[2027] Real-time delivery and feedback

[2028] User (student): If a question arises during learning, the user sends the question to the AI ​​from their device.

[2029] Terminal: Sends questions from students to the server in real time.

[2030] Server: Receives the question and starts the process of analyzing it and generating the best answer.

[2031] Server: Sends generated answers and additional assignments to student devices.

[2032] Device: Show students answers and assignments so they can continue learning.

[2033] Teacher Support and Monitoring

[2034] User (Teacher): Accesses the server using a dedicated device or management app.

[2035] Server: Authenticates the teacher access request and grants access to the Teacher Dashboard.

[2036] Server: Retrieves student learning data, progress, comprehension, and emotional data from the database and displays them on a dashboard.

[2037] Users (Teachers): Check the dashboard to understand individual students' understanding, progress, and emotional changes.

[2038] User (Teacher): If the teacher determines that a student's progress is falling behind, he / she plans supplementary lessons and additional assignments. If necessary, he / she prepares individual supplementary materials and provides them to the student via the server.

[2039] Specific examples

[2040] Example 1: A student takes a test

[2041] 1. User (student): Take a test during class.

[2042] 2. Terminal: Records students' answers and transmits them to the server in real time.

[2043] 3. Server: Receives and stores data.

[2044] 4. Server: The emotion engine analyzes the student's facial expressions and tone of voice to generate emotion data during the test.

[2045] 5. Server: Analyzes the test results and emotion data and updates the personal training data.

[2046] Example 2: When a student asks an AI a question

[2047] 1. User (student): If a question arises during learning, the user sends the question to the AI ​​from their device.

[2048] 2. Terminal: Sends questions from students to the server in real time.

[2049] 3. Server: Receives the question and starts the process of analyzing it and generating the best answer.

[2050] 4. Server: Sends the generated answers to the students' devices.

[2051] 5. Device: Show students their answers and continue learning.

[2052] Teachers checking student progress

[2053] 1. User (teacher): Log in to a dedicated device or the management app and access the server.

[2054] 2. Server: Authenticates the teacher access request and grants access to the Teacher Dashboard.

[2055] 3. Server: Retrieves students' learning data, progress, and emotional data from the database and displays them on the dashboard.

[2056] 4. Users (Teachers): Check the dashboard to understand individual students' understanding, progress, and emotional changes.

[2057] 5. User (Teacher): If you notice that a student is falling behind, plan a make-up lesson or additional assignment.

[2058] The processing flow will be explained below.

[2059] Step 1:

[2060] User (Teacher): Uploads tests and supplementary materials to be used in class to the server. Materials include workbooks, videos, slides, etc.

[2061] Server: Organizes uploaded data by format and links it to the corresponding lesson plans.

[2062] Devices: Student devices are prepared and configured to access materials.

[2063] Step 2:

[2064] User (student): Takes tests or submits assignments during class. Study the designated material on the device.

[2065] Terminal: Collects student answers and submissions (text, images, audio, etc.) and sends them to the server in real time.

[2066] Server: Receives the transmitted data and immediately stores it in a database, along with recording the timestamp information.

[2067] Step 3:

[2068] Devices: Data is collected using sensors that detect students' facial expressions and tone of voice, for example, cameras and microphones that analyze real-time video and audio.

[2069] Server: Analyzes facial expressions, tone of voice, and input content collected by the emotion engine to generate emotional data.

[2070] Server: Receives emotional data and stores it in a database. The emotional data includes students' reactions (happiness, confusion, etc.).

[2071] Step 4:

[2072] Server: Launches the data analysis module periodically or when new data is added.

[2073] Server: Retrieves students' test results, lesson output, and emotion data from the database.

[2074] Server: Comprehensively analyzes the acquired data and calculates each student's level of understanding, learning progress, and emotional patterns. For example, it evaluates the student's level of concentration when working on a specific task.

[2075] Server: Based on the analysis results, generate personal learning data and update each student's learning profile.

[2076] Step 5:

[2077] Server: Based on the updated personal learning data, the generative AI module is launched.

[2078] Server: Selects appropriate learning content for each student and customizes it taking into account emotional data, for example, including explanatory videos or additional practice questions for difficult assignments.

[2079] Server: Sends the generated learning content to students' devices in real time.

[2080] Device: Display received learning content and make it accessible to students.

[2081] Step 6:

[2082] User (student): If a question arises during learning, the user sends the question to the AI ​​from their device.

[2083] Terminal: Sends questions from students to the server in real time.

[2084] Server: Receives the question and starts the process of analyzing it and generating the best answer.

[2085] Server: Sends generated answers and additional assignments to student devices.

[2086] Device: Show students their answers and assignments and keep them learning.

[2087] Step 7:

[2088] User (Teacher): Accesses the server using a dedicated device or management app.

[2089] Server: Authenticates the teacher access request and grants access to the Teacher Dashboard.

[2090] Server: Retrieves student learning data, progress, comprehension, and emotion data from the database and displays them on a dashboard.

[2091] Users (Teachers): Check the dashboard to understand individual students' understanding, progress, and emotional changes.

[2092] User (Teacher): If it is determined that Student A's progress is falling behind, the teacher plans supplementary lessons and additional assignments. If necessary, the teacher conducts individual follow-up based on the emotional data.

[2093] This configuration allows a learning system incorporating an emotion engine to comprehensively evaluate not only a student's level of understanding and progress, but also their emotional changes, and provide optimal learning content in real time. This also makes it easier for teachers to grasp the psychological state of their students, enabling more detailed instruction.

[2094] Example 2

[2095] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2096] Conventional learning systems have the problem that it is difficult to accurately grasp each student's level of understanding and learning progress, and they can only provide uniform learning content. Furthermore, they are unable to provide learning content that takes into account students' emotional data, which affects students' motivation and learning efficiency. As a result, they are unable to provide an optimal learning experience for each student, and there are issues with not being able to provide effective learning support.

[2097] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[2098] In this invention, the server includes means for collecting students' test results and lesson output to generate individual student learning data, means for analyzing the learning data and students' emotional data to grasp each student's level of understanding, learning progress, and emotional patterns, and means for generating optimal learning content for each student based on the generated learning data and emotional data. This makes it possible to provide optimal learning content based not only on each student's level of understanding and learning progress, but also on their emotional patterns.

[2099] "Individual student learning data" refers to data that records in detail each student's learning situation, achievements, and level of understanding.

[2100] "Test results" are data that indicate the scores and answers given by students on tests they have taken.

[2101] "Class output" refers to data that includes assignments, reports, and notes that students submit during and after class.

[2102] "Emotional data" is data about emotions obtained by analyzing students' facial expressions, tone of voice, content of speech, etc.

[2103] A "learning profile" is individual data that compiles each student's level of understanding, learning progress, emotional patterns, etc.

[2104] A "generative AI module" is a program or system that uses artificial intelligence technology to generate optimal learning content for students.

[2105] "Learning content" refers to the learning materials and assignments provided to students for learning, and includes various formats such as text, images, and videos.

[2106] The "Teacher Dashboard" is a management screen that allows teachers to check students' learning status, level of understanding, progress, and emotional patterns in real time.

[2107] "Providing in real time" means providing learning content and feedback instantly in line with the current learning environment and situation.

[2108] "Feedback" is information that includes evaluations of students' learning activities, advice, and areas for improvement.

[2109] "Additional teaching materials and assignments" refers to new learning materials and assignments that teachers add to reinforce what students have learned.

[2110] MODE FOR CARRYING OUT THE INVENTION

[2111] This invention is a learning system that provides optimal learning content to students, utilizing a generative AI model and an emotion engine. The system collects students' test results, class output, and emotion data, generates and analyzes individual learning data, and provides learning content in real time based on that data.

[2112] Hardware and software used

[2113] Device:

[2114] Student computers or tablets

[2115] Camera and microphone (for collecting emotional data)

[2116] server:

[2117] Database management systems (e.g., MySQL, PostgreSQL)

[2118] Emotion analysis API (e.g. Microsoft Emotion API, Google Cloud Vision)

[2119] Generative AI modules (e.g., OpenAI GPT-4, Google BERT)

[2120] System Description

[2121] Data collection and storage

[2122] Device: Students take tests and submit assignments during class. The device collects this data (test answers and assignment images / text data) and sends it to the server.

[2123] Server: Stores the received data in a database. Collected data includes students' test scores, answers, submitted papers, and images and text of assignments.

[2124] Specific examples of behavior:

[2125] Students take math tests and upload the results from their devices to the server.

[2126] The server stores the received data in a database for subsequent analysis.

[2127] Collecting Emotional Data

[2128] Device: The emotion engine analyzes the student's facial expressions and tone of voice. For example, emotion data is generated from changes in the student's facial expressions and tone of voice.

[2129] Server: Receives the analyzed emotion data and stores it in a database. The emotion data includes information such as "interested," "understanding," and "confused."

[2130] Specific examples of behavior:

[2131] If a student shows a confused expression during a test, the emotion engine analyzes the facial expression data and generates emotion data for "confusion."

[2132] The generated emotion data is sent to a server and stored in a database.

[2133] Data analysis

[2134] Server: Launches a module that periodically analyzes test results, lesson output, and emotional data. It retrieves the necessary data from the database and analyzes each student's level of understanding, learning progress, and emotional patterns.

[2135] Server: Based on the analysis results, generate personal learning data and update the learning profile.

[2136] Specific examples of behavior:

[2137] Student A's level of understanding is evaluated based on his / her test results, and his / her learning profile is updated by integrating it with lesson output and emotional data.

[2138] Generating learning content

[2139] Server: Based on the generated personal learning data, a generation AI module is used to generate optimal learning content for students.

[2140] Server: Selects appropriate learning content for each student and customizes it taking into account emotional data.

[2141] Server: Sends the generated learning content to students' devices in real time, and the devices display the content.

[2142] Specific examples of behavior:

[2143] Prompt the generative AI model with learning content that includes gentle explanations on topics that Student A has low understanding of.

[2144] The generated content is sent to Student A's device and displayed on the screen.

[2145] Real-time delivery and feedback

[2146] User (student): If a question arises during learning, the user sends the question to the AI ​​from their device.

[2147] Terminal: Sends questions from students to the server in real time.

[2148] Server: Receives the question, analyzes it, and generates the best answer.

[2149] Server: Sends the generated answers and additional assignments to the student's device, which displays them.

[2150] Examples of prompts:

[2151] Question: "Why does water boil at 100 degrees?"

[2152] Answer: "When water reaches 100°C at standard atmospheric pressure, its vapor pressure becomes equal to atmospheric pressure and it begins to boil."

[2153] Teacher Support and Monitoring

[2154] User (Teacher): Accesses the server by logging in to a dedicated device or the management app.

[2155] Server: Authenticates the teacher access request and grants access to the Teacher Dashboard.

[2156] Server: Retrieves students' learning data, progress, and emotional data from the database and displays them on a dashboard.

[2157] Users (Teachers): Check the dashboard to understand each student's level of understanding, progress, and emotional changes, and plan supplementary lessons or additional assignments as needed.

[2158] Example prompt

[2159] Examples of prompt sentences are shown below.

[2160] Sample prompts: Examples of inputs to the AI ​​model that analyzes the content of students' answers

[2161] text

[2162] Analyze the students' test answers below and evaluate their level of understanding and emotional response.

[2163] Test Answers:

[2164] 1. Q1: The answer is "A"

[2165] 2. Q2: The answer is "B"

[2166] ...

[2167] Emotional Data:

[2168] When answering Q1: Facial expression "confused"

[2169] When answering Q2: Facial expression "understood"

[2170] By implementing the system based on this flow of specific data collection, analysis, generation and provision of learning content, and feedback, it is possible to maximize the learning efficiency of each student.

[2171] The flow of the identification process in the second embodiment will be described with reference to FIG.

[2172] Program processing flow

[2173] Step 1: Collect and send data

[2174] Terminal: During class, students take tests and submit assignments. The terminal collects this input data (test answers and assignment images and text data).

[2175] Specific behavior:

[2176] When students answer a test, they enter their answers into the device.

[2177] When submitting an assignment, the text data and images of the assignment are uploaded to the device.

[2178] Input: Test answers, assignment text data, assignment images

[2179] Output: Data transmission request from the device to the server

[2180] Terminal: Sends collected data to the server in real time.

[2181] Specific behavior:

[2182] Generate an API request to the server based on the collected data and POST the data.

[2183] Input: Collected answer data, assignment data

[2184] Output: API request to the server

[2185] Step 2: Receiving and storing data

[2186] Server: Stores the received data in a database, including students' test scores, answers, and images and text of submitted reports and assignments.

[2187] Specific behavior:

[2188] The server executes queries to extract data from incoming API requests and store it in a database.

[2189] Input: Answer data and assignment data sent from the device

[2190] Output: Test results and lesson output stored in the database

[2191] Step 3: Collect and analyze emotion data

[2192] Device: The emotion engine analyzes the student's facial expressions and tone of voice. For example, emotion data is generated from changes in the student's facial expressions and tone of voice.

[2193] Specific behavior:

[2194] The device captures students' facial expressions and voices using the camera and microphone, and calls the emotion engine API to perform emotion analysis.

[2195] Input: Student facial and voice data from camera and microphone

[2196] Output: Parsed emotion data

[2197] Terminal: Transmits the analyzed emotion data to the server.

[2198] Specific behavior:

[2199] Emotion data is sent to the server as a POST request.

[2200] Input: Parsed emotion data

[2201] Output: API request to the server

[2202] Step 4: Receiving and storing emotion data

[2203] Server: Receives emotion data and stores it in a database. Emotion data includes information such as "interested," "understanding," and "confused."

[2204] Specific behavior:

[2205] The server executes a query to store the received emotion data in a database.

[2206] Input: Emotion data sent from the device

[2207] Output: Emotion data stored in a database

[2208] Step 5: Data analysis

[2209] Server: Launches a module that periodically analyzes test results, lesson output, and emotional data. It retrieves the necessary data from the database and analyzes each student's level of understanding, learning progress, and emotional patterns.

[2210] Specific behavior:

[2211] Batch processing is performed to retrieve the necessary data from the database and execute the analysis algorithm.

[2212] Input: Test results, lesson output, and emotional data stored in the database

[2213] Output: Analysis results (personal learning data, learning profile)

[2214] Server: Generates personal learning data based on the analysis results and updates the learning profile.

[2215] Specific behavior:

[2216] Based on the analysis results, a query is executed to generate personal learning data and store it in the database.

[2217] Input: Analysis results

[2218] Output: Updated personal learning data, learning profile

[2219] Step 6: Generate learning content

[2220] Server: Based on the generated personal learning data, a generation AI module is used to generate optimal learning content for students.

[2221] Specific behavior:

[2222] The necessary information is extracted from the student's learning profile and used to create prompts, which are then fed into a generative AI model.

[2223] Input: personal training data, prompt sentence

[2224] Output: Learning content output from the generative AI model

[2225] Server: Selects appropriate learning content for each student and customizes it taking into account emotional data.

[2226] Specific behavior:

[2227] Adjust the generated learning content (e.g., change the difficulty level of the content) based on emotional data.

[2228] Input: Generated learning content, emotion data

[2229] Output: Customized learning content

[2230] Server: Sends the generated learning content to students' devices in real time.

[2231] Specific behavior:

[2232] Send an API response containing customized learning content to the device.

[2233] Input: Customized learning content

[2234] Output: Learning content sent to your device

[2235] Device: Display received learning content and make it accessible to students.

[2236] Specific behavior:

[2237] Display the received learning content on the screen.

[2238] Input: Learning content sent to your device

[2239] Output: Learning content displayed on the screen

[2240] Step 7: Real-time delivery and feedback

[2241] User (student): If a question arises during learning, the user sends the question to the AI ​​from their device.

[2242] Specific behavior:

[2243] Enter your question in the form and press the submit button.

[2244] Input: Question

[2245] Output: Sending query data from the terminal to the server

[2246] Terminal: Sends questions from students to the server in real time.

[2247] Specific behavior:

[2248] Sends the question data to the server as a POST request.

[2249] Input: Question

[2250] Output: API request to the server

[2251] Server: Receives the question, analyzes it, and generates the best answer.

[2252] Specific behavior:

[2253] The question content is analyzed, a corresponding prompt sentence is generated, and input into the AI ​​model.

[2254] Receive and format answers from the AI ​​model.

[2255] Input: Question

[2256] Output: The generated answer

[2257] Server: Sends generated answers and additional assignments to student devices.

[2258] Specific behavior:

[2259] Sends an API response containing the answer to the device.

[2260] Input: Generated Answer

[2261] Output: Answer sent to terminal

[2262] Device: Show students their answers and assignments and keep them learning.

[2263] Specific behavior:

[2264] The received response is displayed on the screen.

[2265] Input: Answer sent to the terminal

[2266] Output: Answers displayed on the screen

[2267] Step 8: Teacher support and monitoring

[2268] User (Teacher): Logs in to a dedicated device or management app and accesses the server.

[2269] Specific behavior:

[2270] Enter your authentication information into the login form and submit it.

[2271] Input: Credentials

[2272] Output: Login request

[2273] Server: Authenticates the teacher access request and grants access to the Teacher Dashboard.

[2274] Specific behavior:

[2275] Generate an authentication token to grant access to the dashboard.

[2276] Input: Login request

[2277] Output: Authentication token

[2278] Server: Retrieves student learning data, progress, and emotional data from the database and displays them on a dashboard.

[2279] Specific behavior:

[2280] Execute SQL queries to retrieve data and display it in a dashboard in real time.

[2281] Input: Authenticated teacher request

[2282] Output: Training data, progress, and sentiment data on a dashboard

[2283] Users (Teachers): Check the dashboard to understand individual students' understanding, progress, and emotional changes.

[2284] Specific behavior:

[2285] Check out the graphs and statistics on the dashboard.

[2286] Input: Data on the dashboard

[2287] Output: Understanding level of understanding, progress, and emotional changes

[2288] User (Teacher): If you notice that a student is falling behind, plan a make-up lesson or additional assignment.

[2289] Specific behavior:

[2290] Enter and set up supplementary lesson plans and additional assignments into the system.

[2291] Input: Supplementary lesson plan details, additional assignment details

[2292] Output: Set supplementary lessons and additional assignments

[2293] User (Teacher): Prepares individual supplementary teaching materials as needed and provides them to students via the server.

[2294] Specific behavior:

[2295] Upload supplementary materials and send them to students' devices.

[2296] Input: Supplementary materials

[2297] Output: Supplementary materials sent to the device

[2298] This enables the system of the present invention to provide optimal learning content according to each student's level of understanding, learning progress, and emotional pattern, and also to support teachers.

[2299] (Application example 2)

[2300] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2301] It is difficult for stores to provide services and product suggestions that meet the needs of each individual customer in real time. Using conventional methods, it is not realistic for store staff to provide optimal service to every customer, and there is a lack of means to quickly and accurately analyze large amounts of customer data. Furthermore, there is a growing need for systems that can grasp customer emotions and behavior in real time and provide services based on that information.

[2302] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for collecting behavioral data and emotional data of customers, means for analyzing the behavioral data and emotional data and providing optimal product proposals and services to individual customers, and means for generating optimal product information and campaign information for each customer using a generative AI model. This makes it possible to propose optimal services and products to each customer individually.

[2303] "Learning data" refers to various data related to student learning, such as student test results, class output, and emotional data.

[2304] "Test results" refers to data such as the scores and answers students receive on tests and exams.

[2305] "Class output" refers to data such as reports, assignments, and presentations submitted by students during class.

[2306] "Emotional data" refers to data on emotions such as interest, understanding, and confusion that are analyzed from students' facial expressions and tone of voice.

[2307] "Level of understanding" is an indicator of how well a student understands the learning content.

[2308] "Learning progress" is an indicator of how far a student has progressed with their learning plan and curriculum.

[2309] "Learning content" refers to content such as teaching materials, assignments, and tests provided to support students' learning.

[2310] "Real time" refers to the time characteristic in which processing is carried out at the exact moment an event occurs.

[2311] "Questions and requests" are actions or requests that indicate a student's doubts or requests for assistance during their studies.

[2312] "Progress" is the progress that indicates how far a student is progressing with a learning activity or assignment.

[2313] "Behavioral data" refers to data on the actions customers take in the store, including their location, route of movement, and direction of gaze.

[2314] "Product proposal" is the act of selecting and recommending an appropriate product to a specific customer.

[2315] "Service" means any support or assistance provided to enhance Customer satisfaction.

[2316] A "generative AI model" is an artificial intelligence model that generates unique information or answers based on input data.

[2317] A "prompt" is an instruction or introductory sentence entered into a generative AI model to prompt it to generate specific information or an answer.

[2318] "Campaign Information" refers to marketing information such as discounts, special offers, and events offered for a specific period or under specif...

Claims

1. A means of collecting student test results and lesson outputs to generate individual student learning data; A means for analyzing the learning data and grasping the level of understanding and learning progress of each student; A means for generating optimal learning content for students based on the generated learning data; means for providing said learning content to students in real time; A means to respond in real time to student questions and requests, and and a means for teachers to check student progress and understanding.

2. Provide appropriate feedback to students based on the generated learning data, 10. The system of claim 1, further comprising means for the teacher to provide additional teaching materials and assignments to reinforce what the student has learned.

3. The learning data includes text data, image data, and video data of students; The system according to claim 1, further comprising means for integrating and analyzing these data.

Images