System

Abstract

A system is provided.SOLUTION: A system comprising: means for collecting learning activity information of a student; means for analyzing the collected learning activity information using a generative AI model; means for generating an individualized learning material or learning plan based on the analysis result; means for providing the generated learning material or learning plan to the student; and means for continuously monitoring the learning progress of the student and reevaluating the learning plan based on new information.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In today's educational environment, despite the existence of students with diverse learning styles and comprehension levels, teachers face the challenge of responding to the progress and comprehension levels of each individual student. In particular, it is difficult to provide appropriate instruction to an average of 23 students per class in real time, and there are limitations to providing an optimal learning experience for all students. As a result, there is a risk of reduced learning effectiveness and educational disparities. [Means for solving the problem]

[0005] To solve the above problems, the present invention provides the following means. First, it includes a means for collecting data on students' learning activities and a means for analyzing the collected learning data using a generative AI model. It also includes a means for generating personalized learning materials and learning plans based on the analysis results, and a means for providing the generated learning materials and learning plans to students. Furthermore, the system includes a means for continuously monitoring students' learning progress and reevaluating learning plans based on new data. This provides a learning method suited to each student, reducing the workload of teachers while realizing an environment in which all students can learn effectively.

[0006] "Student learning activity data" refers to information about a student's behavior and progress while studying, including, specifically, subjects taken, study time, answer history, grades, etc.

[0007] A "generative AI model" refers to a computer program or algorithm that uses artificial intelligence to perform a specific task, and in this invention is specifically intended for analyzing training data and generating teaching materials.

[0008] "Individualized learning materials" refers to learning materials and study plans that are customized based on each student's learning progress and level of understanding.

[0009] A "learning plan" is a plan that includes a timeline and a list of activities designed to help students progress through their studies efficiently.

[0010] "Learning progress" is an indicator of where a student is in the learning process, and specifically includes data showing the student's level of completion and understanding.

[0011] "Reevaluation" is the process of reviewing future learning plans based on existing data and providing the most appropriate learning methods and materials.

[0012] "Means" refer to the methods or tools used to achieve a particular goal.

[0013] The term "system" refers to a set of processes or devices that function in conjunction with multiple interrelated elements, and in this invention refers to the entire technological structure for supporting learning.

[0014] "Analysis" is the process of examining the collected data in detail to derive effective learning strategies and teaching materials.

[0015] "Providing" refers to the act of making particular content or functionality available to users.

[0016] "Monitoring" refers to the continuous observation of a specific process or condition and the collection and recording of that information. [Brief explanation of the drawings]

[0017] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10]1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0018] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0019] First, the terms used in the following description will be explained.

[0020] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0021] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0022] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0023] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0024] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0025] [First embodiment]

[0026] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0027] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0028] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0029] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0030] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0031] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0032] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0033] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0034] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0035] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0036] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0037] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0038] The present invention relates to a learning support system, and in particular to a system for personalized learning support using a generative AI model. Specific embodiments for carrying out the present invention will be described below.

[0039] This system consists of three components: students, terminals, and a server. The roles and processing flow of each component are described in detail below.

[0040] server

[0041] The server is the core of the system, responsible for collecting and analyzing learning activity data, operating the generative AI model, and generating and distributing personalized learning materials. The server first sets up an API endpoint and receives learning data sent from the device. The received data is stored in a database and queued for subsequent analysis jobs.

[0042] The server then uses a generative AI model to analyze the collected learning data. The analysis evaluates each student's learning style, level of understanding, and progress, and generates personalized learning materials and study plans based on the results. These generated learning materials are then managed by the server and sent to the corresponding student's device.

[0043] As a specific example, if the server determines, based on data acquired during a mathematics class, that Student A is struggling with a particular concept in a practice problem, it will generate and distribute additional practice problems and explanatory videos to reinforce that concept.

[0044] Terminal

[0045] The device is the interface for students to study and is typically implemented as a tablet or PC. The device communicates data between the student and the server via a learning app. When a student logs in, the device records their activity in real time and periodically sends data to the server.

[0046] The personalized learning materials and study plans sent from the server are displayed on the device's dashboard. Students can select the materials that are most suitable for them and proceed with their studies from this dashboard. The device also instantly records the student's answers and progress and sends them back to the server.

[0047] As a concrete example, Student B works on a specific math problem set provided on his / her device, and progresses through his / her studies while looking at the answers and explanations for each problem. At this time, the device constantly records progress information and sends it to the server.

[0048] Users (students and teachers)

[0049] The user is also an important part of the system. Students use their devices to study materials that are optimized for them. Their learning behavior and progress are automatically recorded by the devices, allowing students to focus on their studies independently.

[0050] Teachers can access the server and monitor students' learning progress and analysis results in real time, and based on this, teachers can provide feedback to students and offer additional teaching materials and advice.

[0051] For example, Teacher C can use the server's dashboard to monitor the progress of the entire class and individual students, adjust the lesson plan as needed, and provide additional supplementary materials if a particular student is found to have a lack of understanding of a particular concept.

[0052] This system reduces the workload of teachers while providing students with learning methods that are suited to them, achieving the goal of creating an environment in which all students have equal access to learning opportunities and improving the quality of education.

[0053] The processing flow will be explained below.

[0054] Step 1:

[0055] The server opens an API endpoint for collecting learning data and prepares to receive the learning data. Specifically, the server builds an API to receive student learning activity data (e.g., subjects taken, study time, progress) sent from the device. This data is stored in a database.

[0056] Step 2:

[0057] When a student logs in to the learning app, the device starts recording session information. Specifically, the device records the user's ID and login time at login, collects data each time a learning activity is started, and periodically sends the data to the server.

[0058] Step 3:

[0059] Users (students) log in to the learning app on their tablet or PC and select the designated subject or unit. When students solve problems or watch videos, their actions are automatically recorded on the device. When they finish learning, they log out.

[0060] Step 4:

[0061] The server stores the received training data in a database, and at the same time inputs the data into the generative AI model and executes the analysis job. Specifically, the data stored in the database is periodically put into the analysis queue, which triggers the generative AI model to perform analysis.

[0062] Step 5:

[0063] The generative AI model evaluates each student's learning style and level of understanding based on collected learning data. The analysis results are used to generate optimal learning plans and personalized learning materials. Specifically, the model analyzes students' past performance and learning patterns to identify areas that need reinforcement.

[0064] Step 6:

[0065] The server sends the personalized learning materials and learning plans generated by the generative AI model to the device. Specifically, the generated learning materials and plans are delivered to the device via API and displayed on the device's user interface.

[0066] Step 7:

[0067] The device then displays the received personalized learning materials and learning plans on a dashboard for students to access. Specifically, the device analyzes the learning material data received from the server and displays it in a user-friendly format.

[0068] Step 8:

[0069] Students can study using personalized learning materials provided on the device's dashboard. Specifically, students work through the provided practice problems and instructional videos, and their progress and results are automatically recorded on the device.

[0070] Step 9:

[0071] The device records learning activity and progress data in real time and periodically sends it to the server. Specifically, at the end of each learning session, the device compiles the session data and sends it to the server as a POST request.

[0072] Step 10:

[0073] The server continuously receives new training data and re-evaluates the training plan based on the analysis results. Specifically, the server incorporates the newly received data into the analysis job and retrains the generative AI model accordingly to optimize the training plan.

[0074] Step 11:

[0075] Teachers can check students' progress and analysis results in real time through the server's dashboard. Specifically, the teacher's management screen monitors each student's learning progress and level of understanding, and provides feedback as needed.

[0076] Step 12:

[0077] The user (teacher) can input supplementary materials and advice into the system according to each student's learning progress and provide them to the student. Specifically, the teacher uses the feedback function on the dashboard to input individual advice and supplementary materials, which are then automatically sent to the student's device.

[0078] Example 1

[0079] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0080] Conventional learning support systems have had difficulty providing personalized learning materials tailored to each student's learning style and level of understanding in a timely manner. They also lacked the means to continuously monitor learning progress and provide appropriate feedback in real time. This resulted in issues such as students not being able to maximize their learning effectiveness and increasing the workload of teachers.

[0081] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0082] In this invention, the server includes means for collecting student learning activity data from the terminal, means for storing the collected learning data in a database and queueing it for analysis jobs, means for analyzing the learning data using a generative AI model and evaluating each student's learning style, level of understanding, and progress, means for generating personalized learning materials and learning plans based on the analysis results, means for sending the generated learning materials and learning plans to the student's terminal, means for recording students' answers and progress and providing feedback to the server, and means for continuously monitoring students' learning progress and reevaluating the learning plans based on new data. This enables personalized learning support optimized for each student, promoting effective learning and reducing the workload of teachers.

[0083] "Devices" refer to electronic devices such as tablets and computers that function as interfaces for students to learn.

[0084] "Learning activity data" refers to all data generated when a student engages in learning activities, and specifically includes answers to questions, answer times, progress, etc.

[0085] "Database" refers to an information management system for organizing and storing collected learning data.

[0086] An "analysis job" refers to a series of processes set up to analyze collected learning data.

[0087] A "generative AI model" refers to a model that uses artificial intelligence technology to analyze learning data and generate personalized teaching materials and learning plans.

[0088] "Teaching materials" refers to materials, workbooks, explanatory videos, etc. that students use for studying.

[0089] A "study plan" refers to a plan that includes learning methods and goals, specific learning content, and schedules that are optimized for each student.

[0090] "Feedback" refers to the server or teacher providing evaluations and advice on students' learning activities.

[0091] "Monitoring" refers to the activity of continuously observing students' learning progress and collecting data as necessary.

[0092] "Reevaluation" refers to reviewing an existing learning plan based on a student's latest learning data and revising or updating it as necessary.

[0093] MODE FOR CARRYING OUT THE INVENTION

[0094] This invention relates to a learning support system, specifically a system for personalized learning support using a generative AI model. This system consists of three entities: a student, a terminal, and a server. The roles of each entity and their specific processing are described in detail below.

[0095] server

[0096] The server is the core of the system, responsible for collecting and analyzing learning activity data, operating the generative AI model, and generating and distributing personalized learning materials. The specific operations of the server are as follows:

[0097] The server sets up an API endpoint to receive learning data sent from the device, for example, the results of a student's math problem answers.

[0098] The received data is stored in a database and queued for subsequent analysis jobs.

[0099] The server analyzes the collected training data using a generative AI model, such as a natural language processing model like GPT-3.

[0100] The system uses analytics to assess each student's learning style, level of understanding, and progress, and then generates personalized learning materials and lesson plans based on the results. For example, if a student is struggling with a math problem, the system generates materials to reinforce that concept.

[0101] The generated teaching materials are again managed by the server and sent to the corresponding student's terminal.

[0102] For example, the server inputs the following prompt sentence into the generative AI model for analysis:

[0103] "Student A is struggling with solving a quadratic equation during a math class and has sent the data to the server. The server should analyze this data and generate additional practice problems and instructional videos for Student A."

[0104] Terminal

[0105] The device is the interface through which students learn, and is typically implemented as a tablet or PC. The specific operations of the device are as follows:

[0106] When a student logs in to a device, the device records their activity in real time and periodically sends data to the server, such as recording the questions the student answered and their results.

[0107] The personalized learning materials and learning plans sent from the server are displayed on a dashboard on the device, which is built using front-end frameworks such as React and Angular.

[0108] Students can use this dashboard to select the learning materials that are most suitable for them and proceed with their studies.

[0109] Students' answers and progress are recorded by the device and fed back to the server.

[0110] For example, Student B works on a specific math problem set provided on his / her device, and progresses through his / her studies while looking at the answers and explanations for each problem. At this time, the device records the answers and sends them to the server.

[0111] Users (students and teachers)

[0112] The user is also an important part of the system. Here are some specific user actions:

[0113] Students study materials optimized for them through their devices, and their learning behavior and progress are automatically recorded by the devices, allowing them to concentrate on their studies.

[0114] Teachers can access the server and monitor students' learning progress and analysis results in real time. For example, teachers can access the dashboard from a browser to check students' grades and progress.

[0115] Based on this, the teacher can provide feedback to the student and provide additional support materials or advice. For example, if a particular student shows a lack of understanding of a particular theorem, the teacher can provide that student with additional learning materials.

[0116] For example, Teacher C can use the server's dashboard to monitor the progress of the entire class and individual students, adjust the lesson plan as needed, and provide additional supplementary materials if a particular student is found to have a lack of understanding of a particular concept.

[0117] As described above, by using a generative AI model, the present invention can provide personalized educational support tailored to each student's learning style and progress, thereby reducing the workload of teachers and providing a more effective learning environment for students.

[0118] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0119] Step 1: Device collects student learning activity data

[0120] How it works: Students log in to their devices and begin working on math problems. The devices then record data in real time, such as the answers and time it took to complete each problem, via a learning app.

[0121] Input: Answer data, answer time, and progress of students logged in to the device

[0122] Output: Learning activity data recorded by the device

[0123] Step 2: The device sends the learning activity data to the server.

[0124] Specific operation: The device sends the student's learning data to the server at regular intervals or when the learning activity is completed, for example, after each question is answered.

[0125] Input: Learning activity data recorded on the device

[0126] Output: Learning activity data sent to the server

[0127] Step 3: The server saves the training data to a database and queues it for analysis jobs.

[0128] Specific operation: After receiving the training data, the server stores it in a database. The stored data is then queued for analysis jobs. This process is automated, enabling efficient data management.

[0129] Input: Learning activity data sent to the server

[0130] Output: Learning activity data stored in the database, data input to the analysis job

[0131] Step 4: The server analyzes the training data using the generated AI model

[0132] How it works: The server takes the data from the queue and inputs it into the generative AI model, which analyzes each student's learning style and level of understanding, and then generates optimal teaching materials and lesson plans based on the results.

[0133] Input: Training data submitted to the analysis job

[0134] Output: Analysis of each student's learning style, level of understanding, and progress

[0135] Step 5: The server generates personalized learning materials and learning plans based on the analysis results.

[0136] What it does: Based on the analysis, it uses a generative AI model to generate personalized learning materials and lesson plans. For example, if a particular student is struggling with a particular math concept, it generates practice exercises and instructional videos that reinforce that concept.

[0137] Input: Analysis of each student's learning style, level of understanding, and progress

[0138] Output: personalized learning materials and learning plans

[0139] Step 6: The server sends the generated learning materials and lesson plans to the students' devices.

[0140] Specific operation: The server sends the generated personalized teaching materials and learning plans to the corresponding student's device.

[0141] Input: Generated personalized learning materials and learning plans

[0142] Output: Materials and lesson plans sent to the corresponding student devices

[0143] Step 7: The device displays the sent learning material and the student continues learning based on it.

[0144] Specific operation: The device displays the learning materials and study plans received from the server on a dashboard. Students can select the appropriate learning materials from this dashboard and proceed with their studies. For example, they can work on the displayed problem sets and progress through the answers.

[0145] Input: Personalized learning materials and study plans sent from the server

[0146] Output: Materials and lesson plans displayed on the dashboard, and student learning activities

[0147] Step 8: The device records the student's answers and progress and sends them back to the server.

[0148] Specific operation: The device again records the student's answers and progress and sends them back to the server. This data is used for analysis in the next cycle.

[0149] Input: Student answers and progress

[0150] Output: Answer results and progress feedback to the server

[0151] Step 9: The server continuously monitors your progress and reevaluates your learning plan based on new data.

[0152] Specific operation: The server continuously monitors learning progress based on the feedback data, analyzes new data, and reevaluates and modifies the learning plan.

[0153] Input: Feedback of answers and progress data

[0154] Output: Updated learning plan

[0155] This series of steps enables effective learning support that is customized for each student.

[0156] (Application example 1)

[0157] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0158] In modern factories, there is a demand for efficient training and improved skills for engineers and workers, but conventional training methods make it difficult to provide appropriate feedback to individual engineers, limiting productivity improvements. Another issue is the lack of technology to efficiently collect and analyze worker activity data and generate training plans tailored to each individual's skill level.

[0159] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0160] In this invention, the server includes means for collecting work activity data of workers, means for analyzing the collected data using a generative AI model, means for generating personalized training plans and learning materials based on the analysis results, means for providing the generated training plans and learning materials to workers, and means for continuously monitoring the progress of workers and reevaluating the training plans based on new data, thereby enabling efficient and personalized training to be provided to engineers and workers.

[0161] "Worker" refers to a person who performs a specific task in a factory or production site.

[0162] "Work activity data" refers to information about the process and results of work performed by workers, and is data collected through devices such as sensors and tablets.

[0163] A "generative AI model" refers to an artificial intelligence model that analyzes collected data and generates personalized training plans, learning materials, etc.

[0164] A "training plan" is a plan aimed at improving a worker's techniques and skills, and includes learning content optimized for each individual worker.

[0165] "Learning materials" refers to the materials and content that workers use to learn according to the training plan, including videos, workbooks, tutorials, etc.

[0166] "Means of provision" refers to the method of providing the generated training plans and learning materials to workers at an appropriate time.

[0167] "Progress monitoring" refers to the continuous observation and recording of workers' progress as they progress through training and their work.

[0168] "Reassessment" refers to reviewing existing training plans and learning methods based on new data collected and making improvements or changes as needed.

[0169] "Means for analyzing data" refers to methods and techniques for processing collected data and assessing worker skills and understanding.

[0170] This invention relates to a system that supports the skill improvement and efficient training of engineers and workers in factories. This system collects data on workers' work activities, analyzes the data using a generative AI model, and generates and provides personalized training plans and learning materials based on the analysis results.

[0171] Hardware and software used

[0172] Hardware

[0173] Training tablet or computer (device)

[0174] Data collection sensors

[0175] Learning information storage server

[0176] GPU server for high-speed calculations (operation of generative AI models)

[0177] software

[0178] Learning Management System (LMS)

[0179] Python-based analysis program

[0180] Generative AI model library (PyTorch or TensorFlow)

[0181] Database management system (PostgreSQL)

[0182] RESTful API (Flask or Django)

[0183] Explanation of the processing flow

[0184] 1. Data Collection

[0185] The server uses sensors and devices to collect work activity data when workers run the training program on their devices. This data includes information about the work process and results. For example, sensors record the time and steps for each process in the assembly of parts.

[0186] 2. Data Analysis

[0187] The server stores the collected work activity data in a database and analyzes it using a generative AI model. The analysis evaluates the worker's skill level and tendency for operational errors. The generative AI model is implemented using a Python-based analysis program and libraries such as PyTorch and TensorFlow.

[0188] 3. Training plan generation

[0189] Based on the analysis results, the server generates personalized training plans and learning materials. These training plans include video tutorials and problem sets that include specific work content and areas for improvement. For example, for the assembly procedure of a new part, the server generates materials that include countermeasures for common mistakes made by workers.

[0190] 4. Submissions and Feedback

[0191] The generated training plan and learning materials are sent to the terminal. As the worker progresses through the training, new data is collected and sent back to the server. This allows the server to reevaluate the training plan based on the latest data and make adjustments as necessary. This feedback loop ensures that the most optimal training is always provided.

[0192] Specific examples

[0193] When training workers on assembly procedures for a new part, they watch a video tutorial on a tablet and then perform a training simulation. Sensors and the tablet collect data during the simulation and send it to a server. A generative AI model analyzes the data and generates additional teaching materials for areas where mistakes are common, which are then sent back to the tablet. This process allows workers to effectively improve their skills.

[0194] Prompt Sentence Examples

[0195] "Please suggest the best training content for the next lesson. Operator A made many mistakes when assembling part B. Please generate teaching materials that include specific problem areas and how to improve them."

[0196] This invention makes it possible to provide efficient and individualized training to engineers and workers, which is expected to improve productivity.

[0197] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0198] Step 1:

[0199] The server collects work activity data from sensors and devices when a worker runs a training program on the device. When the worker presses the start button, the device starts collecting sensor data and sends data about the work process and results to the server. The input is data from the sensors and devices, and the output is raw data stored on the server.

[0200] Step 2:

[0201] The server stores the collected data in a database and performs data cleansing, removing incomplete or noisy data and shaping the data. The input is the raw collected data, and the output is cleaned data that can be analyzed.

[0202] Step 3:

[0203] The server analyzes the cleaned data using a generative AI model. A Python-based analysis program is used to evaluate the worker's skill level and tendency for operational errors. The input is the cleaned data, and the output is a technical evaluation report.

[0204] Step 4:

[0205] The server generates personalized training plans and learning materials based on the analysis results. It uses a generative AI model library (PyTorch or TensorFlow) to generate optimal learning materials for each worker. The input is the technical evaluation report, and the output is personalized training plans and learning materials.

[0206] Step 5:

[0207] The server distributes the generated training plan and learning materials to the terminal. The terminal receives them and provides them to the worker. The worker checks the training plan and learning materials on the terminal and progresses with their learning. The input is the individualized training plan and learning materials, and the output is the worker's learning progress.

[0208] Step 6:

[0209] The terminal collects the worker's training progress data and sends it to the server, including the training completion status and any new mistakes. The input is the worker's progress data, and the output is the data sent to the server.

[0210] Step 7:

[0211] The server re-evaluates the training plan based on the new data received, again using the generative AI model to adjust the training content as needed. The input is the new progress data, and the output is the adjusted training plan.

[0212] Step 8:

[0213] The server then delivers the re-evaluated training plan and learning materials back to the terminal and provides them to the worker. This feedback loop ensures continuous optimal training. The input is the adjusted training plan and learning materials, and the output is the worker's latest learning progress.

[0214] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0215] This invention relates to a learning support system that combines a generative AI model and an emotion engine, and provides means for collecting learning activity data, generating personalized learning materials, managing learning progress, and monitoring the user's emotional state. The system of this invention consists of three main entities: a server, a terminal, and a user. The specific roles and processing flow of each entity are explained below.

[0216] server

[0217] The server is the core of the system, responsible for collecting and analyzing learning activity data and emotion data, operating the generative AI model, and generating and providing personalized learning materials and learning plans. First, the server sets up an API endpoint and prepares to receive data sent from the device. The received data is stored in a database and queued for subsequent analysis jobs.

[0218] The server then combines the generative AI model with an emotion engine to perform analysis. It uses the learning data and emotion data to evaluate the student's learning style, comprehension, and emotional state. Based on the evaluation results, it generates an optimal learning plan and personalized learning materials. The generated learning materials are then sent to the corresponding student's device.

[0219] As a specific example, if the server determines that Student A is feeling anxious about a problem based on data and emotional data acquired during a mathematics class, it will generate and deliver additional practice problems that take into account the student's emotional state and explanatory videos for relaxation.

[0220] Terminal

[0221] The device functions as an interface for users (students) to study and is implemented on a tablet or PC. Data is transmitted between the student and the server via a learning app. When a student logs in, the device records their activity in real time and periodically sends the data to the server. Emotional data is also collected in the same way; for example, changes in the student's facial expression or voice are detected using a camera or microphone, and sent to the emotion engine.

[0222] The personalized learning materials and study plans sent from the server are displayed on a dashboard on the device. Students can select the materials that are most suitable for them and proceed with their studies from this dashboard. The device also instantly records the students' answers, progress, and emotional state and provides feedback to the server.

[0223] For example, if Student B is working on a specific math problem set provided on his / her device and loses concentration while solving it, his / her emotional state will be detected by the device, which will record it and send it to the server.

[0224] Users (students and teachers)

[0225] The user is also an important part of the system. Students use their devices to study materials that are optimized for them. Their behavior, progress, and emotional state are automatically recorded by the devices, allowing them to focus on their studies independently.

[0226] Teachers can access the server to monitor students' learning progress, analysis results, and emotional state in real time, and based on this, teachers can provide feedback to students and provide additional teaching materials and advice.

[0227] For example, Teacher C can use the server's dashboard to monitor the progress and emotional state of the entire class and individual students, adjust the lesson plan as needed, and provide additional supplementary materials or relaxation advice if a particular student is found to be struggling with a particular concept and feeling stressed.

[0228] In this way, learning support systems can reduce the workload of teachers while providing students with learning methods that are suited to them. The goal is to create an environment in which all students can enjoy equal learning opportunities and improve the quality of education.

[0229] The processing flow will be explained below.

[0230] Step 1:

[0231] The server opens an API endpoint to collect learning data and emotional data. First, it prepares to receive learning requests, and then receives student learning activity data (e.g., subjects taken, study time, progress) and emotional data sent from the device. This data is then stored in a database.

[0232] Step 2:

[0233] The device starts recording session information when a student logs in to the learning app. Specifically, it records the user ID and login time when logging in to study, and collects data each time a learning activity begins. It also uses the camera and microphone to simultaneously collect emotional data such as facial expressions and voice.

[0234] Step 3:

[0235] Users (students) log in to the learning app and select the designated subject or unit. Every time they solve a problem or watch a video, their actions are automatically recorded on the device. Emotional data obtained using the camera and microphone is also recorded. When they finish learning, they log out.

[0236] Step 4:

[0237] The server stores the received learning data and emotion data in a database, and at the same time inputs the data into the generative AI model and emotion engine to execute the analysis job. Specifically, the data stored in the database is periodically put into the analysis queue, which triggers the generative AI model and emotion engine to perform analysis.

[0238] Step 5:

[0239] The generative AI model and emotion engine evaluate each student's learning style, level of understanding, and emotional state based on collected learning and emotion data. The analysis results are used to generate optimal learning plans and personalized learning materials. Specifically, the generative AI model identifies areas that need reinforcement based on a student's past performance, learning patterns, and emotions.

[0240] Step 6:

[0241] The server sends the personalized learning materials and learning plans generated by the generative AI model and emotion engine to the device. Specifically, the generated learning materials and plans are delivered to the device via API and displayed on the device's user interface.

[0242] Step 7:

[0243] The device then displays the received personalized learning materials and learning plans on a dashboard for students to access. Specifically, the device analyzes the learning material data received from the server and displays it in a user-friendly format.

[0244] Step 8:

[0245] The user (student) progresses through their studies using personalized learning materials provided on the device's dashboard. Specifically, students work through the provided practice problems and instructional videos, and their progress and results are automatically recorded on the device. Emotional data is also continuously recorded and sent to the server in real time.

[0246] Step 9:

[0247] The device records activity, progress, and emotional data during learning in real time and periodically transmits them to the server. Specifically, at the end of each learning session, the device compiles the session data and sends it to the server as a POST request.

[0248] Step 10:

[0249] The server continuously receives new learning and emotion data and re-evaluates the learning plan based on the analysis results. Specifically, it incorporates the newly received data into the analysis job and retrains the generative AI model and emotion engine accordingly to optimize the learning plan.

[0250] Step 11:

[0251] Teachers can check students' progress, analysis results, and emotional state in real time through the server's dashboard. Specifically, the teacher's management screen monitors each student's learning progress, level of understanding, and emotional state, and provides feedback as needed.

[0252] Step 12:

[0253] The user (teacher) can input supplementary materials and advice into the system based on each student's learning progress and emotional state, and provide them to the student. Specifically, the teacher uses the feedback function on the dashboard to input individual advice and supplementary materials, which are then automatically sent to the student's device.

[0254] Example 2

[0255] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0256] Conventional learning support systems often only collect and analyze students' learning activity data and do not consider their emotional state, making it difficult to improve the quality of education. Furthermore, they often fail to generate personalized learning materials and learning plans, resulting in ineffective instruction. Furthermore, teachers have limited means to monitor students' learning progress in real time and provide appropriate feedback. Therefore, there is a need for systems that can provide a more effective learning environment and realize education that meets the needs of each individual student.

[0257] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes means for collecting learning activity data and emotional data of students, means for analyzing the collected learning data and emotional data using a generative AI model and an emotion engine, and means for generating individualized learning materials and learning plans based on the analysis results. This makes it possible to evaluate students' learning styles, levels of comprehension, and emotional states in real time and provide optimal learning plans and learning materials.

[0258] "Student" refers to a learner who uses the learning support system.

[0259] "Learning activity data" refers to information such as operation logs, answer data, and study time generated by students while they are studying.

[0260] "Emotional data" refers to information about a student's emotional state obtained from facial expressions, tone of voice, etc.

[0261] A "generative AI model" refers to an artificial intelligence algorithm that automatically generates learning plans and teaching materials based on data.

[0262] An "emotion engine" refers to software that analyzes students' emotional state and provides the results to a learning support system.

[0263] "Analysis method" refers to the process of analyzing collected data using generative AI models and emotion engines.

[0264] "Personalized learning materials" refers to learning materials that are optimized based on a student's learning style and level of understanding.

[0265] A "learning plan" refers to a set of learning tasks or schedule designed to efficiently carry out students' learning activities.

[0266] "Monitoring" refers to the process of continuously observing students' academic progress and emotional state.

[0267] "Feedback" refers to the guidance and advice that teachers provide to students based on their learning progress and emotional state.

[0268] The present invention relates to a learning support system that combines a generative AI model and an emotion engine, and is specifically implemented as follows.

[0269] server

[0270] The server is the core of the learning support system and performs the following functions:

[0271] 1. Data Collection and Storage

[0272] The server sets up an API endpoint to receive learning activity data and emotion data sent from the device, which is then stored in a database and queued for analysis.

[0273] 2. Data Analysis

[0274] The server combines a generative AI model and an emotion engine to perform analysis. Specifically, it evaluates students' learning style, comprehension, and emotional state based on collected learning activity data and emotion data. These evaluation results are used to generate optimal learning plans and personalized learning materials.

[0275] 3. Creation and distribution of teaching materials

[0276] Based on the evaluation results, the server uses a generative AI model to generate personalized learning materials and study plans. The generated learning materials are sent to the corresponding student's device. For example, if the server determines that Student A is feeling anxious about a problem based on math lesson data and emotional data, it will generate an explanatory video for relaxation and additional practice problems and deliver them to the device.

[0277] Terminal

[0278] The device functions as a learning interface for users (students) and is implemented on a tablet or PC. It performs the following functions:

[0279] 1. Data Collection

[0280] When a student logs in to their device, the learning app starts and records learning activity data and emotional data in real time. The app uses a camera and microphone to collect emotional data from facial expressions and tone of voice, and sends it to a server.

[0281] 2. Viewing learning content

[0282] The personalized learning materials and study plans sent from the server are displayed on a dashboard on the device, where students can select the materials that are most suitable for them and proceed with their studies.

[0283] 3. Feedback

[0284] The students' answers, learning progress, and emotional state are recorded in real time and fed back to the server. For example, if Student B loses concentration while working on a math problem, the device detects this emotional state and immediately sends the data to the server.

[0285] Users (students and teachers)

[0286] The user is also an important part of the system and performs the following functions:

[0287] 1. Students

[0288] Students use their devices to study materials that are optimized for them, and their behavior, progress, and emotional state are automatically recorded by the devices, allowing them to focus on their studies independently.

[0289] 2. Teacher

[0290] Teachers access the server and monitor students' learning progress, analysis results, and emotional states in real time. Based on this, they provide feedback and additional supplementary materials or advice as needed. For example, Teacher C uses the server's dashboard to check the progress and emotional state of a specific student and adjust the teaching plan as needed.

[0291] Prompt Sentence Examples

[0292] Below are some examples of specific prompt sentences.

[0293] 1. Prompts to analyze student progress

[0294] Analyze the following dataset to assess student progress and understanding: {data_set}

[0295] 2. Prompt sentences for generating teaching materials based on emotion data

[0296] Analyze the following emotion data and generate personalized math exercises adapted to the emotion: {emotion_data}

[0297] In this way, this system utilizes generative AI models and emotion engines to provide optimal learning methods for students, reduce the workload of teachers, and improve the quality of education by creating an environment in which all students have equal access to learning opportunities.

[0298] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0299] Step 1: Data collection and transmission

[0300] Terminal

[0301] The device launches the learning app when the user (student) logs in.

[0302] The device records students' learning activity data (operation logs, answer data, study time) in real time through the learning app.

[0303] The device uses a camera and microphone to collect emotional data such as students' facial expressions and tone of voice.

[0304] The terminal periodically transmits this data to the server.

[0305] Input: Learning activity data and emotion data of logged-in students

[0306] Output: Learning activity data and emotion data sent to the server

[0307] Specific behavior:

[0308] While Student A logs in to the device and solves math problems, the device records the operation log and progress of the answer in real time. The camera also captures Student A's facial expressions and generates data to be sent to the emotion engine.

[0309] Step 2: Data storage and analysis

[0310] server

[0311] The server receives the learning activity data and emotion data sent from the terminal at an API endpoint.

[0312] The server stores the received data in a database.

[0313] The server puts the stored data into an analysis queue and performs the analysis using generative AI models and emotion engines.

[0314] The server evaluates the student's learning style, level of understanding, and emotional state based on learning data and emotional data.

[0315] Input: Learning activity data and emotion data sent from the device

[0316] Output: Assessment results of student learning style, comprehension, and emotional state

[0317] Specific behavior:

[0318] The server receives Student A's learning log and emotional data and stores them in a database. Next, it inputs a prompt statement to the generative AI model: "Analyze Student A's learning activity data and emotional data, and evaluate his learning style and level of comprehension.", and obtains the evaluation results.

[0319] Step 3: Generate and deliver personalized learning materials

[0320] server

[0321] The server generates optimal learning plans and personalized learning materials based on the analysis results.

[0322] The server generates teaching materials using the generative AI model and sends the prepared teaching materials to the corresponding student's device.

[0323] Input: Assessment results of student learning style, comprehension, and emotional state

[0324] Output: personalized learning materials and learning plans

[0325] Specific behavior:

[0326] Based on the evaluation results, the server inputs a prompt to the generative AI model, such as "Generate an instructional video for relaxation," and generates a personalized instructional video and additional practice questions, which are then sent to the device.

[0327] Step 4: View your learning and receive progress feedback

[0328] Terminal

[0329] The device displays the learning materials and study plans sent from the server on a dashboard.

[0330] The device records the student's answers, learning progress, and emotional state in real time and provides feedback to the server.

[0331] Input: Personalized learning materials and study plans sent from the server

[0332] Output: Data on students' learning progress and emotional state

[0333] Specific behavior:

[0334] Student B selects a personalized math problem set from the device's dashboard and proceeds with the answering process. Meanwhile, the device records Student B's answers and emotional state in real time and sends the data to the server.

[0335] Step 5: Feedback and Action

[0336] User (Teacher)

[0337] Teachers access the server and monitor students' learning progress, analysis results, and emotional state in real time.

[0338] Teachers provide feedback and supplementary materials and advice as needed.

[0339] Input: Student learning progress data and emotion data stored on the server

[0340] Output: Feedback, teaching aids, and advice for students

[0341] Specific behavior:

[0342] Teacher C uses the server's dashboard to check Student C's progress and emotional state. If Student C lacks understanding of a particular concept and is feeling stressed, Teacher C can provide additional quizzes or relaxation advice.

[0343] (Application example 2)

[0344] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0345] In the work environment of factories, there is a demand for improving work efficiency and safety, but conventional methods have limitations. There is no system that collects and analyzes workers' work data and emotional data in real time, and then provides individualized work guidance and safety measures, which leads to problems such as reduced work efficiency and a lack of safety.

[0346] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for collecting work data and emotion data of workers, means for analyzing the collected work data and emotion data using a generative AI model, and means for generating individualized work instructions and safety measures based on the analysis results. This makes it possible to provide optimal work instructions and safety measures in real time based on the work data and emotion data of workers.

[0347] "Work data" refers to data related to the work content, procedures, and efficiency of workers, and is information collected by measuring devices and sensors.

[0348] "Emotional data" refers to data that indicates the emotional state of a worker, detected from their facial expressions, voice, body movements, etc., and is acquired using sensors such as cameras and microphones.

[0349] A "generative AI model" is an algorithm that analyzes collected data and generates optimal guidance and countermeasures for workers, and refers to an artificial intelligence system that uses machine learning and deep learning technologies.

[0350] "Means of analysis" refers to the process of analyzing data using a generative AI model based on collected task data and emotion data, and extracting meaningful information.

[0351] "Individualized work instruction" refers to information and instructions that guide optimal work procedures and methods according to the abilities and conditions of each individual worker.

[0352] "Safety measures" refers to measures, including actions and protective measures, that must be taken to minimize risks during work.

[0353] "Worker" refers to a person who performs work in a factory or work site.

[0354] "Manager" refers to a person whose role is to monitor the work data and progress of workers and provide feedback as needed.

[0355] "Real-time monitoring" refers to instantly collecting and analyzing work data and emotional data to grasp the worker's current state at every moment.

[0356] "Means for providing feedback" refers to a method or system that provides instructions or advice to workers based on analysis results or monitoring data.

[0357] The present invention relates to a work support system for factories. This system collects and analyzes work data and emotional data of workers, and provides individualized work guidance and safety measures.

[0358] server

[0359] The server is the core of the system and performs the following tasks:

[0360] 1. Collecting and storing task and emotion data.

[0361] 2. Analyze collected data using generative AI models.

[0362] 3. Generate and provide individualized work instructions and safety measures based on the analysis results.

[0363] Specifically, the server sets up an API endpoint and prepares to receive data sent from the device. The data is stored in a database and queued for subsequent analysis jobs. The generative AI model and emotion engine are combined to perform the analysis and evaluate the worker's working style and emotional state. Based on the evaluation results, optimal work instructions and safety measures are generated and sent to the corresponding worker's device.

[0364] The specific hardware used includes a database server and a GPU server for computing, and the software includes a database (e.g., MySQL), a deep learning framework (e.g., TensorFlow), and an emotion analysis library (e.g., EmotionEngine).

[0365] Terminal

[0366] The terminal functions as an interface for the worker and is mainly implemented as a smartphone, tablet, or wearable device. It collects work data and emotional data and sends it to the server in real time. Emotional data is collected using the device's camera and microphone, and changes in the worker's facial expressions and voice are detected and sent to the emotion engine.

[0367] The server sends personalized work instructions and safety measures to the device, which are then displayed on a dashboard. The worker receives instructions from the dashboard and puts them into action. The device also records the worker's work progress and emotional state and sends feedback to the server.

[0368] Users (operators and administrators)

[0369] Workers receive personalized guidance and safety measures via their devices. Their movements and emotional state are automatically recorded by the devices, allowing them to concentrate on their work. Supervisors can access the server and monitor the workers' work progress, analysis results, and emotional state in real time. Based on this, supervisors can provide feedback to workers and offer additional guidance and advice.

[0370] For example, when a worker is assembling parts, work data and emotional data are collected via the device's camera. If the server analyzes this data and determines that the worker is under stress, instructions to adjust the work pace or take a break are sent to the device. The worker can follow these instructions and continue working safely and efficiently.

[0371] An example of a prompt is, "If a tired expression is detected, please generate advice on work pace." The generative AI model generates appropriate advice based on this prompt, optimizing the worker's working environment.

[0372] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0373] Step 1: Data collection

[0374] Terminal: Using cameras and sensors installed on the terminal, the work data and emotional data of the worker are collected in real time. During this collection process, changes in the worker's movements, facial expressions, and voice are detected. The input is the real-time data of the worker, and the output is the collected work data and emotional data.

[0375] Step 2: Send data

[0376] Terminal: The collected task data and emotion data are periodically sent to the server. The input is the collected data, and the output is the data sent to the server. Specifically, the data is sent to the server via an API.

[0377] Step 3: Save Data

[0378] Server: Stores the received task data and emotion data in a database. The input is the data sent from the device, and the output is the stored data. Specifically, it creates a database entry and stores the data.

[0379] Step 4: Submit a data analysis job

[0380] Server: Queues jobs to analyze the stored data. The input is the stored data, and the output is the enqueuing of analysis jobs.

[0381] Step 5: Data analysis

[0382] Server: Analyzes the analysis jobs registered in the queue using a generative AI model and an emotion engine. The input is the data entered in the queue, and the output is the analysis results. Specifically, the generative AI model is used to evaluate the worker's working style and emotional state.

[0383] Step 6: Generate personalized task instructions

[0384] Server: Generates individualized work instructions and safety measures based on the analysis results. The input is the analysis results, and the output is the generated work instructions and safety measures. For example, this includes adjusting work pace and instructions for breaks.

[0385] Step 7: Sending work instructions

[0386] Server: Sends the generated work instructions and safety measures to the terminal. The input is the generated instruction content, and the output is the transmission to the terminal. Specifically, the instruction content is sent to the terminal as JSON format data via API.

[0387] Step 8: Work Instructions

[0388] Terminal: The received work instructions and safety measures are displayed on a dashboard and notified to the worker. The input is the data received from the server, and the output is the displayed instruction content. Specifically, the information is displayed using UI components.

[0389] Step 9: Progress Feedback

[0390] Terminal: The worker continues working based on the instructions and records their progress. The recorded progress data and new emotion data are sent to the server again. The input is the worker's progress data and emotion data, and the output is the data sent to the server.

[0391] Step 10: Reassess your progress data

[0392] Server: Analyzes the received progress data and emotion data again and generates new guidance and countermeasures. The input is the collected data again, and the output is the new guidance content. Specifically, the data is analyzed using the generative AI model and emotion engine, just like last time.

[0393] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0394] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0395] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0396] [Second embodiment]

[0397] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0398] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0399] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0400] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0401] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0402] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0403] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0404] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0405] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0406] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0407] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0408] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0409] The present invention relates to a learning support system, and in particular to a system for personalized learning support using a generative AI model. Specific embodiments for carrying out the present invention will be described below.

[0410] This system consists of three components: students, terminals, and a server. The roles and processing flow of each component are described in detail below.

[0411] server

[0412] The server is the core of the system, responsible for collecting and analyzing learning activity data, operating the generative AI model, and generating and distributing personalized learning materials. The server first sets up an API endpoint and receives learning data sent from the device. The received data is stored in a database and queued for subsequent analysis jobs.

[0413] The server then uses a generative AI model to analyze the collected learning data. The analysis evaluates each student's learning style, level of understanding, and progress, and generates personalized learning materials and study plans based on the results. These generated learning materials are then managed by the server and sent to the corresponding student's device.

[0414] As a specific example, if the server determines, based on data acquired during a mathematics class, that Student A is struggling with a particular concept in a practice problem, it will generate and distribute additional practice problems and explanatory videos to reinforce that concept.

[0415] Terminal

[0416] The device is the interface for students to study and is typically implemented as a tablet or PC. The device communicates data between the student and the server via a learning app. When a student logs in, the device records their activity in real time and periodically sends data to the server.

[0417] The personalized learning materials and study plans sent from the server are displayed on the device's dashboard. Students can select the materials that are most suitable for them and proceed with their studies from this dashboard. The device also instantly records the student's answers and progress and sends them back to the server.

[0418] As a concrete example, Student B works on a specific math problem set provided on his / her device, and progresses through his / her studies while looking at the answers and explanations for each problem. At this time, the device constantly records progress information and sends it to the server.

[0419] Users (students and teachers)

[0420] The user is also an important part of the system. Students use their devices to study materials that are optimized for them. Their learning behavior and progress are automatically recorded by the devices, allowing students to focus on their studies independently.

[0421] Teachers can access the server and monitor students' learning progress and analysis results in real time, and based on this, teachers can provide feedback to students and offer additional teaching materials and advice.

[0422] For example, Teacher C can use the server's dashboard to monitor the progress of the entire class and individual students, adjust the lesson plan as needed, and provide additional supplementary materials if a particular student is found to have a lack of understanding of a particular concept.

[0423] This system reduces the workload of teachers while providing students with learning methods that are suited to them, achieving the goal of creating an environment in which all students have equal access to learning opportunities and improving the quality of education.

[0424] The processing flow will be explained below.

[0425] Step 1:

[0426] The server opens an API endpoint for collecting learning data and prepares to receive the learning data. Specifically, the server builds an API to receive student learning activity data (e.g., subjects taken, study time, progress) sent from the device. This data is stored in a database.

[0427] Step 2:

[0428] When a student logs in to the learning app, the device starts recording session information. Specifically, the device records the user's ID and login time at login, collects data each time a learning activity is started, and periodically sends the data to the server.

[0429] Step 3:

[0430] Users (students) log in to the learning app on their tablet or PC and select the designated subject or unit. When students solve problems or watch videos, their actions are automatically recorded on the device. When they finish learning, they log out.

[0431] Step 4:

[0432] The server stores the received training data in a database, and at the same time inputs the data into the generative AI model and executes the analysis job. Specifically, the data stored in the database is periodically put into the analysis queue, which triggers the generative AI model to perform analysis.

[0433] Step 5:

[0434] The generative AI model evaluates each student's learning style and level of understanding based on collected learning data. The analysis results are used to generate optimal learning plans and personalized learning materials. Specifically, the model analyzes students' past performance and learning patterns to identify areas that need reinforcement.

[0435] Step 6:

[0436] The server sends the personalized learning materials and learning plans generated by the generative AI model to the device. Specifically, the generated learning materials and plans are delivered to the device via API and displayed on the device's user interface.

[0437] Step 7:

[0438] The device then displays the received personalized learning materials and learning plans on a dashboard for students to access. Specifically, the device analyzes the learning material data received from the server and displays it in a user-friendly format.

[0439] Step 8:

[0440] Students can study using personalized learning materials provided on the device's dashboard. Specifically, students work through the provided practice problems and instructional videos, and their progress and results are automatically recorded on the device.

[0441] Step 9:

[0442] The device records learning activity and progress data in real time and periodically sends it to the server. Specifically, at the end of each learning session, the device compiles the session data and sends it to the server as a POST request.

[0443] Step 10:

[0444] The server continuously receives new training data and re-evaluates the training plan based on the analysis results. Specifically, the server incorporates the newly received data into the analysis job and retrains the generative AI model accordingly to optimize the training plan.

[0445] Step 11:

[0446] Teachers can check students' progress and analysis results in real time through the server's dashboard. Specifically, the teacher's management screen monitors each student's learning progress and level of understanding, and provides feedback as needed.

[0447] Step 12:

[0448] The user (teacher) can input supplementary materials and advice into the system according to each student's learning progress and provide them to the student. Specifically, the teacher uses the feedback function on the dashboard to input individual advice and supplementary materials, which are then automatically sent to the student's device.

[0449] Example 1

[0450] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0451] Conventional learning support systems have had difficulty providing personalized learning materials tailored to each student's learning style and level of understanding in a timely manner. They also lacked the means to continuously monitor learning progress and provide appropriate feedback in real time. This resulted in issues such as students not being able to maximize their learning effectiveness and increasing the workload of teachers.

[0452] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0453] In this invention, the server includes means for collecting student learning activity data from the terminal, means for storing the collected learning data in a database and queueing it for analysis jobs, means for analyzing the learning data using a generative AI model and evaluating each student's learning style, level of understanding, and progress, means for generating personalized learning materials and learning plans based on the analysis results, means for sending the generated learning materials and learning plans to the student's terminal, means for recording students' answers and progress and providing feedback to the server, and means for continuously monitoring students' learning progress and reevaluating the learning plans based on new data. This enables personalized learning support optimized for each student, promoting effective learning and reducing the workload of teachers.

[0454] "Devices" refer to electronic devices such as tablets and computers that function as interfaces for students to learn.

[0455] "Learning activity data" refers to all data generated when a student engages in learning activities, and specifically includes answers to questions, answer times, progress, etc.

[0456] "Database" refers to an information management system for organizing and storing collected learning data.

[0457] An "analysis job" refers to a series of processes set up to analyze collected learning data.

[0458] A "generative AI model" refers to a model that uses artificial intelligence technology to analyze learning data and generate personalized teaching materials and learning plans.

[0459] "Teaching materials" refers to materials, workbooks, explanatory videos, etc. that students use for studying.

[0460] A "study plan" refers to a plan that includes learning methods and goals, specific learning content, and schedules that are optimized for each student.

[0461] "Feedback" refers to the server or teacher providing evaluations and advice on students' learning activities.

[0462] "Monitoring" refers to the activity of continuously observing students' learning progress and collecting data as necessary.

[0463] "Reevaluation" refers to reviewing an existing learning plan based on a student's latest learning data and revising or updating it as necessary.

[0464] MODE FOR CARRYING OUT THE INVENTION

[0465] This invention relates to a learning support system, specifically a system for personalized learning support using a generative AI model. This system consists of three entities: a student, a terminal, and a server. The roles of each entity and their specific processing are described in detail below.

[0466] server

[0467] The server is the core of the system, responsible for collecting and analyzing learning activity data, operating the generative AI model, and generating and distributing personalized learning materials. The specific operations of the server are as follows:

[0468] The server sets up an API endpoint to receive learning data sent from the device, for example, the results of a student's math problem answers.

[0469] The received data is stored in a database and queued for subsequent analysis jobs.

[0470] The server analyzes the collected training data using a generative AI model, such as a natural language processing model like GPT-3.

[0471] The system uses analytics to assess each student's learning style, level of understanding, and progress, and then generates personalized learning materials and lesson plans based on the results. For example, if a student is struggling with a math problem, the system generates materials to reinforce that concept.

[0472] The generated teaching materials are again managed by the server and sent to the corresponding student's terminal.

[0473] For example, the server inputs the following prompt sentence into the generative AI model for analysis:

[0474] "Student A is struggling with solving a quadratic equation during a math class and has sent the data to the server. The server should analyze this data and generate additional practice problems and instructional videos for Student A."

[0475] Terminal

[0476] The device is the interface through which students learn, and is typically implemented as a tablet or PC. The specific operations of the device are as follows:

[0477] When a student logs in to a device, the device records their activity in real time and periodically sends data to the server, such as recording the questions the student answered and their results.

[0478] The personalized learning materials and learning plans sent from the server are displayed on a dashboard on the device, which is built using front-end frameworks such as React and Angular.

[0479] Students can use this dashboard to select the learning materials that are most suitable for them and proceed with their studies.

[0480] Students' answers and progress are recorded by the device and fed back to the server.

[0481] For example, Student B works on a specific math problem set provided on his / her device, and progresses through his / her studies while looking at the answers and explanations for each problem. At this time, the device records the answers and sends them to the server.

[0482] Users (students and teachers)

[0483] The user is also an important part of the system. Here are some specific user actions:

[0484] Students study materials optimized for them through their devices, and their learning behavior and progress are automatically recorded by the devices, allowing them to concentrate on their studies.

[0485] Teachers can access the server and monitor students' learning progress and analysis results in real time. For example, teachers can access the dashboard from a browser to check students' grades and progress.

[0486] Based on this, the teacher can provide feedback to the student and provide additional support materials or advice. For example, if a particular student shows a lack of understanding of a particular theorem, the teacher can provide that student with additional learning materials.

[0487] For example, Teacher C can use the server's dashboard to monitor the progress of the entire class and individual students, adjust the lesson plan as needed, and provide additional supplementary materials if a particular student is found to have a lack of understanding of a particular concept.

[0488] As described above, by using a generative AI model, the present invention can provide personalized educational support tailored to each student's learning style and progress, thereby reducing the workload of teachers and providing a more effective learning environment for students.

[0489] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0490] Step 1: Device collects student learning activity data

[0491] How it works: Students log in to their devices and begin working on math problems. The devices then record data in real time, such as the answers and time it took to complete each problem, via a learning app.

[0492] Input: Answer data, answer time, and progress of students logged in to the device

[0493] Output: Learning activity data recorded by the device

[0494] Step 2: The device sends the learning activity data to the server.

[0495] Specific operation: The device sends the student's learning data to the server at regular intervals or when the learning activity is completed, for example, after each question is answered.

[0496] Input: Learning activity data recorded on the device

[0497] Output: Learning activity data sent to the server

[0498] Step 3: The server saves the training data to a database and queues it for analysis jobs.

[0499] Specific operation: After receiving the training data, the server stores it in a database. The stored data is then queued for analysis jobs. This process is automated, enabling efficient data management.

[0500] Input: Learning activity data sent to the server

[0501] Output: Learning activity data stored in the database, data input to the analysis job

[0502] Step 4: The server analyzes the training data using the generated AI model

[0503] How it works: The server takes the data from the queue and inputs it into the generative AI model, which analyzes each student's learning style and level of understanding, and then generates optimal teaching materials and lesson plans based on the results.

[0504] Input: Training data submitted to the analysis job

[0505] Output: Analysis of each student's learning style, level of understanding, and progress

[0506] Step 5: The server generates personalized learning materials and learning plans based on the analysis results.

[0507] What it does: Based on the analysis, it uses a generative AI model to generate personalized learning materials and lesson plans. For example, if a particular student is struggling with a particular math concept, it generates practice exercises and instructional videos that reinforce that concept.

[0508] Input: Analysis of each student's learning style, level of understanding, and progress

[0509] Output: personalized learning materials and learning plans

[0510] Step 6: The server sends the generated learning materials and lesson plans to the students' devices.

[0511] Specific operation: The server sends the generated personalized teaching materials and learning plans to the corresponding student's device.

[0512] Input: Generated personalized learning materials and learning plans

[0513] Output: Materials and lesson plans sent to the corresponding student devices

[0514] Step 7: The device displays the sent learning material and the student continues learning based on it.

[0515] Specific operation: The device displays the learning materials and study plans received from the server on a dashboard. Students can select the appropriate learning materials from this dashboard and proceed with their studies. For example, they can work on the displayed problem sets and progress through the answers.

[0516] Input: Personalized learning materials and study plans sent from the server

[0517] Output: Materials and lesson plans displayed on the dashboard, and student learning activities

[0518] Step 8: The device records the student's answers and progress and sends them back to the server.

[0519] Specific operation: The device again records the student's answers and progress and sends them back to the server. This data is used for analysis in the next cycle.

[0520] Input: Student answers and progress

[0521] Output: Answer results and progress feedback to the server

[0522] Step 9: The server continuously monitors your progress and reevaluates your learning plan based on new data.

[0523] Specific operation: The server continuously monitors learning progress based on the feedback data, analyzes new data, and reevaluates and modifies the learning plan.

[0524] Input: Feedback of answers and progress data

[0525] Output: Updated learning plan

[0526] This series of steps enables effective learning support that is customized for each student.

[0527] (Application example 1)

[0528] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0529] In modern factories, there is a demand for efficient training and improved skills for engineers and workers, but conventional training methods make it difficult to provide appropriate feedback to individual engineers, limiting productivity improvements. Another issue is the lack of technology to efficiently collect and analyze worker activity data and generate training plans tailored to each individual's skill level.

[0530] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0531] In this invention, the server includes means for collecting work activity data of workers, means for analyzing the collected data using a generative AI model, means for generating personalized training plans and learning materials based on the analysis results, means for providing the generated training plans and learning materials to workers, and means for continuously monitoring the progress of workers and reevaluating the training plans based on new data, thereby enabling efficient and personalized training to be provided to engineers and workers.

[0532] "Worker" refers to a person who performs a specific task in a factory or production site.

[0533] "Work activity data" refers to information about the process and results of work performed by workers, and is data collected through devices such as sensors and tablets.

[0534] A "generative AI model" refers to an artificial intelligence model that analyzes collected data and generates personalized training plans, learning materials, etc.

[0535] A "training plan" is a plan aimed at improving a worker's techniques and skills, and includes learning content optimized for each individual worker.

[0536] "Learning materials" refers to the materials and content that workers use to learn according to the training plan, including videos, workbooks, tutorials, etc.

[0537] "Means of provision" refers to the method of providing the generated training plans and learning materials to workers at an appropriate time.

[0538] "Progress monitoring" refers to the continuous observation and recording of workers' progress as they progress through training and their work.

[0539] "Reassessment" refers to reviewing existing training plans and learning methods based on new data collected and making improvements or changes as needed.

[0540] "Means for analyzing data" refers to methods and techniques for processing collected data and assessing worker skills and understanding.

[0541] This invention relates to a system that supports the skill improvement and efficient training of engineers and workers in factories. This system collects data on workers' work activities, analyzes the data using a generative AI model, and generates and provides personalized training plans and learning materials based on the analysis results.

[0542] Hardware and software used

[0543] Hardware

[0544] Training tablet or computer (device)

[0545] Data collection sensors

[0546] Learning information storage server

[0547] GPU server for high-speed calculations (operation of generative AI models)

[0548] software

[0549] Learning Management System (LMS)

[0550] Python-based analysis program

[0551] Generative AI model library (PyTorch or TensorFlow)

[0552] Database management system (PostgreSQL)

[0553] RESTful API (Flask or Django)

[0554] Explanation of the processing flow

[0555] 1. Data Collection

[0556] The server uses sensors and devices to collect work activity data when workers run the training program on their devices. This data includes information about the work process and results. For example, sensors record the time and steps for each process in the assembly of parts.

[0557] 2. Data Analysis

[0558] The server stores the collected work activity data in a database and analyzes it using a generative AI model. The analysis evaluates the worker's skill level and tendency for operational errors. The generative AI model is implemented using a Python-based analysis program and libraries such as PyTorch and TensorFlow.

[0559] 3. Training plan generation

[0560] Based on the analysis results, the server generates personalized training plans and learning materials. These training plans include video tutorials and problem sets that include specific work content and areas for improvement. For example, for the assembly procedure of a new part, the server generates materials that include countermeasures for common mistakes made by workers.

[0561] 4. Submissions and Feedback

[0562] The generated training plan and learning materials are sent to the terminal. As the worker progresses through the training, new data is collected and sent back to the server. This allows the server to reevaluate the training plan based on the latest data and make adjustments as necessary. This feedback loop ensures that the most optimal training is always provided.

[0563] Specific examples

[0564] When training workers on assembly procedures for a new part, they watch a video tutorial on a tablet and then perform a training simulation. Sensors and the tablet collect data during the simulation and send it to a server. A generative AI model analyzes the data and generates additional teaching materials for areas where mistakes are common, which are then sent back to the tablet. This process allows workers to effectively improve their skills.

[0565] Prompt Sentence Examples

[0566] "Please suggest the best training content for the next lesson. Operator A made many mistakes when assembling part B. Please generate teaching materials that include specific problem areas and how to improve them."

[0567] This invention makes it possible to provide efficient and individualized training to engineers and workers, which is expected to improve productivity.

[0568] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0569] Step 1:

[0570] The server collects work activity data from sensors and devices when a worker runs a training program on the device. When the worker presses the start button, the device starts collecting sensor data and sends data about the work process and results to the server. The input is data from the sensors and devices, and the output is raw data stored on the server.

[0571] Step 2:

[0572] The server stores the collected data in a database and performs data cleansing, removing incomplete or noisy data and shaping the data. The input is the raw collected data, and the output is cleaned data that can be analyzed.

[0573] Step 3:

[0574] The server analyzes the cleaned data using a generative AI model. A Python-based analysis program is used to evaluate the worker's skill level and tendency for operational errors. The input is the cleaned data, and the output is a technical evaluation report.

[0575] Step 4:

[0576] The server generates personalized training plans and learning materials based on the analysis results. It uses a generative AI model library (PyTorch or TensorFlow) to generate optimal learning materials for each worker. The input is the technical evaluation report, and the output is personalized training plans and learning materials.

[0577] Step 5:

[0578] The server distributes the generated training plan and learning materials to the terminal. The terminal receives them and provides them to the worker. The worker checks the training plan and learning materials on the terminal and progresses with their learning. The input is the individualized training plan and learning materials, and the output is the worker's learning progress.

[0579] Step 6:

[0580] The terminal collects the worker's training progress data and sends it to the server, including the training completion status and any new mistakes. The input is the worker's progress data, and the output is the data sent to the server.

[0581] Step 7:

[0582] The server re-evaluates the training plan based on the new data received, again using the generative AI model to adjust the training content as needed. The input is the new progress data, and the output is the adjusted training plan.

[0583] Step 8:

[0584] The server then delivers the re-evaluated training plan and learning materials back to the terminal and provides them to the worker. This feedback loop ensures continuous optimal training. The input is the adjusted training plan and learning materials, and the output is the worker's latest learning progress.

[0585] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0586] This invention relates to a learning support system that combines a generative AI model and an emotion engine, and provides means for collecting learning activity data, generating personalized learning materials, managing learning progress, and monitoring the user's emotional state. The system of this invention consists of three main entities: a server, a terminal, and a user. The specific roles and processing flow of each entity are explained below.

[0587] server

[0588] The server is the core of the system, responsible for collecting and analyzing learning activity data and emotion data, operating the generative AI model, and generating and providing personalized learning materials and learning plans. First, the server sets up an API endpoint and prepares to receive data sent from the device. The received data is stored in a database and queued for subsequent analysis jobs.

[0589] The server then combines the generative AI model with an emotion engine to perform analysis. It uses the learning data and emotion data to evaluate the student's learning style, comprehension, and emotional state. Based on the evaluation results, it generates an optimal learning plan and personalized learning materials. The generated learning materials are then sent to the corresponding student's device.

[0590] As a specific example, if the server determines that Student A is feeling anxious about a problem based on data and emotional data acquired during a mathematics class, it will generate and deliver additional practice problems that take into account the student's emotional state and explanatory videos for relaxation.

[0591] Terminal

[0592] The device functions as an interface for users (students) to study and is implemented on a tablet or PC. Data is transmitted between the student and the server via a learning app. When a student logs in, the device records their activity in real time and periodically sends the data to the server. Emotional data is also collected in the same way; for example, changes in the student's facial expression or voice are detected using a camera or microphone, and sent to the emotion engine.

[0593] The personalized learning materials and study plans sent from the server are displayed on a dashboard on the device. Students can select the materials that are most suitable for them and proceed with their studies from this dashboard. The device also instantly records the students' answers, progress, and emotional state and provides feedback to the server.

[0594] For example, if Student B is working on a specific math problem set provided on his / her device and loses concentration while solving it, his / her emotional state will be detected by the device, which will record it and send it to the server.

[0595] Users (students and teachers)

[0596] The user is also an important part of the system. Students use their devices to study materials that are optimized for them. Their behavior, progress, and emotional state are automatically recorded by the devices, allowing them to focus on their studies independently.

[0597] Teachers can access the server to monitor students' learning progress, analysis results, and emotional state in real time, and based on this, teachers can provide feedback to students and provide additional teaching materials and advice.

[0598] For example, Teacher C can use the server's dashboard to monitor the progress and emotional state of the entire class and individual students, adjust the lesson plan as needed, and provide additional supplementary materials or relaxation advice if a particular student is found to be struggling with a particular concept and feeling stressed.

[0599] In this way, learning support systems can reduce the workload of teachers while providing students with learning methods that are suited to them. The goal is to create an environment in which all students can enjoy equal learning opportunities and improve the quality of education.

[0600] The processing flow will be explained below.

[0601] Step 1:

[0602] The server opens an API endpoint to collect learning data and emotional data. First, it prepares to receive learning requests, and then receives student learning activity data (e.g., subjects taken, study time, progress) and emotional data sent from the device. This data is then stored in a database.

[0603] Step 2:

[0604] The device starts recording session information when a student logs in to the learning app. Specifically, it records the user ID and login time when logging in to study, and collects data each time a learning activity begins. It also uses the camera and microphone to simultaneously collect emotional data such as facial expressions and voice.

[0605] Step 3:

[0606] Users (students) log in to the learning app and select the designated subject or unit. Every time they solve a problem or watch a video, their actions are automatically recorded on the device. Emotional data obtained using the camera and microphone is also recorded. When they finish learning, they log out.

[0607] Step 4:

[0608] The server stores the received learning data and emotion data in a database, and at the same time inputs the data into the generative AI model and emotion engine to execute the analysis job. Specifically, the data stored in the database is periodically put into the analysis queue, which triggers the generative AI model and emotion engine to perform analysis.

[0609] Step 5:

[0610] The generative AI model and emotion engine evaluate each student's learning style, level of understanding, and emotional state based on collected learning and emotion data. The analysis results are used to generate optimal learning plans and personalized learning materials. Specifically, the generative AI model identifies areas that need reinforcement based on a student's past performance, learning patterns, and emotions.

[0611] Step 6:

[0612] The server sends the personalized learning materials and learning plans generated by the generative AI model and emotion engine to the device. Specifically, the generated learning materials and plans are delivered to the device via API and displayed on the device's user interface.

[0613] Step 7:

[0614] The device then displays the received personalized learning materials and learning plans on a dashboard for students to access. Specifically, the device analyzes the learning material data received from the server and displays it in a user-friendly format.

[0615] Step 8:

[0616] The user (student) progresses through their studies using personalized learning materials provided on the device's dashboard. Specifically, students work through the provided practice problems and instructional videos, and their progress and results are automatically recorded on the device. Emotional data is also continuously recorded and sent to the server in real time.

[0617] Step 9:

[0618] The device records activity, progress, and emotional data during learning in real time and periodically transmits them to the server. Specifically, at the end of each learning session, the device compiles the session data and sends it to the server as a POST request.

[0619] Step 10:

[0620] The server continuously receives new learning and emotion data and re-evaluates the learning plan based on the analysis results. Specifically, it incorporates the newly received data into the analysis job and retrains the generative AI model and emotion engine accordingly to optimize the learning plan.

[0621] Step 11:

[0622] Teachers can check students' progress, analysis results, and emotional state in real time through the server's dashboard. Specifically, the teacher's management screen monitors each student's learning progress, level of understanding, and emotional state, and provides feedback as needed.

[0623] Step 12:

[0624] The user (teacher) can input supplementary materials and advice into the system based on each student's learning progress and emotional state, and provide them to the student. Specifically, the teacher uses the feedback function on the dashboard to input individual advice and supplementary materials, which are then automatically sent to the student's device.

[0625] Example 2

[0626] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0627] Conventional learning support systems often only collect and analyze students' learning activity data and do not consider their emotional state, making it difficult to improve the quality of education. Furthermore, they often fail to generate personalized learning materials and learning plans, resulting in ineffective instruction. Furthermore, teachers have limited means to monitor students' learning progress in real time and provide appropriate feedback. Therefore, there is a need for systems that can provide a more effective learning environment and realize education that meets the needs of each individual student.

[0628] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes means for collecting learning activity data and emotional data of students, means for analyzing the collected learning data and emotional data using a generative AI model and an emotion engine, and means for generating individualized learning materials and learning plans based on the analysis results. This makes it possible to evaluate students' learning styles, levels of comprehension, and emotional states in real time and provide optimal learning plans and learning materials.

[0629] "Student" refers to a learner who uses the learning support system.

[0630] "Learning activity data" refers to information such as operation logs, answer data, and study time generated by students while they are studying.

[0631] "Emotional data" refers to information about a student's emotional state obtained from facial expressions, tone of voice, etc.

[0632] A "generative AI model" refers to an artificial intelligence algorithm that automatically generates learning plans and teaching materials based on data.

[0633] An "emotion engine" refers to software that analyzes students' emotional state and provides the results to a learning support system.

[0634] "Analysis method" refers to the process of analyzing collected data using generative AI models and emotion engines.

[0635] "Personalized learning materials" refers to learning materials that are optimized based on a student's learning style and level of understanding.

[0636] A "learning plan" refers to a set of learning tasks or schedule designed to efficiently carry out students' learning activities.

[0637] "Monitoring" refers to the process of continuously observing students' academic progress and emotional state.

[0638] "Feedback" refers to the guidance and advice that teachers provide to students based on their learning progress and emotional state.

[0639] The present invention relates to a learning support system that combines a generative AI model and an emotion engine, and is specifically implemented as follows.

[0640] server

[0641] The server is the core of the learning support system and performs the following functions:

[0642] 1. Data Collection and Storage

[0643] The server sets up an API endpoint to receive learning activity data and emotion data sent from the device, which is then stored in a database and queued for analysis.

[0644] 2. Data Analysis

[0645] The server combines a generative AI model and an emotion engine to perform analysis. Specifically, it evaluates students' learning style, comprehension, and emotional state based on collected learning activity data and emotion data. These evaluation results are used to generate optimal learning plans and personalized learning materials.

[0646] 3. Creation and distribution of teaching materials

[0647] Based on the evaluation results, the server uses a generative AI model to generate personalized learning materials and study plans. The generated learning materials are sent to the corresponding student's device. For example, if the server determines that Student A is feeling anxious about a problem based on math lesson data and emotional data, it will generate an explanatory video for relaxation and additional practice problems and deliver them to the device.

[0648] Terminal

[0649] The device functions as a learning interface for users (students) and is implemented on a tablet or PC. It performs the following functions:

[0650] 1. Data Collection

[0651] When a student logs in to their device, the learning app starts and records learning activity data and emotional data in real time. The app uses a camera and microphone to collect emotional data from facial expressions and tone of voice, and sends it to a server.

[0652] 2. Viewing learning content

[0653] The personalized learning materials and study plans sent from the server are displayed on a dashboard on the device, where students can select the materials that are most suitable for them and proceed with their studies.

[0654] 3. Feedback

[0655] The students' answers, learning progress, and emotional state are recorded in real time and fed back to the server. For example, if Student B loses concentration while working on a math problem, the device detects this emotional state and immediately sends the data to the server.

[0656] Users (students and teachers)

[0657] The user is also an important part of the system and performs the following functions:

[0658] 1. Students

[0659] Students use their devices to study materials that are optimized for them, and their behavior, progress, and emotional state are automatically recorded by the devices, allowing them to focus on their studies independently.

[0660] 2. Teacher

[0661] Teachers access the server and monitor students' learning progress, analysis results, and emotional states in real time. Based on this, they provide feedback and additional supplementary materials or advice as needed. For example, Teacher C uses the server's dashboard to check the progress and emotional state of a specific student and adjust the teaching plan as needed.

[0662] Prompt Sentence Examples

[0663] Below are some examples of specific prompt sentences.

[0664] 1. Prompts to analyze student progress

[0665] Analyze the following dataset to assess student progress and understanding: {data_set}

[0666] 2. Prompt sentences for generating teaching materials based on emotion data

[0667] Analyze the following emotion data and generate personalized math exercises adapted to the emotion: {emotion_data}

[0668] In this way, this system utilizes generative AI models and emotion engines to provide optimal learning methods for students, reduce the workload of teachers, and improve the quality of education by creating an environment in which all students have equal access to learning opportunities.

[0669] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0670] Step 1: Data collection and transmission

[0671] Terminal

[0672] The device launches the learning app when the user (student) logs in.

[0673] The device records students' learning activity data (operation logs, answer data, study time) in real time through the learning app.

[0674] The device uses a camera and microphone to collect emotional data such as students' facial expressions and tone of voice.

[0675] The terminal periodically transmits this data to the server.

[0676] Input: Learning activity data and emotion data of logged-in students

[0677] Output: Learning activity data and emotion data sent to the server

[0678] Specific behavior:

[0679] While Student A logs in to the device and solves math problems, the device records the operation log and progress of the answer in real time. The camera also captures Student A's facial expressions and generates data to be sent to the emotion engine.

[0680] Step 2: Data storage and analysis

[0681] server

[0682] The server receives the learning activity data and emotion data sent from the terminal at an API endpoint.

[0683] The server stores the received data in a database.

[0684] The server puts the stored data into an analysis queue and performs the analysis using generative AI models and emotion engines.

[0685] The server evaluates the student's learning style, level of understanding, and emotional state based on learning data and emotional data.

[0686] Input: Learning activity data and emotion data sent from the device

[0687] Output: Assessment results of student learning style, comprehension, and emotional state

[0688] Specific behavior:

[0689] The server receives Student A's learning log and emotional data and stores them in a database. Next, it inputs a prompt statement to the generative AI model: "Analyze Student A's learning activity data and emotional data, and evaluate his learning style and level of comprehension.", and obtains the evaluation results.

[0690] Step 3: Generate and deliver personalized learning materials

[0691] server

[0692] The server generates optimal learning plans and personalized learning materials based on the analysis results.

[0693] The server generates teaching materials using the generative AI model and sends the prepared teaching materials to the corresponding student's device.

[0694] Input: Assessment results of student learning style, comprehension, and emotional state

[0695] Output: personalized learning materials and learning plans

[0696] Specific behavior:

[0697] Based on the evaluation results, the server inputs a prompt to the generative AI model, such as "Generate an instructional video for relaxation," and generates a personalized instructional video and additional practice questions, which are then sent to the device.

[0698] Step 4: View your learning and receive progress feedback

[0699] Terminal

[0700] The device displays the learning materials and study plans sent from the server on a dashboard.

[0701] The device records the student's answers, learning progress, and emotional state in real time and provides feedback to the server.

[0702] Input: Personalized learning materials and study plans sent from the server

[0703] Output: Data on students' learning progress and emotional state

[0704] Specific behavior:

[0705] Student B selects a personalized math problem set from the device's dashboard and proceeds with the answering process. Meanwhile, the device records Student B's answers and emotional state in real time and sends the data to the server.

[0706] Step 5: Feedback and Action

[0707] User (Teacher)

[0708] Teachers access the server and monitor students' learning progress, analysis results, and emotional state in real time.

[0709] Teachers provide feedback and supplementary materials and advice as needed.

[0710] Input: Student learning progress data and emotion data stored on the server

[0711] Output: Feedback, teaching aids, and advice for students

[0712] Specific behavior:

[0713] Teacher C uses the server's dashboard to check Student C's progress and emotional state. If Student C lacks understanding of a particular concept and is feeling stressed, Teacher C can provide additional quizzes or relaxation advice.

[0714] (Application example 2)

[0715] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0716] In the work environment of factories, there is a demand for improving work efficiency and safety, but conventional methods have limitations. There is no system that collects and analyzes workers' work data and emotional data in real time, and then provides individualized work guidance and safety measures, which leads to problems such as reduced work efficiency and a lack of safety.

[0717] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for collecting work data and emotion data of workers, means for analyzing the collected work data and emotion data using a generative AI model, and means for generating individualized work instructions and safety measures based on the analysis results. This makes it possible to provide optimal work instructions and safety measures in real time based on the work data and emotion data of workers.

[0718] "Work data" refers to data related to the work content, procedures, and efficiency of workers, and is information collected by measuring devices and sensors.

[0719] "Emotional data" refers to data that indicates the emotional state of a worker, detected from their facial expressions, voice, body movements, etc., and is acquired using sensors such as cameras and microphones.

[0720] A "generative AI model" is an algorithm that analyzes collected data and generates optimal guidance and countermeasures for workers, and refers to an artificial intelligence system that uses machine learning and deep learning technologies.

[0721] "Means of analysis" refers to the process of analyzing data using a generative AI model based on collected task data and emotion data, and extracting meaningful information.

[0722] "Individualized work instruction" refers to information and instructions that guide optimal work procedures and methods according to the abilities and conditions of each individual worker.

[0723] "Safety measures" refers to measures, including actions and protective measures, that must be taken to minimize risks during work.

[0724] "Worker" refers to a person who performs work in a factory or work site.

[0725] "Manager" refers to a person whose role is to monitor the work data and progress of workers and provide feedback as needed.

[0726] "Real-time monitoring" refers to instantly collecting and analyzing work data and emotional data to grasp the worker's current state at every moment.

[0727] "Means for providing feedback" refers to a method or system that provides instructions or advice to workers based on analysis results or monitoring data.

[0728] The present invention relates to a work support system for factories. This system collects and analyzes work data and emotional data of workers, and provides individualized work guidance and safety measures.

[0729] server

[0730] The server is the core of the system and performs the following tasks:

[0731] 1. Collecting and storing task and emotion data.

[0732] 2. Analyze collected data using generative AI models.

[0733] 3. Generate and provide individualized work instructions and safety measures based on the analysis results.

[0734] Specifically, the server sets up an API endpoint and prepares to receive data sent from the device. The data is stored in a database and queued for subsequent analysis jobs. The generative AI model and emotion engine are combined to perform the analysis and evaluate the worker's working style and emotional state. Based on the evaluation results, optimal work instructions and safety measures are generated and sent to the corresponding worker's device.

[0735] The specific hardware used includes a database server and a GPU server for computing, and the software includes a database (e.g., MySQL), a deep learning framework (e.g., TensorFlow), and an emotion analysis library (e.g., EmotionEngine).

[0736] Terminal

[0737] The terminal functions as an interface for the worker and is mainly implemented as a smartphone, tablet, or wearable device. It collects work data and emotional data and sends it to the server in real time. Emotional data is collected using the device's camera and microphone, and changes in the worker's facial expressions and voice are detected and sent to the emotion engine.

[0738] The server sends personalized work instructions and safety measures to the device, which are then displayed on a dashboard. The worker receives instructions from the dashboard and puts them into action. The device also records the worker's work progress and emotional state and sends feedback to the server.

[0739] Users (operators and administrators)

[0740] Workers receive personalized guidance and safety measures via their devices. Their movements and emotional state are automatically recorded by the devices, allowing them to concentrate on their work. Supervisors can access the server and monitor the workers' work progress, analysis results, and emotional state in real time. Based on this, supervisors can provide feedback to workers and offer additional guidance and advice.

[0741] For example, when a worker is assembling parts, work data and emotional data are collected via the device's camera. If the server analyzes this data and determines that the worker is under stress, instructions to adjust the work pace or take a break are sent to the device. The worker can follow these instructions and continue working safely and efficiently.

[0742] An example of a prompt is, "If a tired expression is detected, please generate advice on work pace." The generative AI model generates appropriate advice based on this prompt, optimizing the worker's working environment.

[0743] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0744] Step 1: Data collection

[0745] Terminal: Using cameras and sensors installed on the terminal, the work data and emotional data of the worker are collected in real time. During this collection process, changes in the worker's movements, facial expressions, and voice are detected. The input is the real-time data of the worker, and the output is the collected work data and emotional data.

[0746] Step 2: Send data

[0747] Terminal: The collected task data and emotion data are periodically sent to the server. The input is the collected data, and the output is the data sent to the server. Specifically, the data is sent to the server via an API.

[0748] Step 3: Save Data

[0749] Server: Stores the received task data and emotion data in a database. The input is the data sent from the device, and the output is the stored data. Specifically, it creates a database entry and stores the data.

[0750] Step 4: Submit a data analysis job

[0751] Server: Queues jobs to analyze the stored data. The input is the stored data, and the output is the enqueuing of analysis jobs.

[0752] Step 5: Data analysis

[0753] Server: Analyzes the analysis jobs registered in the queue using a generative AI model and an emotion engine. The input is the data entered in the queue, and the output is the analysis results. Specifically, the generative AI model is used to evaluate the worker's working style and emotional state.

[0754] Step 6: Generate personalized task instructions

[0755] Server: Generates individualized work instructions and safety measures based on the analysis results. The input is the analysis results, and the output is the generated work instructions and safety measures. For example, this includes adjusting work pace and instructions for breaks.

[0756] Step 7: Sending work instructions

[0757] Server: Sends the generated work instructions and safety measures to the terminal. The input is the generated instruction content, and the output is the transmission to the terminal. Specifically, the instruction content is sent to the terminal as JSON format data via API.

[0758] Step 8: Work Instructions

[0759] Terminal: The received work instructions and safety measures are displayed on a dashboard and notified to the worker. The input is the data received from the server, and the output is the displayed instruction content. Specifically, the information is displayed using UI components.

[0760] Step 9: Progress Feedback

[0761] Terminal: The worker continues working based on the instructions and records their progress. The recorded progress data and new emotion data are sent to the server again. The input is the worker's progress data and emotion data, and the output is the data sent to the server.

[0762] Step 10: Reassess your progress data

[0763] Server: Analyzes the received progress data and emotion data again and generates new guidance and countermeasures. The input is the collected data again, and the output is the new guidance content. Specifically, the data is analyzed using the generative AI model and emotion engine, just like last time.

[0764] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0765] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0766] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0767] [Third embodiment]

[0768] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0769] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0770] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0771] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0772] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0773] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0774] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0775] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0776] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0777] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0778] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0779] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0780] The present invention relates to a learning support system, and in particular to a system for personalized learning support using a generative AI model. Specific embodiments for carrying out the present invention will be described below.

[0781] This system consists of three components: students, terminals, and a server. The roles and processing flow of each component are described in detail below.

[0782] server

[0783] The server is the core of the system, responsible for collecting and analyzing learning activity data, operating the generative AI model, and generating and distributing personalized learning materials. The server first sets up an API endpoint and receives learning data sent from the device. The received data is stored in a database and queued for subsequent analysis jobs.

[0784] The server then uses a generative AI model to analyze the collected learning data. The analysis evaluates each student's learning style, level of understanding, and progress, and generates personalized learning materials and study plans based on the results. These generated learning materials are then managed by the server and sent to the corresponding student's device.

[0785] As a specific example, if the server determines, based on data acquired during a mathematics class, that Student A is struggling with a particular concept in a practice problem, it will generate and distribute additional practice problems and explanatory videos to reinforce that concept.

[0786] Terminal

[0787] The device is the interface for students to study and is typically implemented as a tablet or PC. The device communicates data between the student and the server via a learning app. When a student logs in, the device records their activity in real time and periodically sends data to the server.

[0788] The personalized learning materials and study plans sent from the server are displayed on the device's dashboard. Students can select the materials that are most suitable for them and proceed with their studies from this dashboard. The device also instantly records the student's answers and progress and sends them back to the server.

[0789] As a concrete example, Student B works on a specific math problem set provided on his / her device, and progresses through his / her studies while looking at the answers and explanations for each problem. At this time, the device constantly records progress information and sends it to the server.

[0790] Users (students and teachers)

[0791] The user is also an important part of the system. Students use their devices to study materials that are optimized for them. Their learning behavior and progress are automatically recorded by the devices, allowing students to focus on their studies independently.

[0792] Teachers can access the server and monitor students' learning progress and analysis results in real time, and based on this, teachers can provide feedback to students and offer additional teaching materials and advice.

[0793] For example, Teacher C can use the server's dashboard to monitor the progress of the entire class and individual students, adjust the lesson plan as needed, and provide additional supplementary materials if a particular student is found to have a lack of understanding of a particular concept.

[0794] This system reduces the workload of teachers while providing students with learning methods that are suited to them, achieving the goal of creating an environment in which all students have equal access to learning opportunities and improving the quality of education.

[0795] The processing flow will be explained below.

[0796] Step 1:

[0797] The server opens an API endpoint for collecting learning data and prepares to receive the learning data. Specifically, the server builds an API to receive student learning activity data (e.g., subjects taken, study time, progress) sent from the device. This data is stored in a database.

[0798] Step 2:

[0799] When a student logs in to the learning app, the device starts recording session information. Specifically, the device records the user's ID and login time at login, collects data each time a learning activity is started, and periodically sends the data to the server.

[0800] Step 3:

[0801] Users (students) log in to the learning app on their tablet or PC and select the designated subject or unit. When students solve problems or watch videos, their actions are automatically recorded on the device. When they finish learning, they log out.

[0802] Step 4:

[0803] The server stores the received training data in a database, and at the same time inputs the data into the generative AI model and executes the analysis job. Specifically, the data stored in the database is periodically put into the analysis queue, which triggers the generative AI model to perform analysis.

[0804] Step 5:

[0805] The generative AI model evaluates each student's learning style and level of understanding based on collected learning data. The analysis results are used to generate optimal learning plans and personalized learning materials. Specifically, the model analyzes students' past performance and learning patterns to identify areas that need reinforcement.

[0806] Step 6:

[0807] The server sends the personalized learning materials and learning plans generated by the generative AI model to the device. Specifically, the generated learning materials and plans are delivered to the device via API and displayed on the device's user interface.

[0808] Step 7:

[0809] The device then displays the received personalized learning materials and learning plans on a dashboard for students to access. Specifically, the device analyzes the learning material data received from the server and displays it in a user-friendly format.

[0810] Step 8:

[0811] Students can study using personalized learning materials provided on the device's dashboard. Specifically, students work through the provided practice problems and instructional videos, and their progress and results are automatically recorded on the device.

[0812] Step 9:

[0813] The device records learning activity and progress data in real time and periodically sends it to the server. Specifically, at the end of each learning session, the device compiles the session data and sends it to the server as a POST request.

[0814] Step 10:

[0815] The server continuously receives new training data and re-evaluates the training plan based on the analysis results. Specifically, the server incorporates the newly received data into the analysis job and retrains the generative AI model accordingly to optimize the training plan.

[0816] Step 11:

[0817] Teachers can check students' progress and analysis results in real time through the server's dashboard. Specifically, the teacher's management screen monitors each student's learning progress and level of understanding, and provides feedback as needed.

[0818] Step 12:

[0819] The user (teacher) can input supplementary materials and advice into the system according to each student's learning progress and provide them to the student. Specifically, the teacher uses the feedback function on the dashboard to input individual advice and supplementary materials, which are then automatically sent to the student's device.

[0820] Example 1

[0821] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0822] Conventional learning support systems have had difficulty providing personalized learning materials tailored to each student's learning style and level of understanding in a timely manner. They also lacked the means to continuously monitor learning progress and provide appropriate feedback in real time. This resulted in issues such as students not being able to maximize their learning effectiveness and increasing the workload of teachers.

[0823] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0824] In this invention, the server includes means for collecting student learning activity data from the terminal, means for storing the collected learning data in a database and queueing it for analysis jobs, means for analyzing the learning data using a generative AI model and evaluating each student's learning style, level of understanding, and progress, means for generating personalized learning materials and learning plans based on the analysis results, means for sending the generated learning materials and learning plans to the student's terminal, means for recording students' answers and progress and providing feedback to the server, and means for continuously monitoring students' learning progress and reevaluating the learning plans based on new data. This enables personalized learning support optimized for each student, promoting effective learning and reducing the workload of teachers.

[0825] "Devices" refer to electronic devices such as tablets and computers that function as interfaces for students to learn.

[0826] "Learning activity data" refers to all data generated when a student engages in learning activities, and specifically includes answers to questions, answer times, progress, etc.

[0827] "Database" refers to an information management system for organizing and storing collected learning data.

[0828] An "analysis job" refers to a series of processes set up to analyze collected learning data.

[0829] A "generative AI model" refers to a model that uses artificial intelligence technology to analyze learning data and generate personalized teaching materials and learning plans.

[0830] "Teaching materials" refers to materials, workbooks, explanatory videos, etc. that students use for studying.

[0831] A "study plan" refers to a plan that includes learning methods and goals, specific learning content, and schedules that are optimized for each student.

[0832] "Feedback" refers to the server or teacher providing evaluations and advice on students' learning activities.

[0833] "Monitoring" refers to the activity of continuously observing students' learning progress and collecting data as necessary.

[0834] "Reevaluation" refers to reviewing an existing learning plan based on a student's latest learning data and revising or updating it as necessary.

[0835] MODE FOR CARRYING OUT THE INVENTION

[0836] This invention relates to a learning support system, specifically a system for personalized learning support using a generative AI model. This system consists of three entities: a student, a terminal, and a server. The roles of each entity and their specific processing are described in detail below.

[0837] server

[0838] The server is the core of the system, responsible for collecting and analyzing learning activity data, operating the generative AI model, and generating and distributing personalized learning materials. The specific operations of the server are as follows:

[0839] The server sets up an API endpoint to receive learning data sent from the device, for example, the results of a student's math problem answers.

[0840] The received data is stored in a database and queued for subsequent analysis jobs.

[0841] The server analyzes the collected training data using a generative AI model, such as a natural language processing model like GPT-3.

[0842] The system uses analytics to assess each student's learning style, level of understanding, and progress, and then generates personalized learning materials and lesson plans based on the results. For example, if a student is struggling with a math problem, the system generates materials to reinforce that concept.

[0843] The generated teaching materials are again managed by the server and sent to the corresponding student's terminal.

[0844] For example, the server inputs the following prompt sentence into the generative AI model for analysis:

[0845] "Student A is struggling with solving a quadratic equation during a math class and has sent the data to the server. The server should analyze this data and generate additional practice problems and instructional videos for Student A."

[0846] Terminal

[0847] The device is the interface through which students learn, and is typically implemented as a tablet or PC. The specific operations of the device are as follows:

[0848] When a student logs in to a device, the device records their activity in real time and periodically sends data to the server, such as recording the questions the student answered and their results.

[0849] The personalized learning materials and learning plans sent from the server are displayed on a dashboard on the device, which is built using front-end frameworks such as React and Angular.

[0850] Students can use this dashboard to select the learning materials that are most suitable for them and proceed with their studies.

[0851] Students' answers and progress are recorded by the device and fed back to the server.

[0852] For example, Student B works on a specific math problem set provided on his / her device, and progresses through his / her studies while looking at the answers and explanations for each problem. At this time, the device records the answers and sends them to the server.

[0853] Users (students and teachers)

[0854] The user is also an important part of the system. Here are some specific user actions:

[0855] Students study materials optimized for them through their devices, and their learning behavior and progress are automatically recorded by the devices, allowing them to concentrate on their studies.

[0856] Teachers can access the server and monitor students' learning progress and analysis results in real time. For example, teachers can access the dashboard from a browser to check students' grades and progress.

[0857] Based on this, the teacher can provide feedback to the student and provide additional support materials or advice. For example, if a particular student shows a lack of understanding of a particular theorem, the teacher can provide that student with additional learning materials.

[0858] For example, Teacher C can use the server's dashboard to monitor the progress of the entire class and individual students, adjust the lesson plan as needed, and provide additional supplementary materials if a particular student is found to have a lack of understanding of a particular concept.

[0859] As described above, by using a generative AI model, the present invention can provide personalized educational support tailored to each student's learning style and progress, thereby reducing the workload of teachers and providing a more effective learning environment for students.

[0860] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0861] Step 1: Device collects student learning activity data

[0862] How it works: Students log in to their devices and begin working on math problems. The devices then record data in real time, such as the answers and time it took to complete each problem, via a learning app.

[0863] Input: Answer data, answer time, and progress of students logged in to the device

[0864] Output: Learning activity data recorded by the device

[0865] Step 2: The device sends the learning activity data to the server.

[0866] Specific operation: The device sends the student's learning data to the server at regular intervals or when the learning activity is completed, for example, after each question is answered.

[0867] Input: Learning activity data recorded on the device

[0868] Output: Learning activity data sent to the server

[0869] Step 3: The server saves the training data to a database and queues it for analysis jobs.

[0870] Specific operation: After receiving the training data, the server stores it in a database. The stored data is then queued for analysis jobs. This process is automated, enabling efficient data management.

[0871] Input: Learning activity data sent to the server

[0872] Output: Learning activity data stored in the database, data input to the analysis job

[0873] Step 4: The server analyzes the training data using the generated AI model

[0874] How it works: The server takes the data from the queue and inputs it into the generative AI model, which analyzes each student's learning style and level of understanding, and then generates optimal teaching materials and lesson plans based on the results.

[0875] Input: Training data submitted to the analysis job

[0876] Output: Analysis of each student's learning style, level of understanding, and progress

[0877] Step 5: The server generates personalized learning materials and learning plans based on the analysis results.

[0878] What it does: Based on the analysis, it uses a generative AI model to generate personalized learning materials and lesson plans. For example, if a particular student is struggling with a particular math concept, it generates practice exercises and instructional videos that reinforce that concept.

[0879] Input: Analysis of each student's learning style, level of understanding, and progress

[0880] Output: personalized learning materials and learning plans

[0881] Step 6: The server sends the generated learning materials and lesson plans to the students' devices.

[0882] Specific operation: The server sends the generated personalized teaching materials and learning plans to the corresponding student's device.

[0883] Input: Generated personalized learning materials and learning plans

[0884] Output: Materials and lesson plans sent to the corresponding student devices

[0885] Step 7: The device displays the sent learning material and the student continues learning based on it.

[0886] Specific operation: The device displays the learning materials and study plans received from the server on a dashboard. Students can select the appropriate learning materials from this dashboard and proceed with their studies. For example, they can work on the displayed problem sets and progress through the answers.

[0887] Input: Personalized learning materials and study plans sent from the server

[0888] Output: Materials and lesson plans displayed on the dashboard, and student learning activities

[0889] Step 8: The device records the student's answers and progress and sends them back to the server.

[0890] Specific operation: The device again records the student's answers and progress and sends them back to the server. This data is used for analysis in the next cycle.

[0891] Input: Student answers and progress

[0892] Output: Answer results and progress feedback to the server

[0893] Step 9: The server continuously monitors your progress and reevaluates your learning plan based on new data.

[0894] Specific operation: The server continuously monitors learning progress based on the feedback data, analyzes new data, and reevaluates and modifies the learning plan.

[0895] Input: Feedback of answers and progress data

[0896] Output: Updated learning plan

[0897] This series of steps enables effective learning support that is customized for each student.

[0898] (Application example 1)

[0899] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0900] In modern factories, there is a demand for efficient training and improved skills for engineers and workers, but conventional training methods make it difficult to provide appropriate feedback to individual engineers, limiting productivity improvements. Another issue is the lack of technology to efficiently collect and analyze worker activity data and generate training plans tailored to each individual's skill level.

[0901] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0902] In this invention, the server includes means for collecting work activity data of workers, means for analyzing the collected data using a generative AI model, means for generating personalized training plans and learning materials based on the analysis results, means for providing the generated training plans and learning materials to workers, and means for continuously monitoring the progress of workers and reevaluating the training plans based on new data, thereby enabling efficient and personalized training to be provided to engineers and workers.

[0903] "Worker" refers to a person who performs a specific task in a factory or production site.

[0904] "Work activity data" refers to information about the process and results of work performed by workers, and is data collected through devices such as sensors and tablets.

[0905] A "generative AI model" refers to an artificial intelligence model that analyzes collected data and generates personalized training plans, learning materials, etc.

[0906] A "training plan" is a plan aimed at improving a worker's techniques and skills, and includes learning content optimized for each individual worker.

[0907] "Learning materials" refers to the materials and content that workers use to learn according to the training plan, including videos, workbooks, tutorials, etc.

[0908] "Means of provision" refers to the method of providing the generated training plans and learning materials to workers at an appropriate time.

[0909] "Progress monitoring" refers to the continuous observation and recording of workers' progress as they progress through training and their work.

[0910] "Reassessment" refers to reviewing existing training plans and learning methods based on new data collected and making improvements or changes as needed.

[0911] "Means for analyzing data" refers to methods and techniques for processing collected data and assessing worker skills and understanding.

[0912] This invention relates to a system that supports the skill improvement and efficient training of engineers and workers in factories. This system collects data on workers' work activities, analyzes the data using a generative AI model, and generates and provides personalized training plans and learning materials based on the analysis results.

[0913] Hardware and software used

[0914] Hardware

[0915] Training tablet or computer (device)

[0916] Data collection sensors

[0917] Learning information storage server

[0918] GPU server for high-speed calculations (operation of generative AI models)

[0919] software

[0920] Learning Management System (LMS)

[0921] Python-based analysis program

[0922] Generative AI model library (PyTorch or TensorFlow)

[0923] Database management system (PostgreSQL)

[0924] RESTful API (Flask or Django)

[0925] Explanation of the processing flow

[0926] 1. Data Collection

[0927] The server uses sensors and devices to collect work activity data when workers run the training program on their devices. This data includes information about the work process and results. For example, sensors record the time and steps for each process in the assembly of parts.

[0928] 2. Data Analysis

[0929] The server stores the collected work activity data in a database and analyzes it using a generative AI model. The analysis evaluates the worker's skill level and tendency for operational errors. The generative AI model is implemented using a Python-based analysis program and libraries such as PyTorch and TensorFlow.

[0930] 3. Training plan generation

[0931] Based on the analysis results, the server generates personalized training plans and learning materials. These training plans include video tutorials and problem sets that include specific work content and areas for improvement. For example, for the assembly procedure of a new part, the server generates materials that include countermeasures for common mistakes made by workers.

[0932] 4. Submissions and Feedback

[0933] The generated training plan and learning materials are sent to the terminal. As the worker progresses through the training, new data is collected and sent back to the server. This allows the server to reevaluate the training plan based on the latest data and make adjustments as necessary. This feedback loop ensures that the most optimal training is always provided.

[0934] Specific examples

[0935] When training workers on assembly procedures for a new part, they watch a video tutorial on a tablet and then perform a training simulation. Sensors and the tablet collect data during the simulation and send it to a server. A generative AI model analyzes the data and generates additional teaching materials for areas where mistakes are common, which are then sent back to the tablet. This process allows workers to effectively improve their skills.

[0936] Prompt Sentence Examples

[0937] "Please suggest the best training content for the next lesson. Operator A made many mistakes when assembling part B. Please generate teaching materials that include specific problem areas and how to improve them."

[0938] This invention makes it possible to provide efficient and individualized training to engineers and workers, which is expected to improve productivity.

[0939] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0940] Step 1:

[0941] The server collects work activity data from sensors and devices when a worker runs a training program on the device. When the worker presses the start button, the device starts collecting sensor data and sends data about the work process and results to the server. The input is data from the sensors and devices, and the output is raw data stored on the server.

[0942] Step 2:

[0943] The server stores the collected data in a database and performs data cleansing, removing incomplete or noisy data and shaping the data. The input is the raw collected data, and the output is cleaned data that can be analyzed.

[0944] Step 3:

[0945] The server analyzes the cleaned data using a generative AI model. A Python-based analysis program is used to evaluate the worker's skill level and tendency for operational errors. The input is the cleaned data, and the output is a technical evaluation report.

[0946] Step 4:

[0947] The server generates personalized training plans and learning materials based on the analysis results. It uses a generative AI model library (PyTorch or TensorFlow) to generate optimal learning materials for each worker. The input is the technical evaluation report, and the output is personalized training plans and learning materials.

[0948] Step 5:

[0949] The server distributes the generated training plan and learning materials to the terminal. The terminal receives them and provides them to the worker. The worker checks the training plan and learning materials on the terminal and progresses with their learning. The input is the individualized training plan and learning materials, and the output is the worker's learning progress.

[0950] Step 6:

[0951] The terminal collects the worker's training progress data and sends it to the server, including the training completion status and any new mistakes. The input is the worker's progress data, and the output is the data sent to the server.

[0952] Step 7:

[0953] The server re-evaluates the training plan based on the new data received, again using the generative AI model to adjust the training content as needed. The input is the new progress data, and the output is the adjusted training plan.

[0954] Step 8:

[0955] The server then delivers the re-evaluated training plan and learning materials back to the terminal and provides them to the worker. This feedback loop ensures continuous optimal training. The input is the adjusted training plan and learning materials, and the output is the worker's latest learning progress.

[0956] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0957] This invention relates to a learning support system that combines a generative AI model and an emotion engine, and provides means for collecting learning activity data, generating personalized learning materials, managing learning progress, and monitoring the user's emotional state. The system of this invention consists of three main entities: a server, a terminal, and a user. The specific roles and processing flow of each entity are explained below.

[0958] server

[0959] The server is the core of the system, responsible for collecting and analyzing learning activity data and emotion data, operating the generative AI model, and generating and providing personalized learning materials and learning plans. First, the server sets up an API endpoint and prepares to receive data sent from the device. The received data is stored in a database and queued for subsequent analysis jobs.

[0960] The server then combines the generative AI model with an emotion engine to perform analysis. It uses the learning data and emotion data to evaluate the student's learning style, comprehension, and emotional state. Based on the evaluation results, it generates an optimal learning plan and personalized learning materials. The generated learning materials are then sent to the corresponding student's device.

[0961] As a specific example, if the server determines that Student A is feeling anxious about a problem based on data and emotional data acquired during a mathematics class, it will generate and deliver additional practice problems that take into account the student's emotional state and explanatory videos for relaxation.

[0962] Terminal

[0963] The device functions as an interface for users (students) to study and is implemented on a tablet or PC. Data is transmitted between the student and the server via a learning app. When a student logs in, the device records their activity in real time and periodically sends the data to the server. Emotional data is also collected in the same way; for example, changes in the student's facial expression or voice are detected using a camera or microphone, and sent to the emotion engine.

[0964] The personalized learning materials and study plans sent from the server are displayed on a dashboard on the device. Students can select the materials that are most suitable for them and proceed with their studies from this dashboard. The device also instantly records the students' answers, progress, and emotional state and provides feedback to the server.

[0965] For example, if Student B is working on a specific math problem set provided on his / her device and loses concentration while solving it, his / her emotional state will be detected by the device, which will record it and send it to the server.

[0966] Users (students and teachers)

[0967] The user is also an important part of the system. Students use their devices to study materials that are optimized for them. Their behavior, progress, and emotional state are automatically recorded by the devices, allowing them to focus on their studies independently.

[0968] Teachers can access the server to monitor students' learning progress, analysis results, and emotional state in real time, and based on this, teachers can provide feedback to students and provide additional teaching materials and advice.

[0969] For example, Teacher C can use the server's dashboard to monitor the progress and emotional state of the entire class and individual students, adjust the lesson plan as needed, and provide additional supplementary materials or relaxation advice if a particular student is found to be struggling with a particular concept and feeling stressed.

[0970] In this way, learning support systems can reduce the workload of teachers while providing students with learning methods that are suited to them. The goal is to create an environment in which all students can enjoy equal learning opportunities and improve the quality of education.

[0971] The processing flow will be explained below.

[0972] Step 1:

[0973] The server opens an API endpoint to collect learning data and emotional data. First, it prepares to receive learning requests, and then receives student learning activity data (e.g., subjects taken, study time, progress) and emotional data sent from the device. This data is then stored in a database.

[0974] Step 2:

[0975] The device starts recording session information when a student logs in to the learning app. Specifically, it records the user ID and login time when logging in to study, and collects data each time a learning activity begins. It also uses the camera and microphone to simultaneously collect emotional data such as facial expressions and voice.

[0976] Step 3:

[0977] Users (students) log in to the learning app and select the designated subject or unit. Every time they solve a problem or watch a video, their actions are automatically recorded on the device. Emotional data obtained using the camera and microphone is also recorded. When they finish learning, they log out.

[0978] Step 4:

[0979] The server stores the received learning data and emotion data in a database, and at the same time inputs the data into the generative AI model and emotion engine to execute the analysis job. Specifically, the data stored in the database is periodically put into the analysis queue, which triggers the generative AI model and emotion engine to perform analysis.

[0980] Step 5:

[0981] The generative AI model and emotion engine evaluate each student's learning style, level of understanding, and emotional state based on collected learning and emotion data. The analysis results are used to generate optimal learning plans and personalized learning materials. Specifically, the generative AI model identifies areas that need reinforcement based on a student's past performance, learning patterns, and emotions.

[0982] Step 6:

[0983] The server sends the personalized learning materials and learning plans generated by the generative AI model and emotion engine to the device. Specifically, the generated learning materials and plans are delivered to the device via API and displayed on the device's user interface.

[0984] Step 7:

[0985] The device then displays the received personalized learning materials and learning plans on a dashboard for students to access. Specifically, the device analyzes the learning material data received from the server and displays it in a user-friendly format.

[0986] Step 8:

[0987] The user (student) progresses through their studies using personalized learning materials provided on the device's dashboard. Specifically, students work through the provided practice problems and instructional videos, and their progress and results are automatically recorded on the device. Emotional data is also continuously recorded and sent to the server in real time.

[0988] Step 9:

[0989] The device records activity, progress, and emotional data during learning in real time and periodically transmits them to the server. Specifically, at the end of each learning session, the device compiles the session data and sends it to the server as a POST request.

[0990] Step 10:

[0991] The server continuously receives new learning and emotion data and re-evaluates the learning plan based on the analysis results. Specifically, it incorporates the newly received data into the analysis job and retrains the generative AI model and emotion engine accordingly to optimize the learning plan.

[0992] Step 11:

[0993] Teachers can check students' progress, analysis results, and emotional state in real time through the server's dashboard. Specifically, the teacher's management screen monitors each student's learning progress, level of understanding, and emotional state, and provides feedback as needed.

[0994] Step 12:

[0995] The user (teacher) can input supplementary materials and advice into the system based on each student's learning progress and emotional state, and provide them to the student. Specifically, the teacher uses the feedback function on the dashboard to input individual advice and supplementary materials, which are then automatically sent to the student's device.

[0996] Example 2

[0997] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0998] Conventional learning support systems often only collect and analyze students' learning activity data and do not consider their emotional state, making it difficult to improve the quality of education. Furthermore, they often fail to generate personalized learning materials and learning plans, resulting in ineffective instruction. Furthermore, teachers have limited means to monitor students' learning progress in real time and provide appropriate feedback. Therefore, there is a need for systems that can provide a more effective learning environment and realize education that meets the needs of each individual student.

[0999] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes means for collecting learning activity data and emotional data of students, means for analyzing the collected learning data and emotional data using a generative AI model and an emotion engine, and means for generating individualized learning materials and learning plans based on the analysis results. This makes it possible to evaluate students' learning styles, levels of comprehension, and emotional states in real time and provide optimal learning plans and learning materials.

[1000] "Student" refers to a learner who uses the learning support system.

[1001] "Learning activity data" refers to information such as operation logs, answer data, and study time generated by students while they are studying.

[1002] "Emotional data" refers to information about a student's emotional state obtained from facial expressions, tone of voice, etc.

[1003] A "generative AI model" refers to an artificial intelligence algorithm that automatically generates learning plans and teaching materials based on data.

[1004] An "emotion engine" refers to software that analyzes students' emotional state and provides the results to a learning support system.

[1005] "Analysis method" refers to the process of analyzing collected data using generative AI models and emotion engines.

[1006] "Personalized learning materials" refers to learning materials that are optimized based on a student's learning style and level of understanding.

[1007] A "learning plan" refers to a set of learning tasks or schedule designed to efficiently carry out students' learning activities.

[1008] "Monitoring" refers to the process of continuously observing students' academic progress and emotional state.

[1009] "Feedback" refers to the guidance and advice that teachers provide to students based on their learning progress and emotional state.

[1010] The present invention relates to a learning support system that combines a generative AI model and an emotion engine, and is specifically implemented as follows.

[1011] server

[1012] The server is the core of the learning support system and performs the following functions:

[1013] 1. Data Collection and Storage

[1014] The server sets up an API endpoint to receive learning activity data and emotion data sent from the device, which is then stored in a database and queued for analysis.

[1015] 2. Data Analysis

[1016] The server combines a generative AI model and an emotion engine to perform analysis. Specifically, it evaluates students' learning style, comprehension, and emotional state based on collected learning activity data and emotion data. These evaluation results are used to generate optimal learning plans and personalized learning materials.

[1017] 3. Creation and distribution of teaching materials

[1018] Based on the evaluation results, the server uses a generative AI model to generate personalized learning materials and study plans. The generated learning materials are sent to the corresponding student's device. For example, if the server determines that Student A is feeling anxious about a problem based on math lesson data and emotional data, it will generate an explanatory video for relaxation and additional practice problems and deliver them to the device.

[1019] Terminal

[1020] The device functions as a learning interface for users (students) and is implemented on a tablet or PC. It performs the following functions:

[1021] 1. Data Collection

[1022] When a student logs in to their device, the learning app starts and records learning activity data and emotional data in real time. The app uses a camera and microphone to collect emotional data from facial expressions and tone of voice, and sends it to a server.

[1023] 2. Viewing learning content

[1024] The personalized learning materials and study plans sent from the server are displayed on a dashboard on the device, where students can select the materials that are most suitable for them and proceed with their studies.

[1025] 3. Feedback

[1026] The students' answers, learning progress, and emotional state are recorded in real time and fed back to the server. For example, if Student B loses concentration while working on a math problem, the device detects this emotional state and immediately sends the data to the server.

[1027] Users (students and teachers)

[1028] The user is also an important part of the system and performs the following functions:

[1029] 1. Students

[1030] Students use their devices to study materials that are optimized for them, and their behavior, progress, and emotional state are automatically recorded by the devices, allowing them to focus on their studies independently.

[1031] 2. Teacher

[1032] Teachers access the server and monitor students' learning progress, analysis results, and emotional states in real time. Based on this, they provide feedback and additional supplementary materials or advice as needed. For example, Teacher C uses the server's dashboard to check the progress and emotional state of a specific student and adjust the teaching plan as needed.

[1033] Prompt Sentence Examples

[1034] Below are some examples of specific prompt sentences.

[1035] 1. Prompts to analyze student progress

[1036] Analyze the following dataset to assess student progress and understanding: {data_set}

[1037] 2. Prompt sentences for generating teaching materials based on emotion data

[1038] Analyze the following emotion data and generate personalized math exercises adapted to the emotion: {emotion_data}

[1039] In this way, this system utilizes generative AI models and emotion engines to provide optimal learning methods for students, reduce the workload of teachers, and improve the quality of education by creating an environment in which all students have equal access to learning opportunities.

[1040] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1041] Step 1: Data collection and transmission

[1042] Terminal

[1043] The device launches the learning app when the user (student) logs in.

[1044] The device records students' learning activity data (operation logs, answer data, study time) in real time through the learning app.

[1045] The device uses a camera and microphone to collect emotional data such as students' facial expressions and tone of voice.

[1046] The terminal periodically transmits this data to the server.

[1047] Input: Learning activity data and emotion data of logged-in students

[1048] Output: Learning activity data and emotion data sent to the server

[1049] Specific behavior:

[1050] While Student A logs in to the device and solves math problems, the device records the operation log and progress of the answer in real time. The camera also captures Student A's facial expressions and generates data to be sent to the emotion engine.

[1051] Step 2: Data storage and analysis

[1052] server

[1053] The server receives the learning activity data and emotion data sent from the terminal at an API endpoint.

[1054] The server stores the received data in a database.

[1055] The server puts the stored data into an analysis queue and performs the analysis using generative AI models and emotion engines.

[1056] The server evaluates the student's learning style, level of understanding, and emotional state based on learning data and emotional data.

[1057] Input: Learning activity data and emotion data sent from the device

[1058] Output: Assessment results of student learning style, comprehension, and emotional state

[1059] Specific behavior:

[1060] The server receives Student A's learning log and emotional data and stores them in a database. Next, it inputs a prompt statement to the generative AI model: "Analyze Student A's learning activity data and emotional data, and evaluate his learning style and level of comprehension.", and obtains the evaluation results.

[1061] Step 3: Generate and deliver personalized learning materials

[1062] server

[1063] The server generates optimal learning plans and personalized learning materials based on the analysis results.

[1064] The server generates teaching materials using the generative AI model and sends the prepared teaching materials to the corresponding student's device.

[1065] Input: Assessment results of student learning style, comprehension, and emotional state

[1066] Output: personalized learning materials and learning plans

[1067] Specific behavior:

[1068] Based on the evaluation results, the server inputs a prompt to the generative AI model, such as "Generate an instructional video for relaxation," and generates a personalized instructional video and additional practice questions, which are then sent to the device.

[1069] Step 4: View your learning and receive progress feedback

[1070] Terminal

[1071] The device displays the learning materials and study plans sent from the server on a dashboard.

[1072] The device records the student's answers, learning progress, and emotional state in real time and provides feedback to the server.

[1073] Input: Personalized learning materials and study plans sent from the server

[1074] Output: Data on students' learning progress and emotional state

[1075] Specific behavior:

[1076] Student B selects a personalized math problem set from the device's dashboard and proceeds with the answering process. Meanwhile, the device records Student B's answers and emotional state in real time and sends the data to the server.

[1077] Step 5: Feedback and Action

[1078] User (Teacher)

[1079] Teachers access the server and monitor students' learning progress, analysis results, and emotional state in real time.

[1080] Teachers provide feedback and supplementary materials and advice as needed.

[1081] Input: Student learning progress data and emotion data stored on the server

[1082] Output: Feedback, teaching aids, and advice for students

[1083] Specific behavior:

[1084] Teacher C uses the server's dashboard to check Student C's progress and emotional state. If Student C lacks understanding of a particular concept and is feeling stressed, Teacher C can provide additional quizzes or relaxation advice.

[1085] (Application example 2)

[1086] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1087] In the work environment of factories, there is a demand for improving work efficiency and safety, but conventional methods have limitations. There is no system that collects and analyzes workers' work data and emotional data in real time, and then provides individualized work guidance and safety measures, which leads to problems such as reduced work efficiency and a lack of safety.

[1088] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for collecting work data and emotion data of workers, means for analyzing the collected work data and emotion data using a generative AI model, and means for generating individualized work instructions and safety measures based on the analysis results. This makes it possible to provide optimal work instructions and safety measures in real time based on the work data and emotion data of workers.

[1089] "Work data" refers to data related to the work content, procedures, and efficiency of workers, and is information collected by measuring devices and sensors.

[1090] "Emotional data" refers to data that indicates the emotional state of a worker, detected from their facial expressions, voice, body movements, etc., and is acquired using sensors such as cameras and microphones.

[1091] A "generative AI model" is an algorithm that analyzes collected data and generates optimal guidance and countermeasures for workers, and refers to an artificial intelligence system that uses machine learning and deep learning technologies.

[1092] "Means of analysis" refers to the process of analyzing data using a generative AI model based on collected task data and emotion data, and extracting meaningful information.

[1093] "Individualized work instruction" refers to information and instructions that guide optimal work procedures and methods according to the abilities and conditions of each individual worker.

[1094] "Safety measures" refers to measures, including actions and protective measures, that must be taken to minimize risks during work.

[1095] "Worker" refers to a person who performs work in a factory or work site.

[1096] "Manager" refers to a person whose role is to monitor the work data and progress of workers and provide feedback as needed.

[1097] "Real-time monitoring" refers to instantly collecting and analyzing work data and emotional data to grasp the worker's current state at every moment.

[1098] "Means for providing feedback" refers to a method or system that provides instructions or advice to workers based on analysis results or monitoring data.

[1099] The present invention relates to a work support system for factories. This system collects and analyzes work data and emotional data of workers, and provides individualized work guidance and safety measures.

[1100] server

[1101] The server is the core of the system and performs the following tasks:

[1102] 1. Collecting and storing task and emotion data.

[1103] 2. Analyze collected data using generative AI models.

[1104] 3. Generate and provide individualized work instructions and safety measures based on the analysis results.

[1105] Specifically, the server sets up an API endpoint and prepares to receive data sent from the device. The data is stored in a database and queued for subsequent analysis jobs. The generative AI model and emotion engine are combined to perform the analysis and evaluate the worker's working style and emotional state. Based on the evaluation results, optimal work instructions and safety measures are generated and sent to the corresponding worker's device.

[1106] The specific hardware used includes a database server and a GPU server for computing, and the software includes a database (e.g., MySQL), a deep learning framework (e.g., TensorFlow), and an emotion analysis library (e.g., EmotionEngine).

[1107] Terminal

[1108] The terminal functions as an interface for the worker and is mainly implemented as a smartphone, tablet, or wearable device. It collects work data and emotional data and sends it to the server in real time. Emotional data is collected using the device's camera and microphone, and changes in the worker's facial expressions and voice are detected and sent to the emotion engine.

[1109] The server sends personalized work instructions and safety measures to the device, which are then displayed on a dashboard. The worker receives instructions from the dashboard and puts them into action. The device also records the worker's work progress and emotional state and sends feedback to the server.

[1110] Users (operators and administrators)

[1111] Workers receive personalized guidance and safety measures via their devices. Their movements and emotional state are automatically recorded by the devices, allowing them to concentrate on their work. Supervisors can access the server and monitor the workers' work progress, analysis results, and emotional state in real time. Based on this, supervisors can provide feedback to workers and offer additional guidance and advice.

[1112] For example, when a worker is assembling parts, work data and emotional data are collected via the device's camera. If the server analyzes this data and determines that the worker is under stress, instructions to adjust the work pace or take a break are sent to the device. The worker can follow these instructions and continue working safely and efficiently.

[1113] An example of a prompt is, "If a tired expression is detected, please generate advice on work pace." The generative AI model generates appropriate advice based on this prompt, optimizing the worker's working environment.

[1114] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1115] Step 1: Data collection

[1116] Terminal: Using cameras and sensors installed on the terminal, the work data and emotional data of the worker are collected in real time. During this collection process, changes in the worker's movements, facial expressions, and voice are detected. The input is the real-time data of the worker, and the output is the collected work data and emotional data.

[1117] Step 2: Send data

[1118] Terminal: The collected task data and emotion data are periodically sent to the server. The input is the collected data, and the output is the data sent to the server. Specifically, the data is sent to the server via an API.

[1119] Step 3: Save Data

[1120] Server: Stores the received task data and emotion data in a database. The input is the data sent from the device, and the output is the stored data. Specifically, it creates a database entry and stores the data.

[1121] Step 4: Submit a data analysis job

[1122] Server: Queues jobs to analyze the stored data. The input is the stored data, and the output is the enqueuing of analysis jobs.

[1123] Step 5: Data analysis

[1124] Server: Analyzes the analysis jobs registered in the queue using a generative AI model and an emotion engine. The input is the data entered in the queue, and the output is the analysis results. Specifically, the generative AI model is used to evaluate the worker's working style and emotional state.

[1125] Step 6: Generate personalized task instructions

[1126] Server: Generates individualized work instructions and safety measures based on the analysis results. The input is the analysis results, and the output is the generated work instructions and safety measures. For example, this includes adjusting work pace and instructions for breaks.

[1127] Step 7: Sending work instructions

[1128] Server: Sends the generated work instructions and safety measures to the terminal. The input is the generated instruction content, and the output is the transmission to the terminal. Specifically, the instruction content is sent to the terminal as JSON format data via API.

[1129] Step 8: Work Instructions

[1130] Terminal: The received work instructions and safety measures are displayed on a dashboard and notified to the worker. The input is the data received from the server, and the output is the displayed instruction content. Specifically, the information is displayed using UI components.

[1131] Step 9: Progress Feedback

[1132] Terminal: The worker continues working based on the instructions and records their progress. The recorded progress data and new emotion data are sent to the server again. The input is the worker's progress data and emotion data, and the output is the data sent to the server.

[1133] Step 10: Reassess your progress data

[1134] Server: Analyzes the received progress data and emotion data again and generates new guidance and countermeasures. The input is the collected data again, and the output is the new guidance content. Specifically, the data is analyzed using the generative AI model and emotion engine, just like last time.

[1135] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1136] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1137] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1138] [Fourth embodiment]

[1139] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1140] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1141] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1142] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1143] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1144] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1145] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1146] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1147] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1148] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1149] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1150] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1151] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1152] The present invention relates to a learning support system, and in particular to a system for personalized learning support using a generative AI model. Specific embodiments for carrying out the present invention will be described below.

[1153] This system consists of three components: students, terminals, and a server. The roles and processing flow of each component are described in detail below.

[1154] server

[1155] The server is the core of the system, responsible for collecting and analyzing learning activity data, operating the generative AI model, and generating and distributing personalized learning materials. The server first sets up an API endpoint and receives learning data sent from the device. The received data is stored in a database and queued for subsequent analysis jobs.

[1156] The server then uses a generative AI model to analyze the collected learning data. The analysis evaluates each student's learning style, level of understanding, and progress, and generates personalized learning materials and study plans based on the results. These generated learning materials are then managed by the server and sent to the corresponding student's device.

[1157] As a specific example, if the server determines, based on data acquired during a mathematics class, that Student A is struggling with a particular concept in a practice problem, it will generate and distribute additional practice problems and explanatory videos to reinforce that concept.

[1158] Terminal

[1159] The device is the interface for students to study and is typically implemented as a tablet or PC. The device communicates data between the student and the server via a learning app. When a student logs in, the device records their activity in real time and periodically sends data to the server.

[1160] The personalized learning materials and study plans sent from the server are displayed on the device's dashboard. Students can select the materials that are most suitable for them and proceed with their studies from this dashboard. The device also instantly records the student's answers and progress and sends them back to the server.

[1161] As a concrete example, Student B works on a specific math problem set provided on his / her device, and progresses through his / her studies while looking at the answers and explanations for each problem. At this time, the device constantly records progress information and sends it to the server.

[1162] Users (students and teachers)

[1163] The user is also an important part of the system. Students use their devices to study materials that are optimized for them. Their learning behavior and progress are automatically recorded by the devices, allowing students to focus on their studies independently.

[1164] Teachers can access the server and monitor students' learning progress and analysis results in real time, and based on this, teachers can provide feedback to students and offer additional teaching materials and advice.

[1165] For example, Teacher C can use the server's dashboard to monitor the progress of the entire class and individual students, adjust the lesson plan as needed, and provide additional supplementary materials if a particular student is found to have a lack of understanding of a particular concept.

[1166] This system reduces the workload of teachers while providing students with learning methods that are suited to them, achieving the goal of creating an environment in which all students have equal access to learning opportunities and improving the quality of education.

[1167] The processing flow will be explained below.

[1168] Step 1:

[1169] The server opens an API endpoint for collecting learning data and prepares to receive the learning data. Specifically, the server builds an API to receive student learning activity data (e.g., subjects taken, study time, progress) sent from the device. This data is stored in a database.

[1170] Step 2:

[1171] When a student logs in to the learning app, the device starts recording session information. Specifically, the device records the user's ID and login time at login, collects data each time a learning activity is started, and periodically sends the data to the server.

[1172] Step 3:

[1173] Users (students) log in to the learning app on their tablet or PC and select the designated subject or unit. When students solve problems or watch videos, their actions are automatically recorded on the device. When they finish learning, they log out.

[1174] Step 4:

[1175] The server stores the received training data in a database, and at the same time inputs the data into the generative AI model and executes the analysis job. Specifically, the data stored in the database is periodically put into the analysis queue, which triggers the generative AI model to perform analysis.

[1176] Step 5:

[1177] The generative AI model evaluates each student's learning style and level of understanding based on collected learning data. The analysis results are used to generate optimal learning plans and personalized learning materials. Specifically, the model analyzes students' past performance and learning patterns to identify areas that need reinforcement.

[1178] Step 6:

[1179] The server sends the personalized learning materials and learning plans generated by the generative AI model to the device. Specifically, the generated learning materials and plans are delivered to the device via API and displayed on the device's user interface.

[1180] Step 7:

[1181] The device then displays the received personalized learning materials and learning plans on a dashboard for students to access. Specifically, the device analyzes the learning material data received from the server and displays it in a user-friendly format.

[1182] Step 8:

[1183] Students can study using personalized learning materials provided on the device's dashboard. Specifically, students work through the provided practice problems and instructional videos, and their progress and results are automatically recorded on the device.

[1184] Step 9:

[1185] The device records learning activity and progress data in real time and periodically sends it to the server. Specifically, at the end of each learning session, the device compiles the session data and sends it to the server as a POST request.

[1186] Step 10:

[1187] The server continuously receives new training data and re-evaluates the training plan based on the analysis results. Specifically, the server incorporates the newly received data into the analysis job and retrains the generative AI model accordingly to optimize the training plan.

[1188] Step 11:

[1189] Teachers can check students' progress and analysis results in real time through the server's dashboard. Specifically, the teacher's management screen monitors each student's learning progress and level of understanding, and provides feedback as needed.

[1190] Step 12:

[1191] The user (teacher) can input supplementary materials and advice into the system according to each student's learning progress and provide them to the student. Specifically, the teacher uses the feedback function on the dashboard to input individual advice and supplementary materials, which are then automatically sent to the student's device.

[1192] Example 1

[1193] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1194] Conventional learning support systems have had difficulty providing personalized learning materials tailored to each student's learning style and level of understanding in a timely manner. They also lacked the means to continuously monitor learning progress and provide appropriate feedback in real time. This resulted in issues such as students not being able to maximize their learning effectiveness and increasing the workload of teachers.

[1195] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1196] In this invention, the server includes means for collecting student learning activity data from the terminal, means for storing the collected learning data in a database and queueing it for analysis jobs, means for analyzing the learning data using a generative AI model and evaluating each student's learning style, level of understanding, and progress, means for generating personalized learning materials and learning plans based on the analysis results, means for sending the generated learning materials and learning plans to the student's terminal, means for recording students' answers and progress and providing feedback to the server, and means for continuously monitoring students' learning progress and reevaluating the learning plans based on new data. This enables personalized learning support optimized for each student, promoting effective learning and reducing the workload of teachers.

[1197] "Devices" refer to electronic devices such as tablets and computers that function as interfaces for students to learn.

[1198] "Learning activity data" refers to all data generated when a student engages in learning activities, and specifically includes answers to questions, answer times, progress, etc.

[1199] "Database" refers to an information management system for organizing and storing collected learning data.

[1200] An "analysis job" refers to a series of processes set up to analyze collected learning data.

[1201] A "generative AI model" refers to a model that uses artificial intelligence technology to analyze learning data and generate personalized teaching materials and learning plans.

[1202] "Teaching materials" refers to materials, workbooks, explanatory videos, etc. that students use for studying.

[1203] A "study plan" refers to a plan that includes learning methods and goals, specific learning content, and schedules that are optimized for each student.

[1204] "Feedback" refers to the server or teacher providing evaluations and advice on students' learning activities.

[1205] "Monitoring" refers to the activity of continuously observing students' learning progress and collecting data as necessary.

[1206] "Reevaluation" refers to reviewing an existing learning plan based on a student's latest learning data and revising or updating it as necessary.

[1207] MODE FOR CARRYING OUT THE INVENTION

[1208] This invention relates to a learning support system, specifically a system for personalized learning support using a generative AI model. This system consists of three entities: a student, a terminal, and a server. The roles of each entity and their specific processing are described in detail below.

[1209] server

[1210] The server is the core of the system, responsible for collecting and analyzing learning activity data, operating the generative AI model, and generating and distributing personalized learning materials. The specific operations of the server are as follows:

[1211] The server sets up an API endpoint to receive learning data sent from the device, for example, the results of a student's math problem answers.

[1212] The received data is stored in a database and queued for subsequent analysis jobs.

[1213] The server analyzes the collected training data using a generative AI model, such as a natural language processing model like GPT-3.

[1214] The system uses analytics to assess each student's learning style, level of understanding, and progress, and then generates personalized learning materials and lesson plans based on the results. For example, if a student is struggling with a math problem, the system generates materials to reinforce that concept.

[1215] The generated teaching materials are again managed by the server and sent to the corresponding student's terminal.

[1216] For example, the server inputs the following prompt sentence into the generative AI model for analysis:

[1217] "Student A is struggling with solving a quadratic equation during a math class and has sent the data to the server. The server should analyze this data and generate additional practice problems and instructional videos for Student A."

[1218] Terminal

[1219] The device is the interface through which students learn, and is typically implemented as a tablet or PC. The specific operations of the device are as follows:

[1220] When a student logs in to a device, the device records their activity in real time and periodically sends data to the server, such as recording the questions the student answered and their results.

[1221] The personalized learning materials and learning plans sent from the server are displayed on a dashboard on the device, which is built using front-end frameworks such as React and Angular.

[1222] Students can use this dashboard to select the learning materials that are most suitable for them and proceed with their studies.

[1223] Students' answers and progress are recorded by the device and fed back to the server.

[1224] For example, Student B works on a specific math problem set provided on his / her device, and progresses through his / her studies while looking at the answers and explanations for each problem. At this time, the device records the answers and sends them to the server.

[1225] Users (students and teachers)

[1226] The user is also an important part of the system. Here are some specific user actions:

[1227] Students study materials optimized for them through their devices, and their learning behavior and progress are automatically recorded by the devices, allowing them to concentrate on their studies.

[1228] Teachers can access the server and monitor students' learning progress and analysis results in real time. For example, teachers can access the dashboard from a browser to check students' grades and progress.

[1229] Based on this, the teacher can provide feedback to the student and provide additional support materials or advice. For example, if a particular student shows a lack of understanding of a particular theorem, the teacher can provide that student with additional learning materials.

[1230] For example, Teacher C can use the server's dashboard to monitor the progress of the entire class and individual students, adjust the lesson plan as needed, and provide additional supplementary materials if a particular student is found to have a lack of understanding of a particular concept.

[1231] As described above, by using a generative AI model, the present invention can provide personalized educational support tailored to each student's learning style and progress, thereby reducing the workload of teachers and providing a more effective learning environment for students.

[1232] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1233] Step 1: Device collects student learning activity data

[1234] How it works: Students log in to their devices and begin working on math problems. The devices then record data in real time, such as the answers and time it took to complete each problem, via a learning app.

[1235] Input: Answer data, answer time, and progress of students logged in to the device

[1236] Output: Learning activity data recorded by the device

[1237] Step 2: The device sends the learning activity data to the server.

[1238] Specific operation: The device sends the student's learning data to the server at regular intervals or when the learning activity is completed, for example, after each question is answered.

[1239] Input: Learning activity data recorded on the device

[1240] Output: Learning activity data sent to the server

[1241] Step 3: The server saves the training data to a database and queues it for analysis jobs.

[1242] Specific operation: After receiving the training data, the server stores it in a database. The stored data is then queued for analysis jobs. This process is automated, enabling efficient data management.

[1243] Input: Learning activity data sent to the server

[1244] Output: Learning activity data stored in the database, data input to the analysis job

[1245] Step 4: The server analyzes the training data using the generated AI model

[1246] How it works: The server takes the data from the queue and inputs it into the generative AI model, which analyzes each student's learning style and level of understanding, and then generates optimal teaching materials and lesson plans based on the results.

[1247] Input: Training data submitted to the analysis job

[1248] Output: Analysis of each student's learning style, level of understanding, and progress

[1249] Step 5: The server generates personalized learning materials and learning plans based on the analysis results.

[1250] What it does: Based on the analysis, it uses a generative AI model to generate personalized learning materials and lesson plans. For example, if a particular student is struggling with a particular math concept, it generates practice exercises and instructional videos that reinforce that concept.

[1251] Input: Analysis of each student's learning style, level of understanding, and progress

[1252] Output: personalized learning materials and learning plans

[1253] Step 6: The server sends the generated learning materials and lesson plans to the students' devices.

[1254] Specific operation: The server sends the generated personalized teaching materials and learning plans to the corresponding student's device.

[1255] Input: Generated personalized learning materials and learning plans

[1256] Output: Materials and lesson plans sent to the corresponding student devices

[1257] Step 7: The device displays the sent learning material and the student continues learning based on it.

[1258] Specific operation: The device displays the learning materials and study plans received from the server on a dashboard. Students can select the appropriate learning materials from this dashboard and proceed with their studies. For example, they can work on the displayed problem sets and progress through the answers.

[1259] Input: Personalized learning materials and study plans sent from the server

[1260] Output: Materials and lesson plans displayed on the dashboard, and student learning activities

[1261] Step 8: The device records the student's answers and progress and sends them back to the server.

[1262] Specific operation: The device again records the student's answers and progress and sends them back to the server. This data is used for analysis in the next cycle.

[1263] Input: Student answers and progress

[1264] Output: Answer results and progress feedback to the server

[1265] Step 9: The server continuously monitors your progress and reevaluates your learning plan based on new data.

[1266] Specific operation: The server continuously monitors learning progress based on the feedback data, analyzes new data, and reevaluates and modifies the learning plan.

[1267] Input: Feedback of answers and progress data

[1268] Output: Updated learning plan

[1269] This series of steps enables effective learning support that is customized for each student.

[1270] (Application example 1)

[1271] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1272] In modern factories, there is a demand for efficient training and improved skills for engineers and workers, but conventional training methods make it difficult to provide appropriate feedback to individual engineers, limiting productivity improvements. Another issue is the lack of technology to efficiently collect and analyze worker activity data and generate training plans tailored to each individual's skill level.

[1273] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1274] In this invention, the server includes means for collecting work activity data of workers, means for analyzing the collected data using a generative AI model, means for generating personalized training plans and learning materials based on the analysis results, means for providing the generated training plans and learning materials to workers, and means for continuously monitoring the progress of workers and reevaluating the training plans based on new data, thereby enabling efficient and personalized training to be provided to engineers and workers.

[1275] "Worker" refers to a person who performs a specific task in a factory or production site.

[1276] "Work activity data" refers to information about the process and results of work performed by workers, and is data collected through devices such as sensors and tablets.

[1277] A "generative AI model" refers to an artificial intelligence model that analyzes collected data and generates personalized training plans, learning materials, etc.

[1278] A "training plan" is a plan aimed at improving a worker's techniques and skills, and includes learning content optimized for each individual worker.

[1279] "Learning materials" refers to the materials and content that workers use to learn according to the training plan, including videos, workbooks, tutorials, etc.

[1280] "Means of provision" refers to the method of providing the generated training plans and learning materials to workers at an appropriate time.

[1281] "Progress monitoring" refers to the continuous observation and recording of workers' progress as they progress through training and their work.

[1282] "Reassessment" refers to reviewing existing training plans and learning methods based on new data collected and making improvements or changes as needed.

[1283] "Means for analyzing data" refers to methods and techniques for processing collected data and assessing worker skills and understanding.

[1284] This invention relates to a system that supports the skill improvement and efficient training of engineers and workers in factories. This system collects data on workers' work activities, analyzes the data using a generative AI model, and generates and provides personalized training plans and learning materials based on the analysis results.

[1285] Hardware and software used

[1286] Hardware

[1287] Training tablet or computer (device)

[1288] Data collection sensors

[1289] Learning information storage server

[1290] GPU server for high-speed calculations (operation of generative AI models)

[1291] software

[1292] Learning Management System (LMS)

[1293] Python-based analysis program

[1294] Generative AI model library (PyTorch or TensorFlow)

[1295] Database management system (PostgreSQL)

[1296] RESTful API (Flask or Django)

[1297] Explanation of the processing flow

[1298] 1. Data Collection

[1299] The server uses sensors and devices to collect work activity data when workers run the training program on their devices. This data includes information about the work process and results. For example, sensors record the time and steps for each process in the assembly of parts.

[1300] 2. Data Analysis

[1301] The server stores the collected work activity data in a database and analyzes it using a generative AI model. The analysis evaluates the worker's skill level and tendency for operational errors. The generative AI model is implemented using a Python-based analysis program and libraries such as PyTorch and TensorFlow.

[1302] 3. Training plan generation

[1303] Based on the analysis results, the server generates personalized training plans and learning materials. These training plans include video tutorials and problem sets that include specific work content and areas for improvement. For example, for the assembly procedure of a new part, the server generates materials that include countermeasures for common mistakes made by workers.

[1304] 4. Submissions and Feedback

[1305] The generated training plan and learning materials are sent to the terminal. As the worker progresses through the training, new data is collected and sent back to the server. This allows the server to reevaluate the training plan based on the latest data and make adjustments as necessary. This feedback loop ensures that the most optimal training is always provided.

[1306] Specific examples

[1307] When training workers on assembly procedures for a new part, they watch a video tutorial on a tablet and then perform a training simulation. Sensors and the tablet collect data during the simulation and send it to a server. A generative AI model analyzes the data and generates additional teaching materials for areas where mistakes are common, which are then sent back to the tablet. This process allows workers to effectively improve their skills.

[1308] Prompt Sentence Examples

[1309] "Please suggest the best training content for the next lesson. Operator A made many mistakes when assembling part B. Please generate teaching materials that include specific problem areas and how to improve them."

[1310] This invention makes it possible to provide efficient and individualized training to engineers and workers, which is expected to improve productivity.

[1311] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1312] Step 1:

[1313] The server collects work activity data from sensors and devices when a worker runs a training program on the device. When the worker presses the start button, the device starts collecting sensor data and sends data about the work process and results to the server. The input is data from the sensors and devices, and the output is raw data stored on the server.

[1314] Step 2:

[1315] The server stores the collected data in a database and performs data cleansing, removing incomplete or noisy data and shaping the data. The input is the raw collected data, and the output is cleaned data that can be analyzed.

[1316] Step 3:

[1317] The server analyzes the cleaned data using a generative AI model. A Python-based analysis program is used to evaluate the worker's skill level and tendency for operational errors. The input is the cleaned data, and the output is a technical evaluation report.

[1318] Step 4:

[1319] The server generates personalized training plans and learning materials based on the analysis results. It uses a generative AI model library (PyTorch or TensorFlow) to generate optimal learning materials for each worker. The input is the technical evaluation report, and the output is personalized training plans and learning materials.

[1320] Step 5:

[1321] The server distributes the generated training plan and learning materials to the terminal. The terminal receives them and provides them to the worker. The worker checks the training plan and learning materials on the terminal and progresses with their learning. The input is the individualized training plan and learning materials, and the output is the worker's learning progress.

[1322] Step 6:

[1323] The terminal collects the worker's training progress data and sends it to the server, including the training completion status and any new mistakes. The input is the worker's progress data, and the output is the data sent to the server.

[1324] Step 7:

[1325] The server re-evaluates the training plan based on the new data received, again using the generative AI model to adjust the training content as needed. The input is the new progress data, and the output is the adjusted training plan.

[1326] Step 8:

[1327] The server then delivers the re-evaluated training plan and learning materials back to the terminal and provides them to the worker. This feedback loop ensures continuous optimal training. The input is the adjusted training plan and learning materials, and the output is the worker's latest learning progress.

[1328] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1329] This invention relates to a learning support system that combines a generative AI model and an emotion engine, and provides means for collecting learning activity data, generating personalized learning materials, managing learning progress, and monitoring the user's emotional state. The system of this invention consists of three main entities: a server, a terminal, and a user. The specific roles and processing flow of each entity are explained below.

[1330] server

[1331] The server is the core of the system, responsible for collecting and analyzing learning activity data and emotion data, operating the generative AI model, and generating and providing personalized learning materials and learning plans. First, the server sets up an API endpoint and prepares to receive data sent from the device. The received data is stored in a database and queued for subsequent analysis jobs.

[1332] The server then combines the generative AI model with an emotion engine to perform analysis. It uses the learning data and emotion data to evaluate the student's learning style, comprehension, and emotional state. Based on the evaluation results, it generates an optimal learning plan and personalized learning materials. The generated learning materials are then sent to the corresponding student's device.

[1333] As a specific example, if the server determines that Student A is feeling anxious about a problem based on data and emotional data acquired during a mathematics class, it will generate and deliver additional practice problems that take into account the student's emotional state and explanatory videos for relaxation.

[1334] Terminal

[1335] The device functions as an interface for users (students) to study and is implemented on a tablet or PC. Data is transmitted between the student and the server via a learning app. When a student logs in, the device records their activity in real time and periodically sends the data to the server. Emotional data is also collected in the same way; for example, changes in the student's facial expression or voice are detected using a camera or microphone, and sent to the emotion engine.

[1336] The personalized learning materials and study plans sent from the server are displayed on a dashboard on the device. Students can select the materials that are most suitable for them and proceed with their studies from this dashboard. The device also instantly records the students' answers, progress, and emotional state and provides feedback to the server.

[1337] For example, if Student B is working on a specific math problem set provided on his / her device and loses concentration while solving it, his / her emotional state will be detected by the device, which will record it and send it to the server.

[1338] Users (students and teachers)

[1339] The user is also an important part of the system. Students use their devices to study materials that are optimized for them. Their behavior, progress, and emotional state are automatically recorded by the devices, allowing them to focus on their studies independently.

[1340] Teachers can access the server to monitor students' learning progress, analysis results, and emotional state in real time, and based on this, teachers can provide feedback to students and provide additional teaching materials and advice.

[1341] For example, Teacher C can use the server's dashboard to monitor the progress and emotional state of the entire class and individual students, adjust the lesson plan as needed, and provide additional supplementary materials or relaxation advice if a particular student is found to be struggling with a particular concept and feeling stressed.

[1342] In this way, learning support systems can reduce the workload of teachers while providing students with learning methods that are suited to them. The goal is to create an environment in which all students can enjoy equal learning opportunities and improve the quality of education.

[1343] The processing flow will be explained below.

[1344] Step 1:

[1345] The server opens an API endpoint to collect learning data and emotional data. First, it prepares to receive learning requests, and then receives student learning activity data (e.g., subjects taken, study time, progress) and emotional data sent from the device. This data is then stored in a database.

[1346] Step 2:

[1347] The device starts recording session information when a student logs in to the learning app. Specifically, it records the user ID and login time when logging in to study, and collects data each time a learning activity begins. It also uses the camera and microphone to simultaneously collect emotional data such as facial expressions and voice.

[1348] Step 3:

[1349] Users (students) log in to the learning app and select the designated subject or unit. Every time they solve a problem or watch a video, their actions are automatically recorded on the device. Emotional data obtained using the camera and microphone is also recorded. When they finish learning, they log out.

[1350] Step 4:

[1351] The server stores the received learning data and emotion data in a database, and at the same time inputs the data into the generative AI model and emotion engine to execute the analysis job. Specifically, the data stored in the database is periodically put into the analysis queue, which triggers the generative AI model and emotion engine to perform analysis.

[1352] Step 5:

[1353] The generative AI model and emotion engine evaluate each student's learning style, level of understanding, and emotional state based on collected learning and emotion data. The analysis results are used to generate optimal learning plans and personalized learning materials. Specifically, the generative AI model identifies areas that need reinforcement based on a student's past performance, learning patterns, and emotions.

[1354] Step 6:

[1355] The server sends the personalized learning materials and learning plans generated by the generative AI model and emotion engine to the device. Specifically, the generated learning materials and plans are delivered to the device via API and displayed on the device's user interface.

[1356] Step 7:

[1357] The device then displays the received personalized learning materials and learning plans on a dashboard for students to access. Specifically, the device analyzes the learning material data received from the server and displays it in a user-friendly format.

[1358] Step 8:

[1359] The user (student) progresses through their studies using personalized learning materials provided on the device's dashboard. Specifically, students work through the provided practice problems and instructional videos, and their progress and results are automatically recorded on the device. Emotional data is also continuously recorded and sent to the server in real time.

[1360] Step 9:

[1361] The device records activity, progress, and emotional data during learning in real time and periodically transmits them to the server. Specifically, at the end of each learning session, the device compiles the session data and sends it to the server as a POST request.

[1362] Step 10:

[1363] The server continuously receives new learning and emotion data and re-evaluates the learning plan based on the analysis results. Specifically, it incorporates the newly received data into the analysis job and retrains the generative AI model and emotion engine accordingly to optimize the learning plan.

[1364] Step 11:

[1365] Teachers can check students' progress, analysis results, and emotional state in real time through the server's dashboard. Specifically, the teacher's management screen monitors each student's learning progress, level of understanding, and emotional state, and provides feedback as needed.

[1366] Step 12:

[1367] The user (teacher) can input supplementary materials and advice into the system based on each student's learning progress and emotional state, and provide them to the student. Specifically, the teacher uses the feedback function on the dashboard to input individual advice and supplementary materials, which are then automatically sent to the student's device.

[1368] Example 2

[1369] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1370] Conventional learning support systems often only collect and analyze students' learning activity data and do not consider their emotional state, making it difficult to improve the quality of education. Furthermore, they often fail to generate personalized learning materials and learning plans, resulting in ineffective instruction. Furthermore, teachers have limited means to monitor students' learning progress in real time and provide appropriate feedback. Therefore, there is a need for systems that can provide a more effective learning environment and realize education that meets the needs of each individual student.

[1371] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes means for collecting learning activity data and emotional data of students, means for analyzing the collected learning data and emotional data using a generative AI model and an emotion engine, and means for generating individualized learning materials and learning plans based on the analysis results. This makes it possible to evaluate students' learning styles, levels of comprehension, and emotional states in real time and provide optimal learning plans and learning materials.

[1372] "Student" refers to a learner who uses the learning support system.

[1373] "Learning activity data" refers to information such as operation logs, answer data, and study time generated by students while they are studying.

[1374] "Emotional data" refers to information about a student's emotional state obtained from facial expressions, tone of voice, etc.

[1375] A "generative AI model" refers to an artificial intelligence algorithm that automatically generates learning plans and teaching materials based on data.

[1376] An "emotion engine" refers to software that analyzes students' emotional state and provides the results to a learning support system.

[1377] "Analysis method" refers to the process of analyzing collected data using generative AI models and emotion engines.

[1378] "Personalized learning materials" refers to learning materials that are optimized based on a student's learning style and level of understanding.

[1379] A "learning plan" refers to a set of learning tasks or schedule designed to efficiently carry out students' learning activities.

[1380] "Monitoring" refers to the process of continuously observing students' academic progress and emotional state.

[1381] "Feedback" refers to the guidance and advice that teachers provide to students based on their learning progress and emotional state.

[1382] The present invention relates to a learning support system that combines a generative AI model and an emotion engine, and is specifically implemented as follows.

[1383] server

[1384] The server is the core of the learning support system and performs the following functions:

[1385] 1. Data Collection and Storage

[1386] The server sets up an API endpoint to receive learning activity data and emotion data sent from the device, which is then stored in a database and queued for analysis.

[1387] 2. Data Analysis

[1388] The server combines a generative AI model and an emotion engine to perform analysis. Specifically, it evaluates students' learning style, comprehension, and emotional state based on collected learning activity data and emotion data. These evaluation results are used to generate optimal learning plans and personalized learning materials.

[1389] 3. Creation and distribution of teaching materials

[1390] Based on the evaluation results, the server uses a generative AI model to generate personalized learning materials and study plans. The generated learning materials are sent to the corresponding student's device. For example, if the server determines that Student A is feeling anxious about a problem based on math lesson data and emotional data, it will generate an explanatory video for relaxation and additional practice problems and deliver them to the device.

[1391] Terminal

[1392] The device functions as a learning interface for users (students) and is implemented on a tablet or PC. It performs the following functions:

[1393] 1. Data Collection

[1394] When a student logs in to their device, the learning app starts and records learning activity data and emotional data in real time. The app uses a camera and microphone to collect emotional data from facial expressions and tone of voice, and sends it to a server.

[1395] 2. Viewing learning content

[1396] The personalized learning materials and study plans sent from the server are displayed on a dashboard on the device, where students can select the materials that are most suitable for them and proceed with their studies.

[1397] 3. Feedback

[1398] The students' answers, learning progress, and emotional state are recorded in real time and fed back to the server. For example, if Student B loses concentration while working on a math problem, the device detects this emotional state and immediately sends the data to the server.

[1399] Users (students and teachers)

[1400] The user is also an important part of the system and performs the following functions:

[1401] 1. Students

[1402] Students use their devices to study materials that are optimized for them, and their behavior, progress, and emotional state are automatically recorded by the devices, allowing them to focus on their studies independently.

[1403] 2. Teacher

[1404] Teachers access the server and monitor students' learning progress, analysis results, and emotional states in real time. Based on this, they provide feedback and additional supplementary materials or advice as needed. For example, Teacher C uses the server's dashboard to check the progress and emotional state of a specific student and adjust the teaching plan as needed.

[1405] Prompt Sentence Examples

[1406] Below are some examples of specific prompt sentences.

[1407] 1. Prompts to analyze student progress

[1408] Analyze the following dataset to assess student progress and understanding: {data_set}

[1409] 2. Prompt sentences for generating teaching materials based on emotion data

[1410] Analyze the following emotion data and generate personalized math exercises adapted to the emotion: {emotion_data}

[1411] In this way, this system utilizes generative AI models and emotion engines to provide optimal learning methods for students, reduce the workload of teachers, and improve the quality of education by creating an environment in which all students have equal access to learning opportunities.

[1412] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1413] Step 1: Data collection and transmission

[1414] Terminal

[1415] The device launches the learning app when the user (student) logs in.

[1416] The device records students' learning activity data (operation logs, answer data, study time) in real time through the learning app.

[1417] The device uses a camera and microphone to collect emotional data such as students' facial expressions and tone of voice.

[1418] The terminal periodically transmits this data to the server.

[1419] Input: Learning activity data and emotion data of logged-in students

[1420] Output: Learning activity data and emotion data sent to the server

[1421] Specific behavior:

[1422] While Student A logs in to the device and solves math problems, the device records the operation log and progress of the answer in real time. The camera also captures Student A's facial expressions and generates data to be sent to the emotion engine.

[1423] Step 2: Data storage and analysis

[1424] server

[1425] The server receives the learning activity data and emotion data sent from the terminal at an API endpoint.

[1426] The server stores the received data in a database.

[1427] The server puts the stored data into an analysis queue and performs the analysis using generative AI models and emotion engines.

[1428] The server evaluates the student's learning style, level of understanding, and emotional state based on learning data and emotional data.

[1429] Input: Learning activity data and emotion data sent from the device

[1430] Output: Assessment results of student learning style, comprehension, and emotional state

[1431] Specific behavior:

[1432] The server receives Student A's learning log and emotional data and stores them in a database. Next, it inputs a prompt statement to the generative AI model: "Analyze Student A's learning activity data and emotional data, and evaluate his learning style and level of comprehension.", and obtains the evaluation results.

[1433] Step 3: Generate and deliver personalized learning materials

[1434] server

[1435] The server generates optimal learning plans and personalized learning materials based on the analysis results.

[1436] The server generates teaching materials using the generative AI model and sends the prepared teaching materials to the corresponding student's device.

[1437] Input: Assessment results of student learning style, comprehension, and emotional state

[1438] Output: personalized learning materials and learning plans

[1439] Specific behavior:

[1440] Based on the evaluation results, the server inputs a prompt to the generative AI model, such as "Generate an instructional video for relaxation," and generates a personalized instructional video and additional practice questions, which are then sent to the device.

[1441] Step 4: View your learning and receive progress feedback

[1442] Terminal

[1443] The device displays the learning materials and study plans sent from the server on a dashboard.

[1444] The device records the student's answers, learning progress, and emotional state in real time and provides feedback to the server.

[1445] Input: Personalized learning materials and study plans sent from the server

[1446] Output: Data on students' learning progress and emotional state

[1447] Specific behavior:

[1448] Student B selects a personalized math problem set from the device's dashboard and proceeds with the answering process. Meanwhile, the device records Student B's answers and emotional state in real time and sends the data to the server.

[1449] Step 5: Feedback and Action

[1450] User (Teacher)

[1451] Teachers access the server and monitor students' learning progress, analysis results, and emotional state in real time.

[1452] Teachers provide feedback and supplementary materials and advice as needed.

[1453] Input: Student learning progress data and emotion data stored on the server

[1454] Output: Feedback, teaching aids, and advice for students

[1455] Specific behavior:

[1456] Teacher C uses the server's dashboard to check Student C's progress and emotional state. If Student C lacks understanding of a particular concept and is feeling stressed, Teacher C can provide additional quizzes or relaxation advice.

[1457] (Application example 2)

[1458] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1459] In the work environment of factories, there is a demand for improving work efficiency and safety, but conventional methods have limitations. There is no system that collects and analyzes workers' work data and emotional data in real time, and then provides individualized work guidance and safety measures, which leads to problems such as reduced work efficiency and a lack of safety.

[1460] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for collecting work data and emotion data of workers, means for analyzing the collected work data and emotion data using a generative AI model, and means for generating individualized work instructions and safety measures based on the analysis results. This makes it possible to provide optimal work instructions and safety measures in real time based on the work data and emotion data of workers.

[1461] "Work data" refers to data related to the work content, procedures, and efficiency of workers, and is information collected by measuring devices and sensors.

[1462] "Emotional data" refers to data that indicates the emotional state of a worker, detected from their facial expressions, voice, body movements, etc., and is acquired using sensors such as cameras and microphones.

[1463] A "generative AI model" is an algorithm that analyzes collected data and generates optimal guidance and countermeasures for workers, and refers to an artificial intelligence system that uses machine learning and deep learning technologies.

[1464] "Means of analysis" refers to the process of analyzing data using a generative AI model based on collected task data and emotion data, and extracting meaningful information.

[1465] "Individualized work instruction" refers to information and instructions that guide optimal work procedures and methods according to the abilities and conditions of each individual worker.

[1466] "Safety measures" refers to measures, including actions and protective measures, that must be taken to minimize risks during work.

[1467] "Worker" refers to a person who performs work in a factory or work site.

[1468] "Manager" refers to a person whose role is to monitor the work data and progress of workers and provide feedback as needed.

[1469] "Real-time monitoring" refers to instantly collecting and analyzing work data and emotional data to grasp the worker's current state at every moment.

[1470] "Means for providing feedback" refers to a method or system that provides instructions or advice to workers based on analysis results or monitoring data.

[1471] The present invention relates to a work support system for factories. This system collects and analyzes work data and emotional data of workers, and provides individualized work guidance and safety measures.

[1472] server

[1473] The server is the core of the system and performs the following tasks:

[1474] 1. Collecting and storing task and emotion data.

[1475] 2. Analyze collected data using generative AI models.

[1476] 3. Generate and provide individualized work instructions and safety measures based on the analysis results.

[1477] Specifically, the server sets up an API endpoint and prepares to receive data sent from the device. The data is stored in a database and queued for subsequent analysis jobs. The generative AI model and emotion engine are combined to perform the analysis and evaluate the worker's working style and emotional state. Based on the evaluation results, optimal work instructions and safety measures are generated and sent to the corresponding worker's device.

[1478] The specific hardware used includes a database server and a GPU server for computing, and the software includes a database (e.g., MySQL), a deep learning framework (e.g., TensorFlow), and an emotion analysis library (e.g., EmotionEngine).

[1479] Terminal

[1480] The terminal functions as an interface for the worker and is mainly implemented as a smartphone, tablet, or wearable device. It collects work data and emotional data and sends it to the server in real time. Emotional data is collected using the device's camera and microphone, and changes in the worker's facial expressions and voice are detected and sent to the emotion engine.

[1481] The server sends personalized work instructions and safety measures to the device, which are then displayed on a dashboard. The worker receives instructions from the dashboard and puts them into action. The device also records the worker's work progress and emotional state and sends feedback to the server.

[1482] Users (operators and administrators)

[1483] Workers receive personalized guidance and safety measures via their devices. Their movements and emotional state are automatically recorded by the devices, allowing them to concentrate on their work. Supervisors can access the server and monitor the workers' work progress, analysis results, and emotional state in real time. Based on this, supervisors can provide feedback to workers and offer additional guidance and advice.

[1484] For example, when a worker is assembling parts, work data and emotional data are collected via the device's camera. If the server analyzes this data and determines that the worker is under stress, instructions to adjust the work pace or take a break are sent to the device. The worker can follow these instructions and continue working safely and efficiently.

[1485] An example of a prompt is, "If a tired expression is detected, please generate advice on work pace." The generative AI model generates appropriate advice based on this prompt, optimizing the worker's working environment.

[1486] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1487] Step 1: Data collection

[1488] Terminal: Using cameras and sensors installed on the terminal, the work data and emotional data of the worker are collected in real time. During this collection process, changes in the worker's movements, facial expressions, and voice are detected. The input is the real-time data of the worker, and the output is the collected work data and emotional data.

[1489] Step 2: Send data

[1490] Terminal: The collected task data and emotion data are periodically sent to the server. The input is the collected data, and the output is the data sent to the server. Specifically, the data is sent to the server via an API.

[1491] Step 3: Save Data

[1492] Server: Stores the received task data and emotion data in a database. The input is the data sent from the device, and the output is the stored data. Specifically, it creates a database entry and stores the data.

[1493] Step 4: Submit a data analysis job

[1494] Server: Queues jobs to analyze the stored data. The input is the stored data, and the output is the enqueuing of analysis jobs.

[1495] Step 5: Data analysis

[1496] Server: Analyzes the analysis jobs registered in the queue using a generative AI model and an emotion engine. The input is the data entered in the queue, and the output is the analysis results. Specifically, the generative AI model is used to evaluate the worker's working style and emotional state.

[1497] Step 6: Generate personalized task instructions

[1498] Server: Generates individualized work instructions and safety measures based on the analysis results. The input is the analysis results, and the output is the generated work instructions and safety measures. For example, this includes adjusting work pace and instructions for breaks.

[1499] Step 7: Sending work instructions

[1500] Server: Sends the generated work instructions and safety measures to the terminal. The input is the generated instruction content, and the output is the transmission to the terminal. Specifically, the instruction content is sent to the terminal as JSON format data via API.

[1501] Step 8: Work Instructions

[1502] Terminal: The received work instructions and safety measures are displayed on a dashboard and notified to the worker. The input is the data received from the server, and the output is the displayed instruction content. Specifically, the information is displayed using UI components.

[1503] Step 9: Progress Feedback

[1504] Terminal: The worker continues working based on the instructions and records their progress. The recorded progress data and new emotion data are sent to the server again. The input is the worker's progress data and emotion data, and the output is the data sent to the server.

[1505] Step 10: Reassess your progress data

[1506] Server: Analyzes the received progress data and emotion data again and generates new guidance and countermeasures. The input is the collected data again, and the output is the new guidance content. Specifically, the data is analyzed using the generative AI model and emotion engine, just like last time.

[1507] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1508] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1509] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1510] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1511] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1512] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1513] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1514] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1515] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1516] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1517] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1518] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1519] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1520] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1521] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1522] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1523] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1524] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1525] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1526] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1527] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1528] The following is further disclosed regarding the above embodiment.

[1529] (Claim 1)

[1530] A means of collecting data on student learning activities;

[1531] A means of analyzing the collected learning data using a generative AI model;

[1532] a means for generating personalized learning materials and learning plans based on the analysis results;

[1533] A means of providing generated learning materials and lesson plans to students;

[1534] A means to continually monitor student progress and reassess learning plans based on new data; and

[1535] A system including:

[1536] (Claim 2)

[1537] The system of claim 1, wherein the generative AI model generates more effective lesson plans by taking into account the student's preferences and level of understanding.

[1538] (Claim 3)

[1539] 10. The system of claim 1, further comprising means for a teacher to monitor student learning progress data in real time and provide feedback as needed.

[1540] "Example 1"

[1541] (Claim 1)

[1542] A means for collecting student learning activity data from the device;

[1543] A means for storing the collected training data in a database and queuing it for analysis jobs;

[1544] A means to analyze learning data using generative AI models and evaluate each student's learning style, level of understanding, and progress;

[1545] a means for generating personalized learning materials and learning plans based on the analysis results;

[1546] A means for transmitting the generated teaching materials and learning plans to the student's device;

[1547] A means to record students' answers and progress and provide feedback to the server;

[1548] A means to continually monitor student progress and reassess learning plans based on new data; and

[1549] A system including:

[1550] (Claim 2)

[1551] The system of claim 1, wherein the generative AI model generates more effective lesson plans by taking into account the student's preferences and level of understanding.

[1552] (Claim 3)

[1553] 10. The system of claim 1, further comprising means for a teacher to access the server to monitor the student's learning progress data in real time and provide feedback as needed.

[1554] "Application Example 1"

[1555] (Claim 1)

[1556] means for collecting worker activity data;

[1557] A means of analyzing the collected data using a generative AI model;

[1558] means for generating personalized training plans and learning materials based on the analysis results;

[1559] a means for providing the generated training plans and learning materials to workers;

[1560] A means to continually monitor worker progress and reassess training plans based on new data;

[1561] A system including:

[1562] (Claim 2)

[1563] The system of claim 1, wherein the generative AI model generates more effective training plans by taking into account the worker's preferences and level of understanding.

[1564] (Claim 3)

[1565] 10. The system of claim 1, further comprising means for a manager to monitor worker progress data in real time and provide feedback as needed.

[1566] "Example 2: Combining Emotion Engines"

[1567] (Claim 1)

[1568] a means for collecting student learning activity data and emotional data;

[1569] A means for analyzing the collected learning data and emotion data using a generative AI model and emotion engine;

[1570] a means for generating personalized learning materials and learning plans based on the analysis results;

[1571] A means of providing generated learning materials and lesson plans to students;

[1572] A means of continually monitoring students' academic progress and emotional state and reassessing their learning plans based on new data; and

[1573] A system including:

[1574] (Claim 2)

[1575] The system of claim 1, wherein the generative AI model generates more effective lesson plans by taking into account the student's preferences, level of understanding, and emotional state.

[1576] (Claim 3)

[1577] 10. The system of claim 1, further comprising means for a teacher to monitor the learning progress data and emotional data of the students in real time and provide feedback as needed.

[1578] "Application example 2 when combining emotion engines"

[1579] (Claim 1)

[1580] A means for collecting work data and emotion data of a worker;

[1581] A means for analyzing the collected work data and emotion data using a generative AI model;

[1582] A means for generating individualized work instructions and safety measures based on the analysis results;

[1583] A means for providing the generated work instructions and safety measures to workers;

[1584] A means to continuously monitor the progress of workers and reassess work plans based on new data;

[1585] A system including:

[1586] (Claim 2)

[1587] The system of claim 1, wherein the generative AI model generates more effective work plans by taking into account the capabilities and experience of workers.

[1588] (Claim 3)

[1589] 10. The system according to claim 1, further comprising means for a manager to monitor the work progress data of the workers in real time and provide feedback as necessary. [Explanation of symbols]

[1590] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. A means of collecting data on student learning activities; A means of analyzing the collected learning data using a generative AI model; a means for generating personalized learning materials and learning plans based on the analysis results; A means of providing generated learning materials and lesson plans to students; A means to continually monitor student progress and reassess learning plans based on new data; and A system including:

2. The system of claim 1, wherein the generative AI model generates more effective lesson plans by taking into account the student's preferences and level of understanding.

3. 10. The system of claim 1, further comprising means for a teacher to monitor student learning progress data in real time and provide feedback as needed.

Images