system

Abstract

Provide a system. 【Solution means】Means for tracking the progress data of learners, Means for generating a lecture video optimized based on the progress data, Means for distributing the lecture video to learners, Means for automatically generating a small test or a regular test for learners, Means for performing OCR processing on the answers of the test, Means for automatically grading the OCR-processed answers, Means for generating appropriate answers to the questions of learners, Means for providing the most suitable teaching materials and learning plans for learners, A system including.

Description

Technical Field

[0004] , , , ,

[0005] , , , , , ,

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to the description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance that responds to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Conventional online learning systems have had problems in that it is difficult to provide optimal educational resources for individual learners and they can only provide uniform guidance. Also, in the implementation and grading of tests, human resources are required, and it has been a challenge to perform efficient and accurate evaluations. Furthermore, in answering learners' questions and creating individual learning plans, conventional systems have had problems in that it is difficult to meet individual needs.

Means for Solving the Problems

[0005] This invention provides individually optimized educational resources by providing means for tracking learners' progress data and means for generating optimized lesson videos based on said progress data. Furthermore, by using avatars and a voice generation engine in the lesson videos, a more effective learning experience is realized. In addition, by providing means for automatically generating quizzes or periodic tests for learners, and for processing and automatically grading them using OCR, efficient and accurate evaluation can be performed without requiring human resources. Moreover, by providing means for generating appropriate answers to learners' questions using an AI model, and means for providing optimal teaching materials and learning plans based on learning progress, support that meets the individual needs of each learner is realized.

[0006] "Means for tracking learner progress data" refers to a function that collects, records, and analyzes data such as learning activities, test results, and assignment submission status performed by learners on an online learning platform.

[0007] "Methods for generating optimized lesson videos" refers to a function that creates lesson videos tailored to individual comprehension levels and learning needs based on collected learner progress data.

[0008] "Means of distributing lesson videos to learners" refers to a function that transmits generated lesson videos to learners' devices via the internet and makes them playable.

[0009] "Means for automatically generating quizzes or periodic tests" refers to a function that generates appropriate test questions using AI models or similar technologies, based on the learner's progress and academic level.

[0010] "Method for processing test answers using OCR" refers to a function that converts test answers submitted by learners into digital data using optical character recognition technology.

[0011] "Methods for automatically scoring OCR-processed answers" refers to a function that uses AI models and algorithms to score test answers that have been digitized through OCR processing and generate results.

[0012] "A means of generating appropriate answers to learners' questions" refers to a function that receives questions from learners, uses an AI model to generate appropriate answers, and provides them.

[0013] "Means of providing optimal learning materials and study plans" refers to a function that generates and provides optimal learning materials and study plans to learners based on their progress data and learning needs. [Brief explanation of the drawing]

[0014] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of the data processing device and smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11] This is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12]It is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] It is a sequence diagram showing the processing flow of the data processing system in Embodiment 2 when combined with an emotion engine. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when combined with an emotion engine.

Mode for Carrying Out the Invention

[0015] Hereinafter, an example of an embodiment of the system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0016] First, the language used in the following description will be explained.

[0017] In the following embodiments, the numbered processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0018] In the following embodiments, the numbered RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0019] In the following embodiments, the signed storage is one or more non-volatile storage devices that store various programs and various parameters. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes.

[0020] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0021] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0022] [First Embodiment]

[0023] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0024] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0025] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0026] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0027] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0028] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0029] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0030] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0031] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0032] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0033] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0034] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0035] This invention relates to an AI-powered online learning system that tracks learners' progress and provides individually optimized educational resources. The system consists of a server, a terminal, and a user (learner), each performing a specific function.

[0036] Tracking learning progress

[0037] server

[0038] The server collects data such as learners' learning activities, test results, and assignment submission status, and records it in a database. It also uses an AI model to analyze this data and understand the learners' progress. This allows for the provision of instruction tailored to each individual learner.

[0039] terminal

[0040] When a learner logs in, their device sends authentication information to the server. Upon successful authentication, the learning dashboard sent from the server is displayed. This dashboard shows the learner's progress, the status of submitted assignments, and other relevant information.

[0041] Generation and distribution of lesson videos

[0042] server

[0043] The server uses an AI model to generate individually optimized lesson videos based on the learner's progress data. These lesson videos are visually easy to understand using avatars and include audio explanations using a speech generation engine. Once the lesson videos are generated, the server delivers them to the user's device.

[0044] terminal

[0045] The terminal displays lesson videos received from the server to the learner. By watching these videos, learners can receive personalized instruction.

[0046] Generation and grading of quizzes and regular tests

[0047] server

[0048] The server automatically generates appropriate quizzes or periodic tests based on learning progress data. This allows for effective assessment of learners' understanding. When learners take a test, the server processes the submitted answers using OCR and automatically grades them using an AI model.

[0049] terminal

[0050] The terminal displays test questions sent from the server and provides an interface for learners to input their answers. Once the learner completes the test, the terminal sends the answers to the server, and the server displays the graded results in real time.

[0051] Question and Answer Session

[0052] server

[0053] The server has an AI model installed for question-and-answer sessions. When a learner submits a question, the server analyzes the question and generates the best possible answer.

[0054] terminal

[0055] When a learner enters a question using the terminal's question-and-answer interface, the terminal sends the question to the server. The answer provided by the server is then displayed to the learner through the terminal.

[0056] Provision of teaching materials and learning plans

[0057] server

[0058] The server generates personalized learning materials and study plans based on the learner's progress data. These plans include which subjects to focus on and recommended study times.

[0059] terminal

[0060] The terminal displays learning materials and study plans received from the server to the learner. This allows the learner to study efficiently.

[0061] Specific example

[0062] Tracking learning progress

[0063] When a user (learner) watches a math lesson video and then submits an assignment, the device automatically sends this information to the server. The server uses this data to update the learner's progress and saves it in a database.

[0064] Generation and distribution of lesson videos

[0065] If a user (learner) struggles with the topic of "quadratic equations," the server generates a specialized lesson video on quadratic equations based on past data, adds audio explanations, and then delivers it to the user's device.

[0066] Generation and grading of quizzes and regular tests

[0067] If a user (learner) wants to take a test on quadratic equations, the server uses an AI model to automatically generate questions and delivers them to the device. When the learner enters their answers and the device sends them to the server, the server uses OCR to digitize the answers and the AI ​​model to grade them.

[0068] Question and Answer Session

[0069] When a user (learner) enters a question about "how to solve an equation," the device sends the question to the server. The server analyzes the question using an AI model, generates an appropriate answer, and provides it to the learner through the device.

[0070] Provision of teaching materials and learning plans

[0071] The server analyzes each learner's progress data and generates a study plan that includes subjects to focus on in the following week and recommended study time. The terminal displays this plan to the learner.

[0072] The above describes the embodiment of this invention. This system makes it possible to efficiently and effectively provide individually optimized educational resources.

[0073] The following describes the processing flow.

[0074] Tracking learning progress

[0075] Step 1:

[0076] The user (student) logs into the online learning platform using their device.

[0077] The terminal sends the user's authentication information to the server.

[0078] Step 2:

[0079] The server verifies the authentication information, and if authentication is successful, it generates a learning dashboard and sends it to the device.

[0080] The device displays the received dashboard, showing the user their current learning progress.

[0081] Step 3:

[0082] Users (students) watch lesson videos or take tests.

[0083] The device records these learning activities and sends them to the server in real time.

[0084] Step 4:

[0085] The server records the received training data in a database and performs analysis using an AI model.

[0086] The learner's progress is updated based on the analysis results.

[0087] Generation and distribution of lesson videos

[0088] Step 1:

[0089] The server starts generating individually optimized lesson videos based on the learner's progress data.

[0090] The AI ​​model selects the appropriate content and activates the avatar and voice generation engine.

[0091] Step 2:

[0092] The server generates visually easy-to-understand videos using avatars and adds narration using a voice generation engine.

[0093] The generated lesson videos are saved on the server.

[0094] Step 3:

[0095] The server sends the generated lesson videos to the terminal.

[0096] The device displays the received video to the user (student) and begins playback.

[0097] Generation and grading of quizzes and regular tests

[0098] Step 1:

[0099] The server automatically generates appropriate test questions using an AI model based on the learning progress data.

[0100] The generated tests are saved in digital format.

[0101] Step 2:

[0102] The server sends the generated test questions to the terminal.

[0103] Users (students) take the test through their devices.

[0104] Step 3:

[0105] The user (student) enters their answer and sends it to the server using their device.

[0106] The answers will be formatted appropriately for OCR processing.

[0107] Step 4:

[0108] The server digitizes the answers using an OCR engine and performs automatic scoring using an AI model.

[0109] The scoring results are saved in a database and immediately sent to the device as feedback.

[0110] Step 5:

[0111] The device displays the scoring results to the user (student) and suggests the next learning steps.

[0112] Question and Answer Session

[0113] Step 1:

[0114] The user (student) enters their question using the terminal's question-and-answer interface.

[0115] The terminal sends the question to the server.

[0116] Step 2:

[0117] The server receives the question and analyzes its content using an AI model.

[0118] The system generates the optimal response based on the analysis results.

[0119] Step 3:

[0120] The server sends the generated response to the terminal.

[0121] The device displays the answer to the user (student).

[0122] Provision of teaching materials and learning plans

[0123] Step 1:

[0124] The server generates individually optimized learning materials and study plans based on the learner's progress data.

[0125] The AI ​​model calculates the next learning goal and recommended learning time.

[0126] Step 2:

[0127] The server sends the generated learning materials and study plans to the terminal.

[0128] The terminal displays the received plan and learning materials to the user (student).

[0129] The above describes the specific operations at each processing step of the system. This system makes it possible to provide an optimal learning experience for each individual learner.

[0130] (Example 1)

[0131] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0132] Traditional online learning systems have difficulty individually tracking the progress of each learner and providing them with the most suitable learning materials and teaching methods. Furthermore, manual test creation and grading are time-consuming, making it difficult to quickly assess learners' understanding. Additionally, real-time question-and-answer sessions are not possible, preventing immediate resolution of learners' questions.

[0133] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0134] In this invention, the server includes means for tracking learner progress data, means for generating optimized lesson information based on the progress data, means for distributing the lesson information to learners, means for automatically generating quizzes or periodic tests for learners, means for performing optical character recognition (OCR) processing on the answers to the tests, means for automatically scoring the OCR-processed answers, means for generating appropriate answers to learners' questions, means for providing learners with optimal learning materials and learning plans, means for learners to engage in learning activities using a terminal, means for saving the learners' progress in a database using the progress data and analyzing it using an AI model, means for using an avatar and a voice generation engine to generate the lesson videos, and means for performing OCR processing on the answers and analyzing and displaying the scoring results using an AI model. This makes it possible to efficiently provide individually optimized educational resources and realize flexible instruction according to the learners' progress. Furthermore, the automation of test generation and scoring enables rapid and accurate learning assessment. Real-time question and answer allows learners' questions to be resolved immediately.

[0135] "Learner progress data" refers to data that includes information such as the learning activities undertaken by learners, test results, and assignment submission status.

[0136] "Lesson information" refers to educational content optimized based on learner progress data, and includes visual and audio lesson videos.

[0137] "Automatic generation of quizzes or periodic tests" refers to the process of automatically creating appropriate test questions based on learner progress data.

[0138] Optical character recognition (OCR) is a technology that converts handwritten or printed text into digital data.

[0139] "Automatic scoring" is a process that automatically evaluates and scores test answers converted through optical character recognition using an algorithm.

[0140] "Generating appropriate answers" means that the AI ​​model generates the best possible answer to a question submitted by the learner via their device.

[0141] "Providing optimal learning materials and study plans" means proposing and providing individually optimized learning materials and study plans based on the learner's progress data.

[0142] "Device" refers to a device used by learners to conduct learning activities, and includes, for example, computers, tablets, and smartphones.

[0143] An "avatar" is a visual character used in lesson videos to convey educational content in a visually easy-to-understand way.

[0144] A "speech generation engine" is a technology that converts text into natural-sounding speech and uses it to provide audio explanations in educational videos.

[0145] An "AI model" is a technology that uses machine learning algorithms to perform tasks such as analyzing progress data, automatically grading tests, and generating answers to questions.

[0146] This invention relates to an AI-powered online learning system that tracks learners' progress and provides individually optimized educational resources. This system primarily consists of three components: a server, a terminal, and a user (learner). Specific embodiments are described below.

[0147] Tracking learning progress

[0148] server

[0149] The server collects data such as learners' learning activities, test results, and assignment submission status, and records it in a database. The server also uses AI models (e.g., PyTorch or Tensorflow®) to analyze this data and understand the learners' progress. For example, when a learner watches a math lesson video and then submits an assignment, the device automatically sends this information to the server, which then updates the progress status and saves it in the database.

[0150] terminal

[0151] When a user (learner) logs in, the device sends authentication information to the server. If authentication is successful, the learning dashboard sent from the server is displayed. This dashboard displays the learner's progress, the status of submitted assignments, and other information.

[0152] Generation and distribution of lesson videos

[0153] server

[0154] The server uses an AI model based on learner progress data to generate individually optimized lesson videos. These lesson videos are visually easy to understand using avatars (e.g., Vyond) and include voice explanations using a speech generation engine (e.g., Google® Text-to-Speech). For example, if a learner struggles with "quadratic equations," the server generates a lesson video specifically tailored to quadratic equations based on past data, adds voice explanations, and then delivers it to the device.

[0155] terminal

[0156] The terminal displays lesson videos received from the server to the learner. By watching these videos, learners can receive personalized instruction.

[0157] Generation and grading of quizzes and regular tests

[0158] server

[0159] The server automatically generates appropriate quizzes or periodic tests based on learning progress data. This allows for effective assessment of learners' understanding. For example, if a learner wants to take a test on "quadratic equations," the server uses an AI model to automatically generate the questions and delivers them to the device. Once the learner enters their answers and the device sends them to the server, the server uses an OCR tool (e.g., Tesseract) to convert the handwritten answers into digital data and automatically grades them.

[0160] terminal

[0161] The terminal displays test questions sent from the server and provides an interface for learners to input their answers. Once the learner completes the test, the terminal sends the answers to the server, and the server displays the graded results in real time.

[0162] Question and Answer Session

[0163] server

[0164] The server has an AI model installed for question-and-answer sessions (for example, OpenAI® GPT-3®). When a learner submits a question, the server analyzes the question and generates the most appropriate answer. For example, if a learner enters a question about "how to solve an equation," the server analyzes the question with the AI ​​model, generates an appropriate answer, and provides it to the learner through their terminal.

[0165] terminal

[0166] When a learner enters a question using the terminal's question-and-answer interface, the terminal sends the question to the server. The answer provided by the server is then displayed to the learner through the terminal.

[0167] Provision of teaching materials and learning plans

[0168] server

[0169] The server generates individually optimized learning materials and study plans based on learners' progress data. These study plans include which subjects to focus on and recommended study times. For example, the server analyzes each learner's progress data and generates a study plan that includes subjects to focus on and recommended study times for the following week.

[0170] terminal

[0171] The terminal displays learning materials and study plans received from the server to the learner. This allows the learner to study efficiently.

[0172] Example of a prompt

[0173] "Generate answers to questions from learners about how to solve equations."

[0174] Please explain the process for creating instructional videos on quadratic equations.

[0175] "Please explain how to generate a learning plan based on learner progress data."

[0176] The above describes the embodiment of this invention. This system efficiently provides individually optimized educational resources and enables flexible instruction tailored to the learner's progress. Furthermore, the automation of test generation and scoring allows for rapid and accurate learning assessment. Real-time question and answer allows learners' questions to be resolved immediately.

[0177] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0178] Step 1:

[0179] The user (learner) logs into the device.

[0180] Input: Enter your User ID and password on the device's login screen.

[0181] Action: The terminal sends the entered authentication information to the server.

[0182] Output: The server performs login authentication and returns the authentication result to the terminal.

[0183] Step 2:

[0184] The server authenticates the learner.

[0185] Input: User ID and password sent from the terminal.

[0186] Operation: The server compares the user information in the database and performs authentication.

[0187] Output: If authentication is successful, a login success response is sent to the device, and data is sent to display the learning dashboard.

[0188] Step 3:

[0189] Learners engage in learning activities (watching videos, submitting assignments, etc.).

[0190] Input: The user (learner) selects and watches a lesson video, or enters their answers to an assignment.

[0191] Operation: The device collects learner activity data (viewing history, assignment submission information, etc.).

[0192] Output: The device sends the collected learning activity data to the server.

[0193] Step 4:

[0194] The device sends the training data to the server.

[0195] Input: Learner's video viewing history and assignment submission information.

[0196] Operation: The device automatically sends this data to the server.

[0197] Output: The server receives the learning activity data.

[0198] Step 5:

[0199] The server saves the data to the database.

[0200] Input: Learning activity data sent from the device.

[0201] Operation: The server records and updates data in the database.

[0202] Output: Progress data in the database is updated.

[0203] Step 6:

[0204] The server uses an AI model to analyze the data and update the progress status.

[0205] Input: Learning activity data and progress data from the database.

[0206] Operation: The server uses an AI model (e.g., PyTorch or TensorFlow) to perform data analysis.

[0207] Output: As a result of the analysis, the individual learner's progress is updated and reflected in the dashboard.

[0208] Step 7:

[0209] The server optimizes the content of the lesson videos based on the learner's progress data.

[0210] Input: Learner progress data from the database.

[0211] Operation: The server uses an AI model (e.g., TensorFlow) to determine the optimal content for the lesson videos.

[0212] Output: The content of the generated lesson video (for example, content specifically focused on quadratic equations).

[0213] Step 8:

[0214] The server generates lesson videos using avatars and a voice generation engine.

[0215] Input: Optimized lesson video content.

[0216] Operation: The server generates a visual avatar using an animation tool (e.g., Vyond) and adds a voice description using a speech generation engine (e.g., Google Text-to-Speech).

[0217] Output: Generated lesson video.

[0218] Step 9:

[0219] The server generates and delivers lesson videos to the devices.

[0220] Input: Generated lesson video.

[0221] Operation: The server streams the lesson videos to the terminals.

[0222] Output: Lecture video delivered to the device.

[0223] Step 10:

[0224] The device displays the video to the learner.

[0225] Input: Lecture videos streamed from the server.

[0226] Operation: The device plays the lesson video and allows the learner to watch it.

[0227] Output: Learners watch the lesson videos.

[0228] Step 11:

[0229] The server automatically generates quizzes or periodic tests based on learning progress data.

[0230] Input: Learner progress data.

[0231] Operation: The server uses an AI model (e.g., SciKit-Learn) to automatically generate appropriate test questions.

[0232] Output: Generated test questions.

[0233] Step 12:

[0234] The server delivers the test questions to the terminals.

[0235] Input: Generated test questions.

[0236] Operation: The server sends the test questions to the terminal.

[0237] Output: Test questions delivered to the terminal.

[0238] Step 13:

[0239] The learners answer the test questions.

[0240] Input: Test question.

[0241] Operation: The user (learner) answers test questions on the device and inputs the answers into the device.

[0242] Output: Learner's answer.

[0243] Step 14:

[0244] The device sends the answer to the server.

[0245] Input: Learner's answer.

[0246] Operation: The device sends the collected answer data to the server.

[0247] Output: Answer data sent to the server.

[0248] Step 15:

[0249] The server digitizes the answers using optical character recognition (OCR).

[0250] Input: Submitted answer data.

[0251] Operation: The server uses an OCR tool (e.g., Tesseract) to convert handwritten answers into digital data.

[0252] Output: Digitized answer data.

[0253] Step 16:

[0254] The server uses an AI model to automatically score the results.

[0255] Input: Digitized answer data.

[0256] Operation: The server uses an AI model (e.g., SciKit-Learn) to automatically grade the answers.

[0257] Output: Scoring results.

[0258] Step 17:

[0259] The server delivers the scoring results to the terminal.

[0260] Input: Scoring result.

[0261] Operation: The server sends the scoring results to the terminal in real time.

[0262] Output: The scoring result displayed on the device.

[0263] Step 18:

[0264] The learner enters the question from their device.

[0265] Input: Learner's question.

[0266] Operation: The user (learner) enters a question using a question-and-answer interface.

[0267] Output: Question data sent from the terminal to the server.

[0268] Step 19:

[0269] The terminal sends the question to the server.

[0270] Input: Question data.

[0271] Operation: The terminal sends the question data to the server.

[0272] Output: Question data received by the server.

[0273] Step 20:

[0274] The server analyzes the question.

[0275] Input: Received question data.

[0276] Operation: The server uses an AI model (e.g., OpenAI GPT-3) to analyze the question.

[0277] Output: Analysis results.

[0278] Step 21:

[0279] The server generates the answer.

[0280] Input: Analysis results of the question.

[0281] Action: The server generates an optimal answer.

[0282] Output: The generated answer.

[0283] Step 22:

[0284] The server sends the answer to the terminal.

[0285] Input: The generated answer.

[0286] Action: The server sends the answer to the terminal.

[0287] Output: The answer received by the terminal.

[0288] Step 23:

[0289] The terminal displays the answer to the learner.

[0290] Input: The answer sent from the server.

[0291] Action: The terminal displays the answer to the learner.

[0292] Output: The learner views the answer.

[0293] Step 24:

[0294] The server generates a learning plan based on the learner's progress data.

[0295] Input: The learner's progress data.

[0296] Action: The server uses an AI model to generate an individually optimized learning plan.

[0297] Output: The generated learning plan.

[0298] Step 25:

[0299] The server distributes the learning plan to the terminal.

[0300] Input: The generated learning plan.

[0301] Operation: The server sends the learning plan to the terminal.

[0302] Output: The learning plan displayed on the terminal.

[0303] The above is the specific processing flow of this system. By performing specific operations in each step, it becomes possible to efficiently provide individually optimized educational resources and realize flexible guidance according to the progress of the learner.

[0304] (Application Example 1)

[0305] Next, Application Example 1 will be described. In the following description, the data processing device 12 is referred to as the "server", and the smart device 14 is referred to as the "terminal".

[0306] In the conventional online learning system, it was difficult to effectively track the progress of learners and provide individually optimized educational resources. Also, there was a lack of means to realize a more intuitive and immersive learning environment by providing a learning experience in a virtual environment. Furthermore, it was difficult to automatically generate and immediately evaluate optimal mini-tests and regular tests based on the progress of learners.

[0307] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following respective means.

[0308] In this invention, the server includes means for tracking learner progress data, means for generating optimized lesson content based on the progress data, means for delivering the lesson content to learners, means for automatically generating quizzes or periodic tests for learners in a virtual environment, means for optical character recognition processing of the test answers, means for automatically scoring the optical character recognition processed answers, means for generating appropriate answers to learners' questions, means for providing learners with optimal learning resources and learning plans, and a learning experience in a virtual environment. This makes it possible to grasp learners' progress in real time and provide individually optimized educational resources. Furthermore, a more intuitive and immersive learning experience can be achieved through the learning experience in the virtual environment.

[0309] "Learner progress data" refers to data generated when learners engage in learning activities, and includes information such as the content of lessons viewed, submitted assignments, and test results.

[0310] "Lesson content" refers to learning materials provided to learners, including learning resources in formats such as videos, audio, and text.

[0311] A "virtual environment" refers to a virtual learning space created using computer simulations, where learners can engage in learning activities while immersed in the environment, just as they would in the real world.

[0312] A "quiz or periodic test" is an examination administered periodically to measure a learner's understanding of the material, and includes both small-scale review tests and tests that assess long-term learning outcomes.

[0313] Optical Character Recognition (OCR) is a technology that reads text information from scanned documents and images, and is a process for converting them into a digital format.

[0314] "Automated scoring" refers to the process of using AI technology to mechanically evaluate test answers and assign scores.

[0315] "Means for generating appropriate answers" refers to technology in which AI automatically generates the most effective answers based on the learner's questions.

[0316] "Learning resources" refer to educational content such as textbooks, videos, and audio guides that learners use to study efficiently.

[0317] A "learning plan" is a schedule or guideline that shows how learners should proceed with their studies within a certain period of time.

[0318] This invention relates to an AI-powered online learning system that tracks learners' progress and provides individually optimized educational resources within a virtual environment based on that progress. Specific embodiments are described below.

[0319] Tracking learning progress

[0320] server

[0321] The server collects data such as learners' learning activities, test results, and assignment submission status, and records it in a database. It also uses a generative AI model to analyze this data and understand the learners' progress. This allows for the provision of instruction tailored to each individual learner.

[0322] terminal

[0323] When a learner logs into the virtual environment, the device sends authentication information to the server. If authentication is successful, the learning dashboard sent from the server is displayed within the virtual environment. This dashboard displays the learner's progress, the status of submitted assignments, and other information.

[0324] Generation and distribution of course content

[0325] server

[0326] The server uses a generative AI model to generate individually optimized lesson content based on the learner's progress data. This lesson content is visually easy to understand using avatars and includes voice explanations using an artificial speech generation engine. Once the lesson content is generated, the server delivers it to terminals within the virtual environment.

[0327] terminal

[0328] The terminal displays lesson content received from the server to the learner within a virtual environment. By viewing this content, learners can receive individually optimized learning guidance.

[0329] Generation and grading of quizzes and regular tests

[0330] server

[0331] The server automatically generates appropriate quizzes or periodic tests based on learning progress data. This allows for effective assessment of learners' understanding. When learners take a test, the server processes the submitted answers using optical character recognition and automatically grades them using a generative AI model.

[0332] terminal

[0333] The terminal displays test questions sent from the server within a virtual environment and provides an interface for learners to input their answers. Once the learner completes the test, the terminal sends the answers to the server, and the server displays the graded results in real time.

[0334] Question and Answer Session

[0335] server

[0336] The server has a generative AI model installed for question and answering. When a learner submits a question, the server analyzes the question and generates the best possible answer.

[0337] terminal

[0338] When a learner enters a question using the terminal's question-and-answer interface, the terminal sends the question to the server. The answer provided by the server is then displayed to the learner within the virtual environment via the terminal.

[0339] Provision of teaching materials and learning plans

[0340] server

[0341] The server generates personalized learning materials and study plans based on the learner's progress data. These plans include which subjects to focus on and recommended study times.

[0342] terminal

[0343] The terminal displays learning materials and study plans received from the server to the learner within a virtual environment. This allows learners to study efficiently.

[0344] Specific example

[0345] Example of a learning progress tracking prompt message

[0346] Tracking learning progress

[0347] When a learner watches math lesson content and then submits an assignment, the device automatically sends this information to the server. The server uses this data to update the learner's progress and saves it in a database.

[0348] Generation and distribution of course content

[0349] If a learner struggles with the topic of "quadratic equations," the server generates specialized lesson content based on past data, adds an artificial voice explanation, and then delivers it to the terminal in the virtual environment.

[0350] Examples of prompt messages for generating quizzes and periodic tests.

[0351] Please generate a short quiz on "quadratic equations" for learner ID: 12345.

[0352] Examples of question-and-answer prompts

[0353] Learner ID: 12345 has a question about "How to solve equations". Please generate the best answer.

[0354] Provision of teaching materials and learning plans

[0355] The server analyzes each learner's progress data and generates a study plan that includes subjects to focus on in the following week and recommended study time. The terminal displays this plan to the learner within the virtual environment.

[0356] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0357] Step 1:

[0358] The user logs into the virtual environment.

[0359] Input: User ID, Password

[0360] Operation: Enter your user ID and password, and the terminal will send them to the server.

[0361] Output: Authentication result

[0362] Specific operation: The server compares user information with the database and returns whether authentication was successful or not.

[0363] Step 2:

[0364] The device displays the learner's learning dashboard.

[0365] Input: Authentication result, User ID

[0366] Operation: If authentication is successful, learner progress data is retrieved from the server and a learning dashboard is generated.

[0367] Output: Learning Dashboard

[0368] Specific operation: The dashboard displays learning progress, viewed content, submitted assignments, and more.

[0369] Step 3:

[0370] The server generates optimized lesson content.

[0371] Input: Learner progress data

[0372] Operation: Uses a generative AI model to analyze learner progress data and generate optimal lesson content.

[0373] Output: Course content

[0374] Specific actions: The lesson content will include avatars and artificial voice explanations.

[0375] Step 4:

[0376] The device displays the lesson content within a virtual environment.

[0377] Input: Lesson content

[0378] Operation: Displays lecture content received from the server through a screen or avatar in a virtual environment.

[0379] Output: Lesson content viewable by learners

[0380] Specific actions: Learners watch lesson content and complete related assignments.

[0381] Step 5:

[0382] The server automatically generates short quizzes or periodic tests.

[0383] Input: Learner progress data, test requests

[0384] Operation: Generates appropriate test questions using a generative AI model.

[0385] Output: Test questions

[0386] Specific operation: The AI ​​automatically generates the most suitable problems based on progress data and provides them in the form of tests.

[0387] Step 6:

[0388] The terminal displays the test questions within the virtual environment.

[0389] Input: Test Questions

[0390] Operation: Displays test questions received from the server on the interface and accepts answer input.

[0391] Output: Learner's answer

[0392] Specific actions: The learner takes a test and enters their answers.

[0393] Step 7:

[0394] The server processes the learner's answers using optical character recognition (OCR).

[0395] Input: Learner's answer

[0396] Operation: Performs optical character recognition processing to convert answers from handwritten text or images into digital data.

[0397] Output: Digital answer data

[0398] Specific operation: The AI ​​model recognizes the answer and converts it into a format that can be stored in the database.

[0399] Step 8:

[0400] The server automatically scores the answers that have been processed using optical character recognition.

[0401] Input: Digital answer data

[0402] Operation: Use a generative AI model to score the answers and calculate the results.

[0403] Output: Scoring results

[0404] Specific operation: A score is assigned to each answer based on the scoring algorithm.

[0405] Step 9:

[0406] The device displays the scoring results in real time.

[0407] Input: Scoring result

[0408] Operation: Displays the scoring results received from the server on the dashboard.

[0409] Output: Scoring results displayed to learners

[0410] Specific operation: Scoring results are displayed on the screen in real time, allowing learners to check them.

[0411] Step 10:

[0412] The server manages the Q&A session.

[0413] Input: Learner's question

[0414] Operation: Uses a generative AI model to analyze the question and generate an appropriate answer.

[0415] Output: Answer text

[0416] Specific operation: The AI ​​receives a question from the learner, analyzes it, generates an answer, and sends it back.

[0417] Step 11:

[0418] The device displays the answers to the questions and answers.

[0419] Input: Answer text

[0420] Operation: Displays the response provided by the server within the virtual environment.

[0421] Output: Answers that learners can see.

[0422] Specific action: Learners check their answers via their devices and deepen their understanding.

[0423] Step 12:

[0424] The server generates individually optimized learning materials and study plans for each learner.

[0425] Input: Learner progress data

[0426] Operation: Uses a generative AI model to analyze data and create new learning plans and materials.

[0427] Output: Individually optimized learning materials and study plans

[0428] Specific action: A study plan is created, including recommended subjects and study time for the following week.

[0429] Step 13:

[0430] The device displays individually optimized learning materials and study plans.

[0431] Input: Course materials and study plans

[0432] Operation: Displays the learning plan and materials received from the server within the virtual environment.

[0433] Output: Learning plan and materials presented to the learner

[0434] Specific operation: The learning dashboard displays the learning plan and recommended materials for the following week, allowing learners to proceed with their studies accordingly.

[0435] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0436] This invention combines an AI-powered online learning system that tracks learners' progress and provides individually optimized educational resources with an emotion engine that recognizes learners' emotions. The system consists of a server, a terminal, and a user (learner), each performing a specific function.

[0437] Tracking learning progress

[0438] server

[0439] The server collects data such as learners' learning activities, test results, and assignment submission status, and records it in a database. An AI model is used to analyze this data and understand the learners' progress. The system is configured to provide optimal instruction based on the learners' progress.

[0440] terminal

[0441] When a learner logs in, the device sends authentication information to the server. If authentication is successful, the learning dashboard sent from the server is displayed. This dashboard shows the learner's progress, the status of submitted assignments, and other information.

[0442] Generation and distribution of lesson videos

[0443] server

[0444] The server uses an AI model to generate individually optimized lesson videos based on the learner's progress data. The lesson videos are visually easy to understand using avatars, and audio explanations are added using a speech generation engine. The generated lesson videos are stored on the server and delivered to the user's device.

[0445] terminal

[0446] The terminal displays lesson videos received from the server to the learner. Learners can watch these videos and receive personalized instruction.

[0447] Learners' perception and response to emotions

[0448] server

[0449] The server is equipped with an emotion engine that recognizes not only the learner's progress data but also their emotional state. The emotion engine analyzes the learner's facial expression data and voice data to recognize their emotional state in real time.

[0450] terminal

[0451] The device collects the learner's facial expressions and voice through its camera and microphone and transmits them to the server. The server analyzes this data to understand the learner's emotional state.

[0452] server

[0453] The server adjusts the content and difficulty level of the lesson videos in real time based on the recognized emotional state. For example, if a learner is feeling stressed, the server can increase the amount of explanations in a gentle tone or change the content to a lower difficulty level.

[0454] Generation and grading of quizzes and regular tests

[0455] server

[0456] The server automatically generates appropriate test questions using an AI model based on learning progress data and sentiment data. When a learner takes the test, the server processes the submitted answers using OCR and automatically scores them using the AI ​​model.

[0457] terminal

[0458] The terminal displays test questions sent from the server, and the learner enters their answers. Once the test is completed, the answers are sent to the server, and the scoring results are immediately fed back.

[0459] Question and Answer Session

[0460] server

[0461] The server is equipped with an AI model for question-and-answer sessions, which generates the most appropriate answers to learners' questions. An emotion engine allows it to understand the learner's emotional state and provide answers at the right time.

[0462] terminal

[0463] When a learner enters a question using the terminal's question-and-answer interface, the terminal sends the question to the server. The server generates an answer, which is then displayed to the learner through the terminal.

[0464] Provision of teaching materials and learning plans

[0465] server

[0466] The server generates individually optimized learning materials and study plans based on the learner's progress and sentiment data. An AI model calculates the next learning objectives and recommended study time, and then creates the learning materials and study plans.

[0467] terminal

[0468] The terminal supports efficient learning by displaying learning materials and study plans received from the server to the learner.

[0469] Specific example

[0470] Tracking learning progress

[0471] When a user (learner) watches a math lesson video and then submits an assignment, the device automatically sends this information to the server. The server uses this data to update the learner's progress and saves it in a database.

[0472] Generation and distribution of lesson videos

[0473] If a user (learner) struggles with the topic of "quadratic equations," the server generates a specialized lesson video on quadratic equations based on past data, adds audio explanations, and then delivers it to the user's device.

[0474] Learners' perception and response to emotions

[0475] If a user (learner) is experiencing stress during a lesson, the device's camera detects this and notifies the server. The server's emotion engine confirms the high level of stress and adjusts the lesson video content to help the learner relax.

[0476] Generation and grading of quizzes and regular tests

[0477] If a user (learner) wants to take a test on quadratic equations, the server uses an AI model to automatically generate questions and delivers them to the device. When the learner enters their answers and the device sends them to the server, the server uses OCR to digitize the answers and the AI ​​model to grade them.

[0478] Question and Answer Session

[0479] When a user (learner) enters a question about "how to solve an equation," the device sends the question to the server. The server analyzes the question using an AI model, generates an appropriate answer, and provides it to the learner through the device.

[0480] Provision of teaching materials and learning plans

[0481] The server analyzes each learner's progress and sentiment data to generate a study plan that includes subjects to focus on in the following week and recommended study time. The terminal displays this plan to the learner.

[0482] The above describes the embodiment of this invention. This system makes it possible to provide an optimal learning experience for each individual learner. By combining it with an emotional engine, the efficiency of learning can be further improved.

[0483] The following describes the processing flow.

[0484] Tracking learning progress

[0485] Step 1:

[0486] The user (learner) logs into the online learning platform using their device.

[0487] The terminal sends the user's authentication information to the server.

[0488] Step 2:

[0489] The server verifies the authentication information, and if authentication is successful, it generates a learning dashboard and sends it to the device.

[0490] The device displays the received dashboard, showing the user their current learning progress.

[0491] Step 3:

[0492] Users (learners) watch lesson videos, take tests, and submit assignments.

[0493] The device records these learning activities and sends them to the server in real time.

[0494] Step 4:

[0495] The server records the received training data in a database and performs analysis using an AI model.

[0496] The learner's progress is updated based on the analysis results.

[0497] Generation and distribution of lesson videos

[0498] Step 1:

[0499] The server starts generating individually optimized lesson videos based on the learner's progress data.

[0500] The AI ​​model selects the appropriate content and activates the avatar and voice generation engine.

[0501] Step 2:

[0502] The server generates visually easy-to-understand videos using avatars and adds narration using a voice generation engine.

[0503] The generated lesson videos are saved on the server.

[0504] Step 3:

[0505] The server sends the generated lesson videos to the terminal.

[0506] The device displays the received video to the user (learner) and begins playback.

[0507] Learners' perception and response to emotions

[0508] Step 1:

[0509] While the user (learner) is watching the lesson video, the device uses its camera and microphone to collect the learner's facial expression data and voice data.

[0510] Step 2:

[0511] The device sends the collected emotional data to the server in real time.

[0512] The server analyzes this data to recognize the learner's emotional state.

[0513] Step 3:

[0514] The server adjusts the content and difficulty level of the lesson videos in real time based on the emotional state it recognizes.

[0515] For example, if a learner is experiencing stress, the content can be changed to a less difficult version.

[0516] Generation and grading of quizzes and regular tests

[0517] Step 1:

[0518] The server automatically generates appropriate test questions using an AI model based on learning progress data and sentiment data.

[0519] The generated tests are saved in digital format.

[0520] Step 2:

[0521] The server sends the generated test questions to the terminal.

[0522] The user (learner) takes the test through their device.

[0523] Step 3:

[0524] The user (learner) enters their answer and sends it to the server using their device.

[0525] The answers will be formatted appropriately for OCR processing.

[0526] Step 4:

[0527] The server digitizes the answers using an OCR engine and performs automatic scoring using an AI model.

[0528] The scoring results are saved in a database and immediately sent to the device as feedback.

[0529] Step 5:

[0530] The device displays the scoring results to the user (learner) and suggests the next learning steps.

[0531] Question and Answer Session

[0532] Step 1:

[0533] The user (learner) enters their question using the terminal's question-and-answer interface.

[0534] The terminal sends the question to the server.

[0535] Step 2:

[0536] The server receives the question and analyzes its content using an AI model.

[0537] The system generates the optimal response based on the analysis results.

[0538] Step 3:

[0539] The server sends the generated response to the terminal.

[0540] The device displays the answer to the user (learner).

[0541] Provision of teaching materials and learning plans

[0542] Step 1:

[0543] The server generates individually optimized learning materials and study plans based on the learner's progress data and emotional data.

[0544] The AI ​​model calculates the next learning goal and recommended learning time.

[0545] Step 2:

[0546] The server sends the generated learning materials and study plans to the terminal.

[0547] The device displays the received plan and learning materials to the user (learner).

[0548] The above describes the specific operations at each processing step of the system. This system makes it possible to provide an optimal learning experience for each individual learner. By combining it with an emotion engine, the efficiency of learning can be further improved.

[0549] (Example 2)

[0550] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0551] Traditional online learning systems have struggled to monitor learners' progress and emotional states in real time and provide personalized learning resources accordingly. Furthermore, the lack of features to adjust instruction based on learners' emotional states made it difficult to maintain motivation and maximize learning effectiveness. There were also challenges with the generation of quizzes and periodic tests, automated grading, and the quality of question-and-answer sessions, highlighting the need for a system that provides optimal education for each individual learner.

[0552] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for tracking learner progress data, means for generating optimized lesson content based on the progress data, means for delivering the lesson content to learners, means for collecting learner facial expression data and voice data, means for analyzing the facial expression data and voice data to recognize the learner's emotional state, means for adjusting the lesson content based on the emotional state, means for automatically generating quizzes or periodic tests for learners, means for character recognition processing of the test answers, means for automatically scoring the character recognition processed answers, means for generating appropriate answers to learners' questions, and means for providing learners with optimal learning materials and learning plans. This makes it possible to grasp the learner's progress and emotional state in real time and provide optimal learning resources.

[0553] A "learner" refers to a person who uses an online learning system to study educational content.

[0554] "Progress data" refers to information recorded in digital format, such as learners' learning activities, test results, and assignment submission status.

[0555] "Lesson content" refers to educational materials provided to learners, including in the form of text, videos, and audio.

[0556] "Distribution" refers to the act of sending data in digital format from a server to a learner's device.

[0557] "Facial expression data" refers to digital information about a learner's facial expressions acquired through a camera or other recording device.

[0558] "Audio data" refers to digital information about learners' voices acquired through audio collection devices such as microphones.

[0559] "Emotional state" refers to the learner's psychological and emotional state as identified as a result of the analysis of facial expression data and voice data.

[0560] "Character recognition processing" refers to the process of analyzing characters written on paper or in image data as digital data and converting them into text information.

[0561] "Automated scoring" refers to the process of using artificial intelligence models to mechanically evaluate test answers and assign scores.

[0562] "Question answering" refers to the act of a system generating and providing appropriate answers to questions from learners.

[0563] "Educational materials" refer to educational resources provided to learners, and include forms such as textbooks, videos, and practice exercises.

[0564] A "learning plan" refers to a plan created to present learners with the most suitable learning schedule and achievement goals.

[0565] A "virtual character" is an animated character created in a digital environment, and its role is to visually explain and instruct on lesson content.

[0566] "Speech generation means" refers to technology that converts text information into speech and uses it for narration and explanations.

[0567] An "artificial intelligence model" refers to an algorithm that uses machine learning or deep learning to perform data analysis and prediction.

[0568] This invention combines an AI-powered online learning system that tracks learners' progress and provides individually optimized educational resources with an emotion engine that recognizes learners' emotions. The system consists of a server, terminals, and users (learners), each performing a specific function.

[0569] Tracking learning progress

[0570] server

[0571] The server collects data such as learners' learning activities, test results, and assignment submission status, and records it in a database. An AI model implemented in Python is used to analyze this data and evaluate the learners' progress. Based on the results of this analysis, individually optimized instructional content is generated.

[0572] terminal

[0573] When a learner logs in, the device sends authentication information to the server. If authentication is successful, the learning dashboard sent from the server is displayed. This dashboard displays learning progress, the status of submitted assignments, and other information in real time.

[0574] Generation and distribution of lesson videos

[0575] server

[0576] The server generates individually optimized lesson videos using a generative AI model based on the learner's progress data. These lesson videos are created to be visually and aurally easy to understand using virtual characters and voice generation methods (e.g., Google Text-to-Speech API). The generated videos are stored on the server and delivered to the device.

[0577] terminal

[0578] The terminal displays lesson videos received from the server to the learner. The learner can watch these videos and receive individually optimized instruction.

[0579] Learners' perception and response to emotions

[0580] terminal

[0581] While learners are watching the lesson videos, the device's camera and microphone collect facial expression and voice data from the learners and send it to the server.

[0582] server

[0583] The server uses Microsoft® Azure® Face API and Amazon Alexa Voice Service to analyze this data with an emotion engine and recognize the learner's emotional state. Based on the recognition results, it adjusts the content and difficulty level of the lesson videos in real time.

[0584] Generation and grading of quizzes and regular tests

[0585] server

[0586] The server automatically generates appropriate test questions using a generative AI model based on learning progress data and sentiment data. Once a test is submitted, the answers are digitized using an OCR tool (e.g., Tesseract OCR), and the AI ​​model automatically scores them.

[0587] terminal

[0588] The terminal displays test questions sent from the server, and the learner enters their answers. After the test is completed, the answers are sent to the server, and the results are immediately fed back.

[0589] Question and Answer Session

[0590] terminal

[0591] The learner enters a question using a question-and-answer interface, and the device sends the question to the server.

[0592] server

[0593] The server uses OpenAI GPT-4® to analyze the question and generate an appropriate answer. The generated answer is sent to the terminal and displayed to the learner.

[0594] Provision of teaching materials and learning plans

[0595] server

[0596] The server analyzes progress and sentiment data to generate personalized learning materials and study plans, including next learning goals and recommended study times. An AI model then suggests the optimal learning strategy.

[0597] terminal

[0598] The terminal displays learning materials and study plans received from the server to the learner, supporting efficient learning progress.

[0599] Specific example

[0600] Tracking learning progress

[0601] Specific example: When a user (learner) watches a math lesson video and then submits an assignment, the device automatically sends this information to the server. The server updates the learner's progress based on this data and saves it in a database.

[0602] Generation and distribution of lesson videos

[0603] Specific example: If a user (learner) struggles with the topic of "quadratic equations," the server generates a video lesson specifically focused on quadratic equations based on past data, adds audio explanations, and then delivers it to the user's device.

[0604] Learners' perception and response to emotions

[0605] Specific example: If a user (learner) experiences stress during a lesson, the device's camera detects this and notifies the server. The server then uses an emotion engine to analyze the stress and adjust the lesson video and teaching content accordingly.

[0606] Generation and grading of quizzes and regular tests

[0607] Specific example: If a user (learner) wants to take a test on "quadratic equations," the server automatically generates questions using a generative AI model and delivers them to the device. When the learner enters their answer and the device sends it to the server, the server uses OCR to digitize the answer and the AI ​​model scores it.

[0608] Question and Answer Session

[0609] Specific example: When a user (learner) inputs a question about "how to solve an equation," the device sends the question to the server. The server analyzes the question using an AI model, generates an appropriate answer, and sends it back to the device.

[0610] Provision of teaching materials and learning plans

[0611] Specific example: The server analyzes each learner's progress and sentiment data and generates a study plan that includes subjects to focus on in the following week and recommended study time. The terminal displays this plan to the learner.

[0612] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0613] Step 1: Learner Authentication and Login

[0614] ---

[0615] Input: Username, Password

[0616] Output: Authentication token (success), error message (failure)

[0617] terminal

[0618] The user accesses the system and enters their username and password on the login screen. The entered authentication information is sent from the terminal to the server.

[0619] server

[0620] The server compares the received authentication information with the database. If authentication is successful, it generates a session ID and sends the learning dashboard to the device. If authentication fails, it sends an error message to the device.

[0621] Specific operation: The server checks if the username and password are correct. If they are correct, it returns the learning dashboard; otherwise, it returns an error message.

[0622] Step 2: Tracking Learning Progress

[0623] ---

[0624] Input: Learning activity data, test results, assignment submission status

[0625] Output: Updated progress status, analysis results

[0626] terminal

[0627] The user opens the learning dashboard to check their progress and the status of submitted assignments. The learning progress data is then sent to the server.

[0628] server

[0629] The server receives new learning progress data and records it in the database. An AI model implemented in Python is used to analyze the data and evaluate the learner's progress. The analysis results are reflected in the learning dashboard in real time.

[0630] Specific operation: Learning activities and test results are periodically sent to the server, analyzed by an AI model, and the user's learning progress is evaluated and recorded.

[0631] Step 3: Generate and distribute lesson videos

[0632] ---

[0633] Input: Progress data

[0634] Output: Lecture video URL, video content

[0635] server

[0636] The server generates individually optimized lesson videos using a generative AI model based on the learner's progress data. It uses virtual characters and voice generation methods (e.g., Google Text-to-Speech API) to create visually and aurally easy-to-understand lesson videos. The generated videos are saved on the server, and the video URL is sent to the user's device.

[0637] terminal

[0638] The device displays the lesson video using a video player based on the URL received from the server. The user then watches this video to progress with their learning.

[0639] Specific operation: Input learning progress data into the AI ​​model and deliver the generated video to the device.

[0640] Step 4: Learner's emotional recognition and response

[0641] ---

[0642] Input: Facial expression data, audio data

[0643] Output: Emotion recognition results, adjusted lesson content

[0644] terminal

[0645] While learners are watching the lesson videos, the device's camera and microphone collect facial expression and voice data from the learners and send it to the server.

[0646] server

[0647] The server uses the Microsoft Azure Face API and Amazon Alexa Voice Service to analyze this data with an emotion engine and recognize the learner's emotional state. Based on the recognition results, it adjusts the content and difficulty level of the lesson videos in real time.

[0648] Specific operation: Collected facial expressions and audio data are sent to a server, and the lesson content is adaptively modified based on the recognition results.

[0649] Step 5: Generating and grading quizzes and periodic tests

[0650] ---

[0651] Input: Progress data, sentiment data

[0652] Output: Test questions, scoring results

[0653] server

[0654] The server automatically generates appropriate test questions using an AI model based on learning progress data and sentiment data. The generated test questions are then sent to the device.

[0655] terminal

[0656] The terminal displays the test questions, and the learner enters their answers. The answers are then sent to the server.

[0657] server

[0658] The server uses an OCR tool (e.g., Tesseract OCR) to digitize the answers and an AI model to score them. The results are then fed back to the terminal.

[0659] Specific operation: Generate test questions based on data, grade learners' answers using an AI model, and return the results.

[0660] Step 6: Handling questions and answers

[0661] ---

[0662] Input: Learner's question

[0663] Output: Answer

[0664] terminal

[0665] The user enters a question using a question-and-answer interface, and the terminal sends it to the server.

[0666] server

[0667] The server uses OpenAI GPT-4 to analyze the question and generate an appropriate answer. The generated answer is then sent to the terminal.

[0668] Specific operation: The system receives a question, sends it to the server, generates an answer using a generative AI model, and sends it back to the terminal.

[0669] Step 7: Providing learning materials and study plans

[0670] ---

[0671] Input: Progress data, sentiment data

[0672] Output: Individually optimized learning materials and study plans

[0673] server

[0674] The server analyzes progress and sentiment data to generate personalized learning materials and study plans, including next learning goals and recommended study times. An AI model suggests the optimal learning strategy. The generated learning materials and study plans are then sent to the user's device.

[0675] terminal

[0676] The device displays received learning materials and study plans on a learning dashboard, supporting users in efficiently progressing through their studies.

[0677] Specific actions: Analyze data to create optimal learning materials and study plans, and present them to the user on a learning dashboard.

[0678] (Application Example 2)

[0679] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0680] Traditional online learning systems struggle to track learners' progress and provide optimal educational resources. Furthermore, the lack of features to recognize learners' emotional states and adjust instruction in real time limits the potential for reducing learner stress and improving learning effectiveness. Similarly, in factory settings, the absence of systems to track work progress and provide optimal training resources results in insufficient work efficiency and mental well-being.

[0681] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0682] In this invention, the server includes means for tracking learner (or worker) progress data, means for generating optimized lesson videos (or training videos) based on the progress data, and means for delivering the lesson videos (or training videos) to the learner (or worker). This makes it possible to grasp the learner's (or worker's) progress in real time and provide individually optimized educational resources (or training resources). Furthermore, the server includes means for recognizing the learner's (or worker's) emotional state and means for adjusting the instructional content (or training content) based on the emotional state, thereby reducing learner's (or worker's) stress and improving learning (or work) efficiency.

[0683] "Progress data" refers to data that shows the progress of learners' or workers' learning or work.

[0684] "Lesson videos" refer to educational video content provided to learners.

[0685] "Training videos" refer to video content provided to workers for training purposes.

[0686] "Automatic generation" means that the system automatically generates processes or results without requiring manual operation.

[0687] "OCR processing" is the process of digitizing handwritten or printed text using optical character recognition technology.

[0688] "Automated evaluation" refers to a system that automatically scores test answers and learning results.

[0689] "Generating an answer" means that the system creates an appropriate answer to the learner's question.

[0690] "Educational materials" refer to educational resources and content used by learners.

[0691] A "learning plan" is a plan or schedule designed to help learners progress through their studies efficiently.

[0692] "Emotional state" refers to the psychological feelings and moods of learners and workers.

[0693] "Instructional content" refers to the educational material and teaching methods provided to learners.

[0694] "Training content" refers to the training materials and instructional policies provided to workers.

[0695] This invention is a system that tracks the progress of learners and workers and provides individually optimized educational and training resources. Furthermore, it includes a function to recognize the emotional state of learners and workers in real time and adjust the instructional and training content based on that state.

[0696] System Configuration

[0697] The system primarily consists of servers, terminals, and users. Each element performs a specific function, enabling the provision of efficient and effective education and training.

[0698] server

[0699] The server forms the core of the system and implements the following functions:

[0700] 1. Tracking progress data

[0701] The server tracks the learning and work progress of learners and workers. The collected progress data is recorded in a database and analyzed using an AI model.

[0702] 2. Generation of optimized lesson videos and training videos

[0703] Using AI models, individually optimized video content is generated based on progress data. This includes the use of avatars and a voice generation engine to provide content that is easy to understand both visually and aurally.

[0704] 3. Recognition of emotional state

[0705] Equipped with an emotion recognition engine, it analyzes the user's facial expression and voice data to recognize their emotional state in real time. This information is sent to a server and used to adjust instruction and training content.

[0706] 4. Automatic generation and evaluation

[0707] Quiz and periodic tests for learners are automatically generated. Furthermore, submitted test answers are digitized using OCR processing and automatically graded using an AI model.

[0708] 5. Question Answering Function

[0709] It also features an AI model for question answering, which generates appropriate answers to questions entered by learners.

[0710] 6. Provision of teaching materials and learning plans

[0711] Based on progress data and sentiment data, the system generates and provides users with optimal learning materials and study plans.

[0712] terminal

[0713] A terminal is a device that the user directly operates and has the following functions:

[0714] 1. Login and Authentication

[0715] This is used when learners or workers log in to the system and sends authentication information to the server.

[0716] 2. Displaying the dashboard

[0717] You can view the dashboard sent from the server and check the progress, status of submitted assignments, and more.

[0718] 3. Playback of video content

[0719] The system displays lecture and training videos received from the server, allowing users to view them.

[0720] 4. Collection of facial expression and voice data

[0721] The system collects the user's facial expressions and voice through cameras and microphones and transmits them to a server.

[0722] User

[0723] Users utilize the system as learners or workers and perform the following activities:

[0724] 1. Progress of learning and work

[0725] Students progress by watching individually optimized lesson and training videos.

[0726] 2. Taking the test and answering questions

[0727] Students take quizzes and regular tests, and use a question-answering interface to enter questions.

[0728] Hardware and software used

[0729] Hardware: Webcam, smartphone, tablet, PC

[0730] Software: OpenCV (image processing library), TensorFlow / Keras (AI model implementation)

[0731] Specific example

[0732] For example, when a factory worker begins training, a terminal tracks the worker's progress. The server analyzes the collected data and generates and distributes optimized training videos. It also collects the worker's facial expressions and voice through the terminal's camera and microphone, and recognizes their emotional state in real time using an emotion recognition engine. If the worker is experiencing stress, the server adjusts the training content based on that emotional state to reduce the worker's stress.

[0733] Example of a prompt

[0734] Example: "Create a program that recognizes the emotional state of workers in real time and provides the optimal training resources. Use facial expression data captured by a camera to recognize emotions and analyze them with an AI model."

[0735] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0736] Step 1: User login and authentication

[0737] The user logs into the system using a device (smartphone, tablet, or PC). The device sends the user's authentication information to the server. The server verifies the authentication information and, if the login is successful, sends the learning dashboard to the device.

[0738] Input: User authentication information (username, password)

[0739] Output: Authentication result (learning dashboard if authentication is successful, error message if it fails)

[0740] Specific operation: When a user enters their authentication information on the login screen and clicks the "Login" button, that information is sent to the server. The server then checks the database to verify the authentication information.

[0741] Step 2: Display the learning dashboard

[0742] After successful authentication, the server generates a learning dashboard based on the user's progress data and sends it to the device. The device then displays this dashboard.

[0743] Input: User progress data

[0744] Output: Learning Dashboard

[0745] Specific operation: The server retrieves user progress data from the database and analyzes the user's learning status using an AI model. Based on the results, it generates a dashboard that displays progress, submitted assignments, next assignments, etc.

[0746] Step 3: Generate and distribute lesson videos (or training videos).

[0747] The server uses an AI model to generate individually optimized lesson videos (or training videos) based on the user's progress data. The generated videos are then delivered to the user's device.

[0748] Input: User progress data

[0749] Output: Individually optimized lesson videos (or training videos)

[0750] Specific operation: The server combines learning content, training content, avatars, and a voice generation engine to create video content. After video generation, the server sends the video data to the terminal.

[0751] Step 4: Play video content

[0752] The terminal plays lecture videos (or training videos) received from the server for the user. The user watches these videos and engages in learning or training.

[0753] Input: Individually optimized lesson videos (or training videos)

[0754] Output: The video the user will watch.

[0755] Specific operation: The device displays the received video data using a playback application. The user watches the video and proceeds with learning or training.

[0756] Step 5: Recognizing your emotional state

[0757] The device's camera and microphone are used to collect the user's facial expressions and voice data, which are then sent to a server. The server uses an emotion recognition engine to analyze the user's emotional state.

[0758] Input: User facial expression data and voice data

[0759] Output: User's emotional state

[0760] Specific operation: The device periodically activates its camera and microphone to capture facial expressions and audio data. This data is sent to a server in real time, and the server uses an emotion recognition engine to analyze the emotional state.

[0761] Step 6: Adjusting the instruction and training content

[0762] The server adjusts the content and difficulty level of lecture and training videos in real time based on the user's recognized emotional state. If the user is feeling stressed, it will increase the amount of gentle explanations and adjust the difficulty level of the content.

[0763] Input: User's emotional state

[0764] Output: Adjusted lesson video (or training video)

[0765] Specific operation: Based on the emotional state, the server performs processes such as regenerating video content or replacing parts of already delivered content. The adjusted video data is then resent to the terminal.

[0766] Step 7: Automated test generation and evaluation

[0767] The server uses an AI model to automatically generate appropriate test questions based on the user's progress and sentiment data, and delivers them to the device. When the user takes the test, the device sends the answers to the server, which performs OCR processing and scores them using the AI ​​model.

[0768] Input: User progress data, sentiment data, test answers

[0769] Output: Automated test questions, scoring results

[0770] Specific operation: The server generates appropriate test questions and delivers them to the terminal. Once the user completes the test, the terminal sends the answers to the server, which then scores them using OCR processing and an AI model.

[0771] Step 8: The World of Question Answering Functions

[0772] When a user enters a question using their device, the device sends the question to the server. The server uses an AI model to analyze the question, generates an appropriate answer, and provides it to the user through the device.

[0773] Input: User's question

[0774] Output: Response from the server

[0775] Specific operation: The user enters a question using the terminal's question-answering interface. The terminal sends this question to the server, which generates an answer based on the results of analysis by an AI model, and sends it back to the terminal for display.

[0776] Step 9: Provide learning materials and study plans.

[0777] The server generates appropriate learning materials and study plans based on learner progress data and sentiment data, and provides them to the user through the terminal.

[0778] Input: Progress data, sentiment data

[0779] Output: Individually optimized learning materials and study plans

[0780] Specific operation: Based on the collected data, the server generates a learning plan including the next learning objectives and recommended learning time, and sends it to the terminal. The terminal then displays this plan to the user.

[0781] The above outlines the specific processing steps for implementing this invention. In each step, it is possible to take appropriate action in real time based on the collected data.

[0782] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0783] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0784] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0785] [Second Embodiment]

[0786] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0787] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0788] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0789] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0790] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0791] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0792] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0793] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0794] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0795] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0796] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0797] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0798] This invention relates to an AI-powered online learning system that tracks learners' progress and provides individually optimized educational resources. The system consists of a server, a terminal, and a user (learner), each performing a specific function.

[0799] Tracking learning progress

[0800] server

[0801] The server collects data such as learners' learning activities, test results, and assignment submission status, and records it in a database. It also uses an AI model to analyze this data and understand the learners' progress. This allows for the provision of instruction tailored to each individual learner.

[0802] terminal

[0803] When a learner logs in, their device sends authentication information to the server. Upon successful authentication, the learning dashboard sent from the server is displayed. This dashboard shows the learner's progress, the status of submitted assignments, and other relevant information.

[0804] Generation and distribution of lesson videos

[0805] server

[0806] The server uses an AI model to generate individually optimized lesson videos based on the learner's progress data. These lesson videos are visually easy to understand using avatars and include audio explanations using a speech generation engine. Once the lesson videos are generated, the server delivers them to the user's device.

[0807] terminal

[0808] The terminal displays lesson videos received from the server to the learner. By watching these videos, learners can receive personalized instruction.

[0809] Generation and grading of quizzes and regular tests

[0810] server

[0811] The server automatically generates appropriate quizzes or periodic tests based on learning progress data. This allows for effective assessment of learners' understanding. When learners take a test, the server processes the submitted answers using OCR and automatically grades them using an AI model.

[0812] terminal

[0813] The terminal displays test questions sent from the server and provides an interface for learners to input their answers. Once the learner completes the test, the terminal sends the answers to the server, and the server displays the graded results in real time.

[0814] Question and Answer Session

[0815] server

[0816] The server has an AI model installed for question-and-answer sessions. When a learner submits a question, the server analyzes the question and generates the best possible answer.

[0817] terminal

[0818] When a learner enters a question using the terminal's question-and-answer interface, the terminal sends the question to the server. The answer provided by the server is then displayed to the learner through the terminal.

[0819] Provision of teaching materials and learning plans

[0820] server

[0821] The server generates personalized learning materials and study plans based on the learner's progress data. These plans include which subjects to focus on and recommended study times.

[0822] terminal

[0823] The terminal displays learning materials and study plans received from the server to the learner. This allows the learner to study efficiently.

[0824] Specific example

[0825] Tracking learning progress

[0826] When a user (learner) watches a math lesson video and then submits an assignment, the device automatically sends this information to the server. The server uses this data to update the learner's progress and saves it in a database.

[0827] Generation and distribution of lesson videos

[0828] If a user (learner) struggles with the topic of "quadratic equations," the server generates a specialized lesson video on quadratic equations based on past data, adds audio explanations, and then delivers it to the user's device.

[0829] Generation and grading of quizzes and regular tests

[0830] If a user (learner) wants to take a test on quadratic equations, the server uses an AI model to automatically generate questions and delivers them to the device. When the learner enters their answers and the device sends them to the server, the server uses OCR to digitize the answers and the AI ​​model to grade them.

[0831] Question and Answer Session

[0832] When a user (learner) enters a question about "how to solve an equation," the device sends the question to the server. The server analyzes the question using an AI model, generates an appropriate answer, and provides it to the learner through the device.

[0833] Provision of teaching materials and learning plans

[0834] The server analyzes each learner's progress data and generates a study plan that includes subjects to focus on in the following week and recommended study time. The terminal displays this plan to the learner.

[0835] The above describes the embodiment of this invention. This system makes it possible to efficiently and effectively provide individually optimized educational resources.

[0836] The following describes the processing flow.

[0837] Tracking learning progress

[0838] Step 1:

[0839] The user (student) logs into the online learning platform using their device.

[0840] The terminal sends the user's authentication information to the server.

[0841] Step 2:

[0842] The server verifies the authentication information, and if authentication is successful, it generates a learning dashboard and sends it to the device.

[0843] The device displays the received dashboard, showing the user their current learning progress.

[0844] Step 3:

[0845] Users (students) watch lesson videos or take tests.

[0846] The device records these learning activities and sends them to the server in real time.

[0847] Step 4:

[0848] The server records the received training data in a database and performs analysis using an AI model.

[0849] The learner's progress is updated based on the analysis results.

[0850] Generation and distribution of lesson videos

[0851] Step 1:

[0852] The server starts generating individually optimized lesson videos based on the learner's progress data.

[0853] The AI ​​model selects the appropriate content and activates the avatar and voice generation engine.

[0854] Step 2:

[0855] The server generates visually easy-to-understand videos using avatars and adds narration using a voice generation engine.

[0856] The generated lesson videos are saved on the server.

[0857] Step 3:

[0858] The server sends the generated lesson videos to the terminal.

[0859] The device displays the received video to the user (student) and begins playback.

[0860] Generation and grading of quizzes and regular tests

[0861] Step 1:

[0862] The server automatically generates appropriate test questions using an AI model based on the learning progress data.

[0863] The generated tests are saved in digital format.

[0864] Step 2:

[0865] The server sends the generated test questions to the terminal.

[0866] Users (students) take the test through their devices.

[0867] Step 3:

[0868] The user (student) enters their answer and sends it to the server using their device.

[0869] The answers will be formatted appropriately for OCR processing.

[0870] Step 4:

[0871] The server digitizes the answers using an OCR engine and performs automatic scoring using an AI model.

[0872] The scoring results are saved in a database and immediately sent to the device as feedback.

[0873] Step 5:

[0874] The device displays the scoring results to the user (student) and suggests the next learning steps.

[0875] Question and Answer Session

[0876] Step 1:

[0877] The user (student) enters their question using the terminal's question-and-answer interface.

[0878] The terminal sends the question to the server.

[0879] Step 2:

[0880] The server receives the question and analyzes its content using an AI model.

[0881] The system generates the optimal response based on the analysis results.

[0882] Step 3:

[0883] The server sends the generated response to the terminal.

[0884] The device displays the answer to the user (student).

[0885] Provision of teaching materials and learning plans

[0886] Step 1:

[0887] The server generates individually optimized learning materials and study plans based on the learner's progress data.

[0888] The AI ​​model calculates the next learning goal and recommended learning time.

[0889] Step 2:

[0890] The server sends the generated learning materials and study plans to the terminal.

[0891] The terminal displays the received plan and learning materials to the user (student).

[0892] The above describes the specific operations at each processing step of the system. This system makes it possible to provide an optimal learning experience for each individual learner.

[0893] (Example 1)

[0894] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0895] Traditional online learning systems have difficulty individually tracking the progress of each learner and providing them with the most suitable learning materials and teaching methods. Furthermore, manual test creation and grading are time-consuming, making it difficult to quickly assess learners' understanding. Additionally, real-time question-and-answer sessions are not possible, preventing immediate resolution of learners' questions.

[0896] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0897] In this invention, the server includes means for tracking learner progress data, means for generating optimized lesson information based on the progress data, means for distributing the lesson information to learners, means for automatically generating quizzes or periodic tests for learners, means for performing optical character recognition (OCR) processing on the answers to the tests, means for automatically scoring the OCR-processed answers, means for generating appropriate answers to learners' questions, means for providing learners with optimal learning materials and learning plans, means for learners to engage in learning activities using a terminal, means for saving the learners' progress in a database using the progress data and analyzing it using an AI model, means for using an avatar and a voice generation engine to generate the lesson videos, and means for performing OCR processing on the answers and analyzing and displaying the scoring results using an AI model. This makes it possible to efficiently provide individually optimized educational resources and realize flexible instruction according to the learners' progress. Furthermore, the automation of test generation and scoring enables rapid and accurate learning assessment. Real-time question and answer allows learners' questions to be resolved immediately.

[0898] "Learner progress data" refers to data that includes information such as the learning activities undertaken by learners, test results, and assignment submission status.

[0899] "Lesson information" refers to educational content optimized based on learner progress data, and includes visual and audio lesson videos.

[0900] "Automatic generation of quizzes or periodic tests" refers to the process of automatically creating appropriate test questions based on learner progress data.

[0901] Optical character recognition (OCR) is a technology that converts handwritten or printed text into digital data.

[0902] "Automatic scoring" is a process that automatically evaluates and scores test answers converted through optical character recognition using an algorithm.

[0903] "Generating appropriate answers" means that the AI ​​model generates the best possible answer to a question submitted by the learner via their device.

[0904] "Providing optimal learning materials and study plans" means proposing and providing individually optimized learning materials and study plans based on the learner's progress data.

[0905] "Device" refers to a device used by learners to conduct learning activities, and includes, for example, computers, tablets, and smartphones.

[0906] An "avatar" is a visual character used in lesson videos to convey educational content in a visually easy-to-understand way.

[0907] A "speech generation engine" is a technology that converts text into natural-sounding speech and uses it to provide audio explanations in educational videos.

[0908] An "AI model" is a technology that uses machine learning algorithms to perform tasks such as analyzing progress data, automatically grading tests, and generating answers to questions.

[0909] This invention relates to an AI-powered online learning system that tracks learners' progress and provides individually optimized educational resources. This system primarily consists of three components: a server, a terminal, and a user (learner). Specific embodiments are described below.

[0910] Tracking learning progress

[0911] server

[0912] The server collects data such as learners' learning activities, test results, and assignment submission status, and records it in a database. The server also uses AI models (such as PyTorch or TensorFlow) to analyze this data and understand the learners' progress. For example, when a learner watches a math lesson video and then submits an assignment, the device automatically sends this information to the server, which then updates the progress status and saves it in the database.

[0913] terminal

[0914] When a user (learner) logs in, the device sends authentication information to the server. If authentication is successful, the learning dashboard sent from the server is displayed. This dashboard displays the learner's progress, the status of submitted assignments, and other information.

[0915] Generation and distribution of lesson videos

[0916] server

[0917] The server uses an AI model based on the learner's progress data to generate individually optimized lesson videos. These lesson videos are visually easy to understand using avatars (e.g., Vyond) and include voice explanations using a speech generation engine (e.g., Google Text-to-Speech). For example, if a learner struggles with "quadratic equations," the server generates a lesson video specifically tailored to quadratic equations based on past data, adds voice explanations, and then delivers it to the device.

[0918] terminal

[0919] The terminal displays lesson videos received from the server to the learner. By watching these videos, learners can receive personalized instruction.

[0920] Generation and grading of quizzes and regular tests

[0921] server

[0922] The server automatically generates appropriate quizzes or periodic tests based on learning progress data. This allows for effective assessment of learners' understanding. For example, if a learner wants to take a test on "quadratic equations," the server uses an AI model to automatically generate the questions and delivers them to the device. Once the learner enters their answers and the device sends them to the server, the server uses an OCR tool (e.g., Tesseract) to convert the handwritten answers into digital data and automatically grades them.

[0923] terminal

[0924] The terminal displays test questions sent from the server and provides an interface for learners to input their answers. Once the learner completes the test, the terminal sends the answers to the server, and the server displays the graded results in real time.

[0925] Question and Answer Session

[0926] server

[0927] The server has an AI model (e.g., OpenAI GPT-3) installed for question-and-answer sessions. When a learner submits a question, the server analyzes the question and generates the most appropriate answer. For example, if a learner enters a question about "how to solve equations," the server uses the AI ​​model to analyze the question, generates an appropriate answer, and provides it to the learner via their terminal.

[0928] terminal

[0929] When a learner enters a question using the terminal's question-and-answer interface, the terminal sends the question to the server. The answer provided by the server is then displayed to the learner through the terminal.

[0930] Provision of teaching materials and learning plans

[0931] server

[0932] The server generates individually optimized learning materials and study plans based on learners' progress data. These study plans include which subjects to focus on and recommended study times. For example, the server analyzes each learner's progress data and generates a study plan that includes subjects to focus on and recommended study times for the following week.

[0933] terminal

[0934] The terminal displays learning materials and study plans received from the server to the learner. This allows the learner to study efficiently.

[0935] Example of a prompt

[0936] "Generate answers to questions from learners about how to solve equations."

[0937] Please explain the process for creating instructional videos on quadratic equations.

[0938] "Please explain how to generate a learning plan based on learner progress data."

[0939] The above describes the embodiment of this invention. This system efficiently provides individually optimized educational resources and enables flexible instruction tailored to the learner's progress. Furthermore, the automation of test generation and scoring allows for rapid and accurate learning assessment. Real-time question and answer allows learners' questions to be resolved immediately.

[0940] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0941] Step 1:

[0942] The user (learner) logs into the device.

[0943] Input: Enter your User ID and password on the device's login screen.

[0944] Action: The terminal sends the entered authentication information to the server.

[0945] Output: The server performs login authentication and returns the authentication result to the terminal.

[0946] Step 2:

[0947] The server authenticates the learner.

[0948] Input: User ID and password sent from the terminal.

[0949] Operation: The server compares the user information in the database and performs authentication.

[0950] Output: If authentication is successful, a login success response is sent to the device, and data is sent to display the learning dashboard.

[0951] Step 3:

[0952] Learners engage in learning activities (watching videos, submitting assignments, etc.).

[0953] Input: The user (learner) selects and watches a lesson video, or enters their answers to an assignment.

[0954] Operation: The device collects learner activity data (viewing history, assignment submission information, etc.).

[0955] Output: The device sends the collected learning activity data to the server.

[0956] Step 4:

[0957] The device sends the training data to the server.

[0958] Input: Learner's video viewing history and assignment submission information.

[0959] Operation: The device automatically sends this data to the server.

[0960] Output: The server receives the learning activity data.

[0961] Step 5:

[0962] The server saves the data to the database.

[0963] Input: Learning activity data sent from the device.

[0964] Operation: The server records and updates data in the database.

[0965] Output: Progress data in the database is updated.

[0966] Step 6:

[0967] The server uses an AI model to analyze the data and update the progress status.

[0968] Input: Learning activity data and progress data from the database.

[0969] Operation: The server uses an AI model (e.g., PyTorch or TensorFlow) to perform data analysis.

[0970] Output: As a result of the analysis, the individual learner's progress is updated and reflected in the dashboard.

[0971] Step 7:

[0972] The server optimizes the content of the lesson videos based on the learner's progress data.

[0973] Input: Learner progress data from the database.

[0974] Operation: The server uses an AI model (e.g., TensorFlow) to determine the optimal content for the lesson videos.

[0975] Output: The content of the generated lesson video (for example, content specifically focused on quadratic equations).

[0976] Step 8:

[0977] The server generates lesson videos using avatars and a voice generation engine.

[0978] Input: Optimized lesson video content.

[0979] Operation: The server generates a visual avatar using an animation tool (e.g., Vyond) and adds a voice description using a speech generation engine (e.g., Google Text-to-Speech).

[0980] Output: Generated lesson video.

[0981] Step 9:

[0982] The server generates and delivers lesson videos to the devices.

[0983] Input: Generated lesson video.

[0984] Operation: The server streams the lesson videos to the terminals.

[0985] Output: Lecture video delivered to the device.

[0986] Step 10:

[0987] The device displays the video to the learner.

[0988] Input: Lecture videos streamed from the server.

[0989] Operation: The device plays the lesson video and allows the learner to watch it.

[0990] Output: Learners watch the lesson videos.

[0991] Step 11:

[0992] The server automatically generates quizzes or periodic tests based on learning progress data.

[0993] Input: Learner progress data.

[0994] Operation: The server uses an AI model (e.g., SciKit-Learn) to automatically generate appropriate test questions.

[0995] Output: Generated test questions.

[0996] Step 12:

[0997] The server delivers the test questions to the terminals.

[0998] Input: Generated test questions.

[0999] Operation: The server sends the test questions to the terminal.

[1000] Output: Test questions delivered to the terminal.

[1001] Step 13:

[1002] The learners answer the test questions.

[1003] Input: Test question.

[1004] Operation: The user (learner) answers test questions on the device and inputs the answers into the device.

[1005] Output: Learner's answer.

[1006] Step 14:

[1007] The device sends the answer to the server.

[1008] Input: Learner's answer.

[1009] Operation: The device sends the collected answer data to the server.

[1010] Output: Answer data sent to the server.

[1011] Step 15:

[1012] The server digitizes the answers using optical character recognition (OCR).

[1013] Input: Submitted answer data.

[1014] Operation: The server uses an OCR tool (e.g., Tesseract) to convert handwritten answers into digital data.

[1015] Output: Digitized answer data.

[1016] Step 16:

[1017] The server uses an AI model to automatically score the results.

[1018] Input: Digitized answer data.

[1019] Operation: The server uses an AI model (e.g., SciKit-Learn) to automatically grade the answers.

[1020] Output: Scoring results.

[1021] Step 17:

[1022] The server delivers the scoring results to the terminal.

[1023] Input: Scoring result.

[1024] Operation: The server sends the scoring results to the terminal in real time.

[1025] Output: The scoring result displayed on the device.

[1026] Step 18:

[1027] The learner enters the question from their device.

[1028] Input: Learner's question.

[1029] Operation: The user (learner) enters a question using a question-and-answer interface.

[1030] Output: Question data sent from the terminal to the server.

[1031] Step 19:

[1032] The terminal sends the question to the server.

[1033] Input: Question data.

[1034] Operation: The terminal sends the question data to the server.

[1035] Output: Question data received by the server.

[1036] Step 20:

[1037] The server analyzes the question.

[1038] Input: Received question data.

[1039] Operation: The server uses an AI model (e.g., OpenAI GPT-3) to analyze the question.

[1040] Output: Analysis results.

[1041] Step 21:

[1042] The server generates the answer.

[1043] Input: Analysis results of the question.

[1044] Operation: The server generates the optimal answer.

[1045] Output: Generated answer.

[1046] Step 22:

[1047] The server sends the response to the terminal.

[1048] Input: Generated response.

[1049] Operation: The server sends the response to the terminal.

[1050] Output: The response received by the terminal.

[1051] Step 23:

[1052] The device displays the answer to the learner.

[1053] Input: Response sent from the server.

[1054] Operation: The device displays the answer to the learner.

[1055] Output: Learners see the answers.

[1056] Step 24:

[1057] The server generates a learning plan based on the learner's progress data.

[1058] Input: Learner progress data.

[1059] Operation: The server uses the AI ​​model to generate an individually optimized learning plan.

[1060] Output: The generated training plan.

[1061] Step 25:

[1062] The server delivers the learning plan to the device.

[1063] Input: The generated training plan.

[1064] Operation: The server sends the learning plan to the terminal.

[1065] Output: The learning plan displayed on the device.

[1066] The above outlines the specific processing flow of this system. Each step involves specific actions, enabling the efficient provision of individually optimized educational resources and flexible instruction tailored to the learner's progress.

[1067] (Application Example 1)

[1068] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[1069] Traditional online learning systems have struggled to effectively track learners' progress and provide individually optimized educational resources. Furthermore, they lacked the means to create a more intuitive and immersive learning environment through virtual learning experiences. Additionally, automatically generating and immediately evaluating optimal quizzes and periodic tests based on learners' progress was difficult.

[1070] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[1071] In this invention, the server includes means for tracking learner progress data, means for generating optimized lesson content based on the progress data, means for delivering the lesson content to learners, means for automatically generating quizzes or periodic tests for learners in a virtual environment, means for optical character recognition processing of the test answers, means for automatically scoring the optical character recognition processed answers, means for generating appropriate answers to learners' questions, means for providing learners with optimal learning resources and learning plans, and a learning experience in a virtual environment. This makes it possible to grasp learners' progress in real time and provide individually optimized educational resources. Furthermore, a more intuitive and immersive learning experience can be achieved through the learning experience in the virtual environment.

[1072] "Learner progress data" refers to data generated when learners engage in learning activities, and includes information such as the content of lessons viewed, submitted assignments, and test results.

[1073] "Lesson content" refers to learning materials provided to learners, including learning resources in formats such as videos, audio, and text.

[1074] A "virtual environment" refers to a virtual learning space created using computer simulations, where learners can engage in learning activities while immersed in the environment, just as they would in the real world.

[1075] A "quiz or periodic test" is an examination administered periodically to measure a learner's understanding of the material, and includes both small-scale review tests and tests that assess long-term learning outcomes.

[1076] Optical Character Recognition (OCR) is a technology that reads text information from scanned documents and images, and is a process for converting them into a digital format.

[1077] "Automated scoring" refers to the process of using AI technology to mechanically evaluate test answers and assign scores.

[1078] "Means for generating appropriate answers" refers to technology in which AI automatically generates the most effective answers based on the learner's questions.

[1079] "Learning resources" refer to educational content such as textbooks, videos, and audio guides that learners use to study efficiently.

[1080] A "learning plan" is a schedule or guideline that shows how learners should proceed with their studies within a certain period of time.

[1081] This invention relates to an AI-powered online learning system that tracks learners' progress and provides individually optimized educational resources within a virtual environment based on that progress. Specific embodiments are described below.

[1082] Tracking learning progress

[1083] server

[1084] The server collects data such as learners' learning activities, test results, and assignment submission status, and records it in a database. It also uses a generative AI model to analyze this data and understand the learners' progress. This allows for the provision of instruction tailored to each individual learner.

[1085] terminal

[1086] When a learner logs into the virtual environment, the device sends authentication information to the server. If authentication is successful, the learning dashboard sent from the server is displayed within the virtual environment. This dashboard displays the learner's progress, the status of submitted assignments, and other information.

[1087] Generation and distribution of course content

[1088] server

[1089] The server uses a generative AI model to generate individually optimized lesson content based on the learner's progress data. This lesson content is visually easy to understand using avatars and includes voice explanations using an artificial speech generation engine. Once the lesson content is generated, the server delivers it to terminals within the virtual environment.

[1090] terminal

[1091] The terminal displays lesson content received from the server to the learner within a virtual environment. By viewing this content, learners can receive individually optimized learning guidance.

[1092] Generation and grading of quizzes and regular tests

[1093] server

[1094] The server automatically generates appropriate quizzes or periodic tests based on learning progress data. This allows for effective assessment of learners' understanding. When learners take a test, the server processes the submitted answers using optical character recognition and automatically grades them using a generative AI model.

[1095] terminal

[1096] The terminal displays test questions sent from the server within a virtual environment and provides an interface for learners to input their answers. Once the learner completes the test, the terminal sends the answers to the server, and the server displays the graded results in real time.

[1097] Question and Answer Session

[1098] server

[1099] The server has a generative AI model installed for question and answering. When a learner submits a question, the server analyzes the question and generates the best possible answer.

[1100] terminal

[1101] When a learner enters a question using the terminal's question-and-answer interface, the terminal sends the question to the server. The answer provided by the server is then displayed to the learner within the virtual environment via the terminal.

[1102] Provision of teaching materials and learning plans

[1103] server

[1104] The server generates personalized learning materials and study plans based on the learner's progress data. These plans include which subjects to focus on and recommended study times.

[1105] terminal

[1106] The terminal displays learning materials and study plans received from the server to the learner within a virtual environment. This allows learners to study efficiently.

[1107] Specific example

[1108] Example of a learning progress tracking prompt message

[1109] Tracking learning progress

[1110] When a learner watches math lesson content and then submits an assignment, the device automatically sends this information to the server. The server uses this data to update the learner's progress and saves it in a database.

[1111] Generation and distribution of course content

[1112] If a learner struggles with the topic of "quadratic equations," the server generates specialized lesson content based on past data, adds an artificial voice explanation, and then delivers it to the terminal in the virtual environment.

[1113] Examples of prompt messages for generating quizzes and periodic tests.

[1114] Please generate a short quiz on "quadratic equations" for learner ID: 12345.

[1115] Examples of question-and-answer prompts

[1116] Learner ID: 12345 has a question about "How to solve equations". Please generate the best answer.

[1117] Provision of teaching materials and learning plans

[1118] The server analyzes each learner's progress data and generates a study plan that includes subjects to focus on in the following week and recommended study time. The terminal displays this plan to the learner within the virtual environment.

[1119] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[1120] Step 1:

[1121] The user logs into the virtual environment.

[1122] Input: User ID, Password

[1123] Operation: Enter your user ID and password, and the terminal will send them to the server.

[1124] Output: Authentication result

[1125] Specific operation: The server compares user information with the database and returns whether authentication was successful or not.

[1126] Step 2:

[1127] The device displays the learner's learning dashboard.

[1128] Input: Authentication result, User ID

[1129] Operation: If authentication is successful, learner progress data is retrieved from the server and a learning dashboard is generated.

[1130] Output: Learning Dashboard

[1131] Specific operation: The dashboard displays learning progress, viewed content, submitted assignments, and more.

[1132] Step 3:

[1133] The server generates optimized lesson content.

[1134] Input: Learner progress data

[1135] Operation: Uses a generative AI model to analyze learner progress data and generate optimal lesson content.

[1136] Output: Course content

[1137] Specific actions: The lesson content will include avatars and artificial voice explanations.

[1138] Step 4:

[1139] The device displays the lesson content within a virtual environment.

[1140] Input: Lesson content

[1141] Operation: Displays lecture content received from the server through a screen or avatar in a virtual environment.

[1142] Output: Lesson content viewable by learners

[1143] Specific actions: Learners watch lesson content and complete related assignments.

[1144] Step 5:

[1145] The server automatically generates short quizzes or periodic tests.

[1146] Input: Learner progress data, test requests

[1147] Operation: Generates appropriate test questions using a generative AI model.

[1148] Output: Test questions

[1149] Specific operation: The AI ​​automatically generates the most suitable problems based on progress data and provides them in the form of tests.

[1150] Step 6:

[1151] The terminal displays the test questions within the virtual environment.

[1152] Input: Test Questions

[1153] Operation: Displays test questions received from the server on the interface and accepts answer input.

[1154] Output: Learner's answer

[1155] Specific actions: The learner takes a test and enters their answers.

[1156] Step 7:

[1157] The server processes the learner's answers using optical character recognition (OCR).

[1158] Input: Learner's answer

[1159] Operation: Performs optical character recognition processing to convert answers from handwritten text or images into digital data.

[1160] Output: Digital answer data

[1161] Specific operation: The AI ​​model recognizes the answer and converts it into a format that can be stored in the database.

[1162] Step 8:

[1163] The server automatically scores the answers that have been processed using optical character recognition.

[1164] Input: Digital answer data

[1165] Operation: Use a generative AI model to score the answers and calculate the results.

[1166] Output: Scoring results

[1167] Specific operation: A score is assigned to each answer based on the scoring algorithm.

[1168] Step 9:

[1169] The device displays the scoring results in real time.

[1170] Input: Scoring result

[1171] Operation: Displays the scoring results received from the server on the dashboard.

[1172] Output: Scoring results displayed to learners

[1173] Specific operation: Scoring results are displayed on the screen in real time, allowing learners to check them.

[1174] Step 10:

[1175] The server manages the Q&A session.

[1176] Input: Learner's question

[1177] Operation: Uses a generative AI model to analyze the question and generate an appropriate answer.

[1178] Output: Answer text

[1179] Specific operation: The AI ​​receives a question from the learner, analyzes it, generates an answer, and sends it back.

[1180] Step 11:

[1181] The device displays the answers to the questions.

[1182] Input: Answer text

[1183] Operation: Displays the response provided by the server within the virtual environment.

[1184] Output: Answers that learners can see.

[1185] Specific action: Learners check their answers via their devices and deepen their understanding.

[1186] Step 12:

[1187] The server generates individually optimized learning materials and study plans for each learner.

[1188] Input: Learner progress data

[1189] Operation: Uses a generative AI model to analyze data and create new learning plans and materials.

[1190] Output: Individually optimized learning materials and study plans

[1191] Specific action: A study plan is created, including recommended subjects and study time for the following week.

[1192] Step 13:

[1193] The device displays individually optimized learning materials and study plans.

[1194] Input: Course materials and study plans

[1195] Operation: Displays the learning plan and materials received from the server within the virtual environment.

[1196] Output: Learning plan and materials presented to the learner

[1197] Specific operation: The learning dashboard displays the learning plan and recommended materials for the following week, allowing learners to proceed with their studies accordingly.

[1198] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[1199] This invention combines an AI-powered online learning system that tracks learners' progress and provides individually optimized educational resources with an emotion engine that recognizes learners' emotions. The system consists of a server, a terminal, and a user (learner), each performing a specific function.

[1200] Tracking learning progress

[1201] server

[1202] The server collects data such as learners' learning activities, test results, and assignment submission status, and records it in a database. An AI model is used to analyze this data and understand the learners' progress. The system is configured to provide optimal instruction based on the learners' progress.

[1203] terminal

[1204] When a learner logs in, the device sends authentication information to the server. If authentication is successful, the learning dashboard sent from the server is displayed. This dashboard shows the learner's progress, the status of submitted assignments, and other information.

[1205] Generation and distribution of lesson videos

[1206] server

[1207] The server uses an AI model to generate individually optimized lesson videos based on the learner's progress data. The lesson videos are visually easy to understand using avatars, and audio explanations are added using a speech generation engine. The generated lesson videos are stored on the server and delivered to the user's device.

[1208] terminal

[1209] The terminal displays lesson videos received from the server to the learner. Learners can watch these videos and receive personalized instruction.

[1210] Learners' perception and response to emotions

[1211] server

[1212] The server is equipped with an emotion engine that recognizes not only the learner's progress data but also their emotional state. The emotion engine analyzes the learner's facial expression data and voice data to recognize their emotional state in real time.

[1213] terminal

[1214] The device collects the learner's facial expressions and voice through its camera and microphone and transmits them to the server. The server analyzes this data to understand the learner's emotional state.

[1215] server

[1216] The server adjusts the content and difficulty level of the lesson videos in real time based on the recognized emotional state. For example, if a learner is feeling stressed, the server can increase the amount of explanations in a gentle tone or change the content to a lower difficulty level.

[1217] Generation and grading of quizzes and regular tests

[1218] server

[1219] The server automatically generates appropriate test questions using an AI model based on learning progress data and sentiment data. When a learner takes the test, the server processes the submitted answers using OCR and automatically grades them using the AI ​​model.

[1220] terminal

[1221] The terminal displays test questions sent from the server, and the learner enters their answers. Once the test is completed, the answers are sent to the server, and the scoring results are immediately fed back.

[1222] Question and Answer Session

[1223] server

[1224] The server is equipped with an AI model for question-and-answer sessions, which generates the most appropriate answers to learners' questions. An emotion engine allows it to understand the learner's emotional state and provide answers at the right time.

[1225] terminal

[1226] When a learner enters a question using the terminal's question-and-answer interface, the terminal sends the question to the server. The server generates an answer, which is then displayed to the learner through the terminal.

[1227] Provision of teaching materials and learning plans

[1228] server

[1229] The server generates individually optimized learning materials and study plans based on the learner's progress and sentiment data. An AI model calculates the next learning objectives and recommended study time, and then creates the learning materials and study plans.

[1230] terminal

[1231] The terminal supports efficient learning by displaying learning materials and study plans received from the server to the learner.

[1232] Specific example

[1233] Tracking learning progress

[1234] When a user (learner) watches a math lesson video and then submits an assignment, the device automatically sends this information to the server. The server uses this data to update the learner's progress and saves it in a database.

[1235] Generation and distribution of lesson videos

[1236] If a user (learner) struggles with the topic of "quadratic equations," the server generates a specialized lesson video on quadratic equations based on past data, adds audio explanations, and then delivers it to the user's device.

[1237] Learners' perception and response to emotions

[1238] If a user (learner) is experiencing stress during a lesson, the device's camera detects this and notifies the server. The server's emotion engine confirms the high level of stress and adjusts the lesson video content to help the learner relax.

[1239] Generation and grading of quizzes and regular tests

[1240] If a user (learner) wants to take a test on quadratic equations, the server uses an AI model to automatically generate questions and delivers them to the device. When the learner enters their answers and the device sends them to the server, the server uses OCR to digitize the answers and the AI ​​model to grade them.

[1241] Question and Answer Session

[1242] When a user (learner) enters a question about "how to solve an equation," the device sends the question to the server. The server analyzes the question using an AI model, generates an appropriate answer, and provides it to the learner through the device.

[1243] Provision of teaching materials and learning plans

[1244] The server analyzes each learner's progress and sentiment data to generate a study plan that includes subjects to focus on in the following week and recommended study time. The terminal displays this plan to the learner.

[1245] The above describes the embodiment of this invention. This system makes it possible to provide an optimal learning experience for each individual learner. By combining it with an emotional engine, the efficiency of learning can be further improved.

[1246] The following describes the processing flow.

[1247] Tracking learning progress

[1248] Step 1:

[1249] The user (learner) logs into the online learning platform using their device.

[1250] The terminal sends the user's authentication information to the server.

[1251] Step 2:

[1252] The server verifies the authentication information, and if authentication is successful, it generates a learning dashboard and sends it to the device.

[1253] The device displays the received dashboard, showing the user their current learning progress.

[1254] Step 3:

[1255] Users (learners) watch lesson videos, take tests, and submit assignments.

[1256] The device records these learning activities and sends them to the server in real time.

[1257] Step 4:

[1258] The server records the received training data in a database and performs analysis using an AI model.

[1259] The learner's progress is updated based on the analysis results.

[1260] Generation and distribution of lesson videos

[1261] Step 1:

[1262] The server starts generating individually optimized lesson videos based on the learner's progress data.

[1263] The AI ​​model selects the appropriate content and activates the avatar and voice generation engine.

[1264] Step 2:

[1265] The server generates visually easy-to-understand videos using avatars and adds narration using a voice generation engine.

[1266] The generated lesson videos are saved on the server.

[1267] Step 3:

[1268] The server sends the generated lesson videos to the terminal.

[1269] The device displays the received video to the user (learner) and begins playback.

[1270] Learners' perception and response to emotions

[1271] Step 1:

[1272] While the user (learner) is watching the lesson video, the device uses its camera and microphone to collect the learner's facial expression data and voice data.

[1273] Step 2:

[1274] The device sends the collected emotional data to the server in real time.

[1275] The server analyzes this data to recognize the learner's emotional state.

[1276] Step 3:

[1277] The server adjusts the content and difficulty level of the lesson videos in real time based on the emotional state it recognizes.

[1278] For example, if a learner is experiencing stress, the content can be changed to a less difficult version.

[1279] Generation and grading of quizzes and regular tests

[1280] Step 1:

[1281] The server automatically generates appropriate test questions using an AI model based on learning progress data and sentiment data.

[1282] The generated tests are saved in digital format.

[1283] Step 2:

[1284] The server sends the generated test questions to the terminal.

[1285] The user (learner) takes the test through their device.

[1286] Step 3:

[1287] The user (learner) enters their answer and sends it to the server using their device.

[1288] The answers will be formatted appropriately for OCR processing.

[1289] Step 4:

[1290] The server digitizes the answers using an OCR engine and performs automatic scoring using an AI model.

[1291] The scoring results are saved in a database and immediately sent to the device as feedback.

[1292] Step 5:

[1293] The device displays the scoring results to the user (learner) and suggests the next learning steps.

[1294] Question and Answer Session

[1295] Step 1:

[1296] The user (learner) enters their question using the terminal's question-and-answer interface.

[1297] The terminal sends the question to the server.

[1298] Step 2:

[1299] The server receives the question and analyzes its content using an AI model.

[1300] The system generates the optimal response based on the analysis results.

[1301] Step 3:

[1302] The server sends the generated response to the terminal.

[1303] The device displays the answer to the user (learner).

[1304] Provision of teaching materials and learning plans

[1305] Step 1:

[1306] The server generates individually optimized learning materials and study plans based on learner progress data and sentiment data.

[1307] The AI ​​model calculates the next learning goal and recommended learning time.

[1308] Step 2:

[1309] The server sends the generated learning materials and study plans to the terminal.

[1310] The device displays the received plan and learning materials to the user (learner).

[1311] The above describes the specific operations at each processing step of the system. This system makes it possible to provide an optimal learning experience for each individual learner. By combining it with an emotion engine, the efficiency of learning can be further improved.

[1312] (Example 2)

[1313] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[1314] Traditional online learning systems have struggled to monitor learners' progress and emotional states in real time and provide personalized learning resources accordingly. Furthermore, the lack of features to adjust instruction based on learners' emotional states made it difficult to maintain motivation and maximize learning effectiveness. There were also challenges with the generation of quizzes and periodic tests, automated grading, and the quality of question-and-answer sessions, highlighting the need for a system that provides optimal education for each individual learner.

[1315] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for tracking learner progress data, means for generating optimized lesson content based on the progress data, means for delivering the lesson content to learners, means for collecting learner facial expression data and voice data, means for analyzing the facial expression data and voice data to recognize the learner's emotional state, means for adjusting the lesson content based on the emotional state, means for automatically generating quizzes or periodic tests for learners, means for character recognition processing of the test answers, means for automatically scoring the character recognition processed answers, means for generating appropriate answers to learners' questions, and means for providing learners with optimal learning materials and learning plans. This makes it possible to grasp the learner's progress and emotional state in real time and provide optimal learning resources.

[1316] A "learner" refers to a person who uses an online learning system to study educational content.

[1317] "Progress data" refers to information recorded in digital format, such as learners' learning activities, test results, and assignment submission status.

[1318] "Lesson content" refers to educational materials provided to learners, including in the form of text, videos, and audio.

[1319] "Distribution" refers to the act of sending data in digital format from a server to a learner's device.

[1320] "Facial expression data" refers to digital information about a learner's facial expressions acquired through a camera or other recording device.

[1321] "Audio data" refers to digital information about learners' voices acquired through audio collection devices such as microphones.

[1322] "Emotional state" refers to the learner's psychological and emotional state as identified as a result of the analysis of facial expression data and voice data.

[1323] "Character recognition processing" refers to the process of analyzing characters written on paper or in image data as digital data and converting them into text information.

[1324] "Automated scoring" refers to the process of using artificial intelligence models to mechanically evaluate test answers and assign scores.

[1325] "Question answering" refers to the act of a system generating and providing appropriate answers to questions from learners.

[1326] "Educational materials" refer to educational resources provided to learners, and include forms such as textbooks, videos, and practice exercises.

[1327] A "learning plan" refers to a plan created to present learners with the most suitable learning schedule and achievement goals.

[1328] A "virtual character" is an animated character created in a digital environment, and its role is to visually explain and instruct on lesson content.

[1329] "Speech generation means" refers to technology that converts text information into speech and uses it for narration and explanations.

[1330] An "artificial intelligence model" refers to an algorithm that uses machine learning or deep learning to perform data analysis and prediction.

[1331] This invention combines an AI-powered online learning system that tracks learners' progress and provides individually optimized educational resources with an emotion engine that recognizes learners' emotions. The system consists of a server, terminals, and users (learners), each performing a specific function.

[1332] Tracking learning progress

[1333] server

[1334] The server collects data such as learners' learning activities, test results, and assignment submission status, and records it in a database. An AI model implemented in Python is used to analyze this data and evaluate the learners' progress. Based on the results of this analysis, individually optimized instructional content is generated.

[1335] terminal

[1336] When a learner logs in, the device sends authentication information to the server. If authentication is successful, the learning dashboard sent from the server is displayed. This dashboard displays learning progress, the status of submitted assignments, and other information in real time.

[1337] Generation and distribution of lesson videos

[1338] server

[1339] The server generates individually optimized lesson videos using a generative AI model based on the learner's progress data. These lesson videos are created to be visually and aurally easy to understand using virtual characters and voice generation methods (e.g., Google Text-to-Speech API). The generated videos are stored on the server and delivered to the device.

[1340] terminal

[1341] The terminal displays lesson videos received from the server to the learner. Learners can watch these videos and receive individually optimized instruction.

[1342] Learners' perception and response to emotions

[1343] terminal

[1344] While learners are watching the lesson videos, the device's camera and microphone collect facial expression and voice data from the learners and send it to the server.

[1345] server

[1346] The server uses the Microsoft Azure Face API and Amazon Alexa Voice Service to analyze this data with an emotion engine and recognize the learner's emotional state. Based on the recognition results, it adjusts the content and difficulty level of the lesson videos in real time.

[1347] Generation and grading of quizzes and regular tests

[1348] server

[1349] The server automatically generates appropriate test questions using a generative AI model based on learning progress data and sentiment data. Once a test is submitted, the answers are digitized using an OCR tool (e.g., Tesseract OCR), and the AI ​​model automatically scores them.

[1350] terminal

[1351] The terminal displays test questions sent from the server, and the learner enters their answers. After the test is completed, the answers are sent to the server, and the results are immediately fed back.

[1352] Question and Answer Session

[1353] terminal

[1354] The learner enters a question using a question-and-answer interface, and the device sends the question to the server.

[1355] server

[1356] The server uses OpenAI GPT-4 to analyze the question and generate an appropriate answer. The generated answer is sent to the terminal and displayed to the learner.

[1357] Provision of teaching materials and learning plans

[1358] server

[1359] The server analyzes progress and sentiment data to generate personalized learning materials and study plans, including next learning goals and recommended study times. An AI model then suggests the optimal learning strategy.

[1360] terminal

[1361] The terminal displays learning materials and study plans received from the server to the learner, supporting efficient learning progress.

[1362] Specific example

[1363] Tracking learning progress

[1364] Specific example: When a user (learner) watches a math lesson video and then submits an assignment, the device automatically sends this information to the server. The server updates the learner's progress based on this data and saves it in a database.

[1365] Generation and distribution of lesson videos

[1366] Specific example: If a user (learner) struggles with the topic of "quadratic equations," the server generates a video lesson specifically focused on quadratic equations based on past data, adds audio explanations, and then delivers it to the user's device.

[1367] Learners' perception and response to emotions

[1368] Specific example: If a user (learner) experiences stress during a lesson, the device's camera detects this and notifies the server. The server then uses an emotion engine to analyze the stress and adjust the lesson video and teaching content accordingly.

[1369] Generation and grading of quizzes and regular tests

[1370] Specific example: If a user (learner) wants to take a test on "quadratic equations," the server automatically generates questions using a generative AI model and delivers them to the device. When the learner enters their answer and the device sends it to the server, the server uses OCR to digitize the answer and the AI ​​model scores it.

[1371] Question and Answer Session

[1372] Specific example: When a user (learner) inputs a question about "how to solve an equation," the device sends the question to the server. The server analyzes the question using an AI model, generates an appropriate answer, and sends it back to the device.

[1373] Provision of teaching materials and learning plans

[1374] Specific example: The server analyzes each learner's progress and sentiment data and generates a study plan that includes subjects to focus on in the following week and recommended study time. The terminal displays this plan to the learner.

[1375] The flow of the specific processing in Example 2 will be explained using Figure 13.

[1376] Step 1: Learner Authentication and Login

[1377] ---

[1378] Input: Username, Password

[1379] Output: Authentication token (success), error message (failure)

[1380] terminal

[1381] The user accesses the system and enters their username and password on the login screen. The entered authentication information is sent from the terminal to the server.

[1382] server

[1383] The server compares the received authentication information with the database. If authentication is successful, it generates a session ID and sends the learning dashboard to the device. If authentication fails, it sends an error message to the device.

[1384] Specific operation: The server checks if the username and password are correct. If they are correct, it returns the learning dashboard; otherwise, it returns an error message.

[1385] Step 2: Tracking Learning Progress

[1386] ---

[1387] Input: Learning activity data, test results, assignment submission status

[1388] Output: Updated progress status, analysis results

[1389] terminal

[1390] The user opens the learning dashboard to check their progress and the status of submitted assignments. The learning progress data is then sent to the server.

[1391] server

[1392] The server receives new learning progress data and records it in the database. An AI model implemented in Python is used to analyze the data and evaluate the learner's progress. The analysis results are reflected in the learning dashboard in real time.

[1393] Specific operation: Learning activities and test results are periodically sent to the server, analyzed by an AI model, and the user's learning progress is evaluated and recorded.

[1394] Step 3: Generate and distribute lesson videos

[1395] ---

[1396] Input: Progress data

[1397] Output: Lecture video URL, video content

[1398] server

[1399] The server generates individually optimized lesson videos using a generative AI model based on the learner's progress data. It uses virtual characters and voice generation methods (e.g., Google Text-to-Speech API) to create visually and aurally easy-to-understand lesson videos. The generated videos are saved on the server, and the video URL is sent to the user's device.

[1400] terminal

[1401] The device displays the lesson video using a video player based on the URL received from the server. The user then watches this video to progress with their learning.

[1402] Specific operation: Input learning progress data into the AI ​​model and deliver the generated video to the device.

[1403] Step 4: Learner's emotional recognition and response

[1404] ---

[1405] Input: Facial expression data, audio data

[1406] Output: Emotion recognition results, adjusted lesson content

[1407] terminal

[1408] While learners are watching the lesson videos, the device's camera and microphone collect facial expression and voice data from the learners and send it to the server.

[1409] server

[1410] The server uses the Microsoft Azure Face API and Amazon Alexa Voice Service to analyze this data with an emotion engine and recognize the learner's emotional state. Based on the recognition results, it adjusts the content and difficulty level of the lesson videos in real time.

[1411] Specific operation: Collected facial expressions and audio data are sent to a server, and the lesson content is adaptively modified based on the recognition results.

[1412] Step 5: Generating and grading quizzes and periodic tests

[1413] ---

[1414] Input: Progress data, sentiment data

[1415] Output: Test questions, scoring results

[1416] server

[1417] The server automatically generates appropriate test questions using an AI model based on learning progress data and sentiment data. The generated test questions are then sent to the device.

[1418] terminal

[1419] The terminal displays the test questions, and the learner enters their answers. The answers are then sent to the server.

[1420] server

[1421] The server uses an OCR tool (e.g., Tesseract OCR) to digitize the answers and an AI model to score them. The results are then fed back to the terminal.

[1422] Specific operation: Generate test questions based on data, grade learners' answers using an AI model, and return the results.

[1423] Step 6: Handling questions and answers

[1424] ---

[1425] Input: Learner's question

[1426] Output: Answer

[1427] terminal

[1428] The user enters a question using a question-and-answer interface, and the terminal sends it to the server.

[1429] server

[1430] The server uses OpenAI GPT-4 to analyze the question and generate an appropriate answer. The generated answer is then sent to the terminal.

[1431] Specific operation: The system receives a question, sends it to the server, generates an answer using a generative AI model, and sends it back to the terminal.

[1432] Step 7: Providing learning materials and study plans

[1433] ---

[1434] Input: Progress data, sentiment data

[1435] Output: Individually optimized learning materials and study plans

[1436] server

[1437] The server analyzes progress and sentiment data to generate personalized learning materials and study plans, including next learning goals and recommended study times. An AI model suggests the optimal learning strategy. The generated learning materials and study plans are then sent to the user's device.

[1438] terminal

[1439] The device displays received learning materials and study plans on a learning dashboard, supporting users in efficiently progressing through their studies.

[1440] Specific actions: Analyze data to create optimal learning materials and study plans, and present them to the user on a learning dashboard.

[1441] (Application Example 2)

[1442] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[1443] Traditional online learning systems struggle to track learners' progress and provide optimal educational resources. Furthermore, the lack of features to recognize learners' emotional states and adjust instruction in real time limits the potential for reducing learner stress and improving learning effectiveness. Similarly, in factory settings, the absence of systems to track work progress and provide optimal training resources results in insufficient work efficiency and mental well-being.

[1444] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[1445] In this invention, the server includes means for tracking learner (or worker) progress data, means for generating optimized lesson videos (or training videos) based on the progress data, and means for delivering the lesson videos (or training videos) to the learner (or worker). This makes it possible to grasp the learner's (or worker's) progress in real time and provide individually optimized educational resources (or training resources). Furthermore, the server includes means for recognizing the learner's (or worker's) emotional state and means for adjusting the instructional content (or training content) based on the emotional state, thereby reducing learner's (or worker's) stress and improving learning (or work) efficiency.

[1446] "Progress data" refers to data that shows the progress of learners' or workers' learning or work.

[1447] "Lesson videos" refer to educational video content provided to learners.

[1448] "Training videos" refer to video content provided to workers for training purposes.

[1449] "Automatic generation" means that the system automatically generates processes or results without requiring manual operation.

[1450] "OCR processing" is the process of digitizing handwritten or printed text using optical character recognition technology.

[1451] "Automated evaluation" refers to a system that automatically scores test answers and learning results.

[1452] "Generating an answer" means that the system creates an appropriate answer to the learner's question.

[1453] "Educational materials" refer to educational resources and content used by learners.

[1454] A "learning plan" is a plan or schedule designed to help learners progress through their studies efficiently.

[1455] "Emotional state" refers to the psychological feelings and moods of learners and workers.

[1456] "Instructional content" refers to the educational material and teaching methods provided to learners.

[1457] "Training content" refers to the training materials and instructional policies provided to workers.

[1458] This invention is a system that tracks the progress of learners and workers and provides individually optimized educational and training resources. Furthermore, it includes a function to recognize the emotional state of learners and workers in real time and adjust the instructional and training content based on that state.

[1459] System Configuration

[1460] The system primarily consists of servers, terminals, and users. Each element performs a specific function, enabling the provision of efficient and effective education and training.

[1461] server

[1462] The server forms the core of the system and implements the following functions:

[1463] 1. Tracking progress data

[1464] The server tracks the learning and work progress of learners and workers. The collected progress data is recorded in a database and analyzed using an AI model.

[1465] 2. Generation of optimized lesson videos and training videos

[1466] Using AI models, individually optimized video content is generated based on progress data. This includes the use of avatars and a voice generation engine to provide content that is easy to understand both visually and aurally.

[1467] 3. Recognition of emotional state

[1468] Equipped with an emotion recognition engine, it analyzes the user's facial expression and voice data to recognize their emotional state in real time. This information is sent to a server and used to adjust instruction and training content.

[1469] 4. Automatic generation and evaluation

[1470] Quiz and periodic tests for learners are automatically generated. Furthermore, submitted test answers are digitized using OCR processing and automatically graded using an AI model.

[1471] 5. Question Answering Function

[1472] It also features an AI model for question answering, which generates appropriate answers to questions entered by learners.

[1473] 6. Provision of teaching materials and learning plans

[1474] Based on progress data and sentiment data, the system generates and provides users with optimal learning materials and study plans.

[1475] terminal

[1476] A terminal is a device that the user directly operates and has the following functions:

[1477] 1. Login and Authentication

[1478] This is used when learners or workers log in to the system and sends authentication information to the server.

[1479] 2. Displaying the dashboard

[1480] You can view the dashboard sent from the server and check the progress, status of submitted assignments, and more.

[1481] 3. Playback of video content

[1482] The system displays lecture and training videos received from the server, allowing users to view them.

[1483] 4. Collection of facial expression and voice data

[1484] The system collects the user's facial expressions and voice through cameras and microphones and transmits them to a server.

[1485] User

[1486] Users utilize the system as learners or workers and perform the following activities:

[1487] 1. Progress of learning and work

[1488] Students progress by watching individually optimized lesson and training videos.

[1489] 2. Taking the test and answering questions

[1490] Students take quizzes and regular tests, and use a question-answering interface to enter questions.

[1491] Hardware and software used

[1492] Hardware: Webcam, smartphone, tablet, PC

[1493] Software: OpenCV (image processing library), TensorFlow / Keras (AI model implementation)

[1494] Specific example

[1495] For example, when a factory worker begins training, a terminal tracks the worker's progress. The server analyzes the collected data and generates and distributes optimized training videos. It also collects the worker's facial expressions and voice through the terminal's camera and microphone, and recognizes their emotional state in real time using an emotion recognition engine. If the worker is experiencing stress, the server adjusts the training content based on that emotional state to reduce the worker's stress.

[1496] Example of a prompt

[1497] Example: "Create a program that recognizes the emotional state of workers in real time and provides the optimal training resources. Use facial expression data captured by a camera to recognize emotions and analyze them with an AI model."

[1498] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1499] Step 1: User login and authentication

[1500] The user logs into the system using a device (smartphone, tablet, or PC). The device sends the user's authentication information to the server. The server verifies the authentication information and, if the login is successful, sends the learning dashboard to the device.

[1501] Input: User authentication information (username, password)

[1502] Output: Authentication result (learning dashboard if authentication is successful, error message if it fails)

[1503] Specific operation: When a user enters their authentication information on the login screen and clicks the "Login" button, that information is sent to the server. The server then checks the database to verify the authentication information.

[1504] Step 2: Display the learning dashboard

[1505] After successful authentication, the server generates a learning dashboard based on the user's progress data and sends it to the device. The device then displays this dashboard.

[1506] Input: User progress data

[1507] Output: Learning Dashboard

[1508] Specific operation: The server retrieves user progress data from the database and analyzes the user's learning status using an AI model. Based on the results, it generates a dashboard that displays progress, submitted assignments, next assignments, etc.

[1509] Step 3: Generate and distribute lesson videos (or training videos).

[1510] The server uses an AI model to generate individually optimized lesson videos (or training videos) based on the user's progress data. The generated videos are then delivered to the user's device.

[1511] Input: User progress data

[1512] Output: Individually optimized lesson videos (or training videos)

[1513] Specific operation: The server combines learning content, training content, avatars, and a voice generation engine to create video content. After video generation, the server sends the video data to the terminal.

[1514] Step 4: Play video content

[1515] The terminal plays lecture videos (or training videos) received from the server for the user. The user watches these videos and engages in learning or training.

[1516] Input: Individually optimized lesson videos (or training videos)

[1517] Output: The video the user will watch.

[1518] Specific operation: The device displays the received video data using a playback application. The user watches the video and proceeds with learning or training.

[1519] Step 5: Recognizing your emotional state

[1520] The device's camera and microphone are used to collect the user's facial expressions and voice data, which are then sent to a server. The server uses an emotion recognition engine to analyze the user's emotional state.

[1521] Input: User facial expression data and voice data

[1522] Output: User's emotional state

[1523] Specific operation: The device periodically activates its camera and microphone to capture facial expressions and audio data. This data is sent to a server in real time, and the server uses an emotion recognition engine to analyze the emotional state.

[1524] Step 6: Adjusting the instruction and training content

[1525] The server adjusts the content and difficulty level of lecture and training videos in real time based on the user's recognized emotional state. If the user is feeling stressed, it will increase the amount of gentle explanations and adjust the difficulty level of the content.

[1526] Input: User's emotional state

[1527] Output: Adjusted lesson video (or training video)

[1528] Specific operation: Based on the emotional state, the server performs processes such as regenerating video content or replacing parts of already delivered content. The adjusted video data is then resent to the terminal.

[1529] Step 7: Automated test generation and evaluation

[1530] The server uses an AI model to automatically generate appropriate test questions based on the user's progress and sentiment data, and delivers them to the device. When the user takes the test, the device sends the answers to the server, which performs OCR processing and scores them using the AI ​​model.

[1531] Input: User progress data, sentiment data, test answers

[1532] Output: Automated test questions, scoring results

[1533] Specific operation: The server generates appropriate test questions and delivers them to the terminal. Once the user completes the test, the terminal sends the answers to the server, which then scores them using OCR processing and an AI model.

[1534] Step 8: The World of Question Answering Functions

[1535] When a user enters a question using their device, the device sends the question to the server. The server uses an AI model to analyze the question, generates an appropriate answer, and provides it to the user through the device.

[1536] Input: User's question

[1537] Output: Response from the server

[1538] Specific operation: The user enters a question using the terminal's question-answering interface. The terminal sends this question to the server, which generates an answer based on the results of analysis by an AI model, and sends it back to the terminal for display.

[1539] Step 9: Provide learning materials and study plans.

[1540] The server generates appropriate learning materials and study plans based on learner progress data and sentiment data, and provides them to the user through the terminal.

[1541] Input: Progress data, sentiment data

[1542] Output: Individually optimized learning materials and study plans

[1543] Specific operation: Based on the collected data, the server generates a learning plan including the next learning objectives and recommended learning time, and sends it to the terminal. The terminal then displays this plan to the user.

[1544] The above outlines the specific processing steps for implementing this invention. In each step, it is possible to take appropriate action in real time based on the collected data.

[1545] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1546] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1547] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[1548] [Third Embodiment]

[1549] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[1550] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[1551] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1552] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[1553] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[1554] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[1555] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[1556] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[1557] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1558] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1559] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[1560] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[1561] This invention relates to an AI-powered online learning system that tracks learners' progress and provides individually optimized educational resources. The system consists of a server, a terminal, and a user (learner), each performing a specific function.

[1562] Tracking learning progress

[1563] server

[1564] The server collects data such as learners' learning activities, test results, and assignment submission status, and records it in a database. It also uses an AI model to analyze this data and understand the learners' progress. This allows for the provision of instruction tailored to each individual learner.

[1565] terminal

[1566] When a learner logs in, their device sends authentication information to the server. Upon successful authentication, the learning dashboard sent from the server is displayed. This dashboard shows the learner's progress, the status of submitted assignments, and other relevant information.

[1567] Generation and distribution of lesson videos

[1568] server

[1569] The server uses an AI model to generate individually optimized lesson videos based on the learner's progress data. These lesson videos are visually easy to understand using avatars and include audio explanations using a speech generation engine. Once the lesson videos are generated, the server delivers them to the user's device.

[1570] terminal

[1571] The terminal displays lesson videos received from the server to the learner. By watching these videos, learners can receive personalized instruction.

[1572] Generation and grading of quizzes and regular tests

[1573] server

[1574] The server automatically generates appropriate quizzes or periodic tests based on learning progress data. This allows for effective assessment of learners' understanding. When learners take a test, the server processes the submitted answers using OCR and automatically grades them using an AI model.

[1575] terminal

[1576] The terminal displays test questions sent from the server and provides an interface for learners to input their answers. Once the learner completes the test, the terminal sends the answers to the server, and the server displays the graded results in real time.

[1577] Question and Answer Session

[1578] server

[1579] The server has an AI model installed for question-and-answer sessions. When a learner submits a question, the server analyzes the question and generates the best possible answer.

[1580] terminal

[1581] When a learner enters a question using the terminal's question-and-answer interface, the terminal sends the question to the server. The answer provided by the server is then displayed to the learner through the terminal.

[1582] Provision of teaching materials and learning plans

[1583] server

[1584] The server generates personalized learning materials and study plans based on the learner's progress data. These plans include which subjects to focus on and recommended study times.

[1585] terminal

[1586] The terminal displays learning materials and study plans received from the server to the learner. This allows the learner to study efficiently.

[1587] Specific example

[1588] Tracking learning progress

[1589] When a user (learner) watches a math lesson video and then submits an assignment, the device automatically sends this information to the server. The server uses this data to update the learner's progress and saves it in a database.

[1590] Generation and distribution of lesson videos

[1591] If a user (learner) struggles with the topic of "quadratic equations," the server generates a specialized lesson video on quadratic equations based on past data, adds audio explanations, and then delivers it to the user's device.

[1592] Generation and grading of quizzes and regular tests

[1593] If a user (learner) wants to take a test on quadratic equations, the server uses an AI model to automatically generate questions and delivers them to the device. When the learner enters their answers and the device sends them to the server, the server uses OCR to digitize the answers and the AI ​​model to grade them.

[1594] Question and Answer Session

[1595] When a user (learner) enters a question about "how to solve an equation," the device sends the question to the server. The server analyzes the question using an AI model, generates an appropriate answer, and provides it to the learner through the device.

[1596] Provision of teaching materials and learning plans

[1597] The server analyzes each learner's progress data and generates a study plan that includes subjects to focus on in the following week and recommended study time. The terminal displays this plan to the learner.

[1598] The above describes the embodiment of this invention. This system makes it possible to efficiently and effectively provide individually optimized educational resources.

[1599] The following describes the processing flow.

[1600] Tracking learning progress

[1601] Step 1:

[1602] The user (student) logs into the online learning platform using their device.

[1603] The terminal sends the user's authentication information to the server.

[1604] Step 2:

[1605] The server verifies the authentication information, and if authentication is successful, it generates a learning dashboard and sends it to the device.

[1606] The device displays the received dashboard, showing the user their current learning progress.

[1607] Step 3:

[1608] Users (students) watch lesson videos or take tests.

[1609] The device records these learning activities and sends them to the server in real time.

[1610] Step 4:

[1611] The server records the received training data in a database and performs analysis using an AI model.

[1612] The learner's progress is updated based on the analysis results.

[1613] Generation and distribution of lesson videos

[1614] Step 1:

[1615] The server starts generating individually optimized lesson videos based on the learner's progress data.

[1616] The AI ​​model selects the appropriate content and activates the avatar and voice generation engine.

[1617] Step 2:

[1618] The server generates visually easy-to-understand videos using avatars and adds narration using a voice generation engine.

[1619] The generated lesson videos are saved on the server.

[1620] Step 3:

[1621] The server sends the generated lesson videos to the terminal.

[1622] The device displays the received video to the user (student) and begins playback.

[1623] Generation and grading of quizzes and regular tests

[1624] Step 1:

[1625] The server automatically generates appropriate test questions using an AI model based on the learning progress data.

[1626] The generated tests are saved in digital format.

[1627] Step 2:

[1628] The server sends the generated test questions to the terminal.

[1629] Users (students) take the test through their devices.

[1630] Step 3:

[1631] The user (student) enters their answer and sends it to the server using their device.

[1632] The answers will be formatted appropriately for OCR processing.

[1633] Step 4:

[1634] The server digitizes the answers using an OCR engine and performs automatic scoring using an AI model.

[1635] The scoring results are saved in a database and immediately sent to the device as feedback.

[1636] Step 5:

[1637] The device displays the scoring results to the user (student) and suggests the next learning steps.

[1638] Question and Answer Session

[1639] Step 1:

[1640] The user (student) enters their question using the terminal's question-and-answer interface.

[1641] The terminal sends the question to the server.

[1642] Step 2:

[1643] The server receives the question and analyzes its content using an AI model.

[1644] The system generates the optimal response based on the analysis results.

[1645] Step 3:

[1646] The server sends the generated response to the terminal.

[1647] The device displays the answer to the user (student).

[1648] Provision of teaching materials and learning plans

[1649] Step 1:

[1650] The server generates individually optimized learning materials and study plans based on the learner's progress data.

[1651] The AI ​​model calculates the next learning goal and recommended learning time.

[1652] Step 2:

[1653] The server sends the generated learning materials and study plans to the terminal.

[1654] The terminal displays the received plan and learning materials to the user (student).

[1655] The above describes the specific operations at each processing step of the system. This system makes it possible to provide an optimal learning experience for each individual learner.

[1656] (Example 1)

[1657] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1658] Traditional online learning systems have difficulty individually tracking the progress of each learner and providing them with the most suitable learning materials and teaching methods. Furthermore, manual test creation and grading are time-consuming, making it difficult to quickly assess learners' understanding. Additionally, real-time question-and-answer sessions are not possible, preventing immediate resolution of learners' questions.

[1659] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[1660] In this invention, the server includes means for tracking learner progress data, means for generating optimized lesson information based on the progress data, means for distributing the lesson information to learners, means for automatically generating quizzes or periodic tests for learners, means for performing optical character recognition (OCR) processing on the answers to the tests, means for automatically scoring the OCR-processed answers, means for generating appropriate answers to learners' questions, means for providing learners with optimal learning materials and learning plans, means for learners to engage in learning activities using a terminal, means for saving the learners' progress in a database using the progress data and analyzing it using an AI model, means for using an avatar and a voice generation engine to generate the lesson videos, and means for performing OCR processing on the answers and analyzing and displaying the scoring results using an AI model. This makes it possible to efficiently provide individually optimized educational resources and realize flexible instruction according to the learners' progress. Furthermore, the automation of test generation and scoring enables rapid and accurate learning assessment. Real-time question and answer allows learners' questions to be resolved immediately.

[1661] "Learner progress data" refers to data that includes information such as the learning activities undertaken by learners, test results, and assignment submission status.

[1662] "Lesson information" refers to educational content optimized based on learner progress data, and includes visual and audio lesson videos.

[1663] "Automatic generation of quizzes or periodic tests" refers to the process of automatically creating appropriate test questions based on learner progress data.

[1664] Optical character recognition (OCR) is a technology that converts handwritten or printed text into digital data.

[1665] "Automatic scoring" is a process that automatically evaluates and scores test answers converted through optical character recognition using an algorithm.

[1666] "Generating appropriate answers" means that the AI ​​model generates the best possible answer to a question submitted by the learner via their device.

[1667] "Providing optimal learning materials and study plans" means proposing and providing individually optimized learning materials and study plans based on the learner's progress data.

[1668] "Device" refers to a device used by learners to conduct learning activities, and includes, for example, computers, tablets, and smartphones.

[1669] An "avatar" is a visual character used in lesson videos to convey educational content in a visually easy-to-understand way.

[1670] A "speech generation engine" is a technology that converts text into natural-sounding speech and uses it to provide audio explanations in educational videos.

[1671] An "AI model" is a technology that uses machine learning algorithms to perform tasks such as analyzing progress data, automatically grading tests, and generating answers to questions.

[1672] This invention relates to an AI-powered online learning system that tracks learners' progress and provides individually optimized educational resources. This system primarily consists of three components: a server, a terminal, and a user (learner). Specific embodiments are described below.

[1673] Tracking learning progress

[1674] server

[1675] The server collects data such as learners' learning activities, test results, and assignment submission status, and records it in a database. The server also uses AI models (such as PyTorch or TensorFlow) to analyze this data and understand the learners' progress. For example, when a learner watches a math lesson video and then submits an assignment, the device automatically sends this information to the server, which then updates the progress status and saves it in the database.

[1676] terminal

[1677] When a user (learner) logs in, the device sends authentication information to the server. If authentication is successful, the learning dashboard sent from the server is displayed. This dashboard displays the learner's progress, the status of submitted assignments, and other information.

[1678] Generation and distribution of lesson videos

[1679] server

[1680] The server uses an AI model based on the learner's progress data to generate individually optimized lesson videos. These lesson videos are visually easy to understand using avatars (e.g., Vyond) and include voice explanations using a speech generation engine (e.g., Google Text-to-Speech). For example, if a learner struggles with "quadratic equations," the server generates a lesson video specifically tailored to quadratic equations based on past data, adds voice explanations, and then delivers it to the device.

[1681] terminal

[1682] The terminal displays lesson videos received from the server to the learner. By watching these videos, learners can receive personalized instruction.

[1683] Generation and grading of quizzes and regular tests

[1684] server

[1685] The server automatically generates appropriate quizzes or periodic tests based on learning progress data. This allows for effective assessment of learners' understanding. For example, if a learner wants to take a test on "quadratic equations," the server uses an AI model to automatically generate the questions and delivers them to the device. Once the learner enters their answers and the device sends them to the server, the server uses an OCR tool (e.g., Tesseract) to convert the handwritten answers into digital data and automatically grades them.

[1686] terminal

[1687] The terminal displays test questions sent from the server and provides an interface for learners to input their answers. Once the learner completes the test, the terminal sends the answers to the server, and the server displays the graded results in real time.

[1688] Question and Answer Session

[1689] server

[1690] The server has an AI model (e.g., OpenAI GPT-3) installed for question-and-answer sessions. When a learner submits a question, the server analyzes the question and generates the most appropriate answer. For example, if a learner enters a question about "how to solve equations," the server uses the AI ​​model to analyze the question, generates an appropriate answer, and provides it to the learner via their terminal.

[1691] terminal

[1692] When a learner enters a question using the terminal's question-and-answer interface, the terminal sends the question to the server. The answer provided by the server is then displayed to the learner through the terminal.

[1693] Provision of teaching materials and learning plans

[1694] server

[1695] The server generates individually optimized learning materials and study plans based on learners' progress data. These study plans include which subjects to focus on and recommended study times. For example, the server analyzes each learner's progress data and generates a study plan that includes subjects to focus on and recommended study times for the following week.

[1696] terminal

[1697] The terminal displays learning materials and study plans received from the server to the learner. This allows the learner to study efficiently.

[1698] Example of a prompt

[1699] "Generate answers to questions from learners about how to solve equations."

[1700] Please explain the process for creating instructional videos on quadratic equations.

[1701] "Please explain how to generate a learning plan based on learner progress data."

[1702] The above describes the embodiment of this invention. This system efficiently provides individually optimized educational resources and enables flexible instruction tailored to the learner's progress. Furthermore, the automation of test generation and scoring allows for rapid and accurate learning assessment. Real-time question and answer allows learners' questions to be resolved immediately.

[1703] The flow of the specific processing in Example 1 will be explained using Figure 11.

[1704] Step 1:

[1705] The user (learner) logs into the device.

[1706] Input: Enter your User ID and password on the device's login screen.

[1707] Action: The terminal sends the entered authentication information to the server.

[1708] Output: The server performs login authentication and returns the authentication result to the terminal.

[1709] Step 2:

[1710] The server authenticates the learner.

[1711] Input: User ID and password sent from the terminal.

[1712] Operation: The server compares the user information in the database and performs authentication.

[1713] Output: If authentication is successful, a login success response is sent to the device, and data is sent to display the learning dashboard.

[1714] Step 3:

[1715] Learners engage in learning activities (watching videos, submitting assignments, etc.).

[1716] Input: The user (learner) selects and watches a lesson video, or enters their answers to an assignment.

[1717] Operation: The device collects learner activity data (viewing history, assignment submission information, etc.).

[1718] Output: The device sends the collected learning activity data to the server.

[1719] Step 4:

[1720] The device sends the training data to the server.

[1721] Input: Learner's video viewing history and assignment submission information.

[1722] Operation: The device automatically sends this data to the server.

[1723] Output: The server receives the learning activity data.

[1724] Step 5:

[1725] The server saves the data to the database.

[1726] Input: Learning activity data sent from the device.

[1727] Operation: The server records and updates data in the database.

[1728] Output: Progress data in the database is updated.

[1729] Step 6:

[1730] The server uses an AI model to analyze the data and update the progress status.

[1731] Input: Learning activity data and progress data from the database.

[1732] Operation: The server uses an AI model (e.g., PyTorch or TensorFlow) to perform data analysis.

[1733] Output: As a result of the analysis, the individual learner's progress is updated and reflected in the dashboard.

[1734] Step 7:

[1735] The server optimizes the content of the lesson videos based on the learner's progress data.

[1736] Input: Learner progress data from the database.

[1737] Operation: The server uses an AI model (e.g., TensorFlow) to determine the optimal content for the lesson videos.

[1738] Output: The content of the generated lesson video (for example, content specifically focused on quadratic equations).

[1739] Step 8:

[1740] The server generates lesson videos using avatars and a voice generation engine.

[1741] Input: Optimized lesson video content.

[1742] Operation: The server generates a visual avatar using an animation tool (e.g., Vyond) and adds a voice description using a speech generation engine (e.g., Google Text-to-Speech).

[1743] Output: Generated lesson video.

[1744] Step 9:

[1745] The server generates and delivers lesson videos to the devices.

[1746] Input: Generated lesson video.

[1747] Operation: The server streams the lesson videos to the terminals.

[1748] Output: Lecture video delivered to the device.

[1749] Step 10:

[1750] The device displays the video to the learner.

[1751] Input: Lecture videos streamed from the server.

[1752] Operation: The device plays the lesson video and allows the learner to watch it.

[1753] Output: Learners watch the lesson videos.

[1754] Step 11:

[1755] The server automatically generates quizzes or periodic tests based on learning progress data.

[1756] Input: Learner progress data.

[1757] Operation: The server uses an AI model (e.g., SciKit-Learn) to automatically generate appropriate test questions.

[1758] Output: Generated test questions.

[1759] Step 12:

[1760] The server delivers the test questions to the terminals.

[1761] Input: Generated test questions.

[1762] Operation: The server sends the test questions to the terminal.

[1763] Output: Test questions delivered to the terminal.

[1764] Step 13:

[1765] The learners answer the test questions.

[1766] Input: Test question.

[1767] Operation: The user (learner) answers test questions on the device and inputs the answers into the device.

[1768] Output: Learner's answer.

[1769] Step 14:

[1770] The device sends the answer to the server.

[1771] Input: Learner's answer.

[1772] Operation: The device sends the collected answer data to the server.

[1773] Output: Answer data sent to the server.

[1774] Step 15:

[1775] The server digitizes the answers using optical character recognition (OCR).

[1776] Input: Submitted answer data.

[1777] Operation: The server uses an OCR tool (e.g., Tesseract) to convert handwritten answers into digital data.

[1778] Output: Digitized answer data.

[1779] Step 16:

[1780] The server uses an AI model to automatically score the results.

[1781] Input: Digitized answer data.

[1782] Operation: The server uses an AI model (e.g., SciKit-Learn) to automatically grade the answers.

[1783] Output: Scoring results.

[1784] Step 17:

[1785] The server delivers the scoring results to the terminal.

[1786] Input: Scoring result.

[1787] Operation: The server sends the scoring results to the terminal in real time.

[1788] Output: The scoring result displayed on the device.

[1789] Step 18:

[1790] The learner enters the question from their device.

[1791] Input: Learner's question.

[1792] Operation: The user (learner) enters a question using a question-and-answer interface.

[1793] Output: Question data sent from the terminal to the server.

[1794] Step 19:

[1795] The terminal sends the question to the server.

[1796] Input: Question data.

[1797] Operation: The terminal sends the question data to the server.

[1798] Output: Question data received by the server.

[1799] Step 20:

[1800] The server analyzes the question.

[1801] Input: Received question data.

[1802] Operation: The server uses an AI model (e.g., OpenAI GPT-3) to analyze the question.

[1803] Output: Analysis results.

[1804] Step 21:

[1805] The server generates the answer.

[1806] Input: Analysis results of the question.

[1807] Operation: The server generates the optimal answer.

[1808] Output: Generated answer.

[1809] Step 22:

[1810] The server sends the response to the terminal.

[1811] Input: Generated response.

[1812] Operation: The server sends the response to the terminal.

[1813] Output: The response received by the terminal.

[1814] Step 23:

[1815] The device displays the answer to the learner.

[1816] Input: Response sent from the server.

[1817] Operation: The device displays the answer to the learner.

[1818] Output: Learners see the answers.

[1819] Step 24:

[1820] The server generates a learning plan based on the learner's progress data.

[1821] Input: Learner progress data.

[1822] Operation: The server uses the AI ​​model to generate an individually optimized learning plan.

[1823] Output: The generated training plan.

[1824] Step 25:

[1825] The server delivers the learning plan to the device.

[1826] Input: The generated training plan.

[1827] Operation: The server sends the learning plan to the terminal.

[1828] Output: The learning plan displayed on the device.

[1829] The above outlines the specific processing flow of this system. Each step involves specific actions, enabling the efficient provision of individually optimized educational resources and flexible instruction tailored to the learner's progress.

[1830] (Application Example 1)

[1831] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1832] Traditional online learning systems have struggled to effectively track learners' progress and provide individually optimized educational resources. Furthermore, they lacked the means to create a more intuitive and immersive learning environment through virtual learning experiences. Additionally, automatically generating and immediately evaluating optimal quizzes and periodic tests based on learners' progress was difficult.

[1833] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[1834] In this invention, the server includes means for tracking learner progress data, means for generating optimized lesson content based on the progress data, means for delivering the lesson content to learners, means for automatically generating quizzes or periodic tests for learners in a virtual environment, means for optical character recognition processing of the test answers, means for automatically scoring the optical character recognition processed answers, means for generating appropriate answers to learners' questions, means for providing learners with optimal learning resources and learning plans, and a learning experience in a virtual environment. This makes it possible to grasp learners' progress in real time and provide individually optimized educational resources. Furthermore, a more intuitive and immersive learning experience can be achieved through the learning experience in the virtual environment.

[1835] "Learner progress data" refers to data generated when learners engage in learning activities, and includes information such as the content of lessons viewed, assignments submitted, and test results.

[1836] "Lesson content" refers to learning materials provided to learners, including learning resources in formats such as videos, audio, and text.

[1837] A "virtual environment" refers to a virtual learning space created using computer simulations, where learners can engage in learning activities while immersed in the environment, just as they would in the real world.

[1838] A "quiz or periodic test" is an examination administered periodically to measure a learner's understanding of the material, and includes both small-scale review tests and tests that assess long-term learning outcomes.

[1839] Optical Character Recognition (OCR) is a technology that reads text information from scanned documents and images, and is a process for converting them into a digital format.

[1840] "Automated scoring" refers to the process of using AI technology to mechanically evaluate test answers and assign scores.

[1841] "Means for generating appropriate answers" refers to technology in which AI automatically generates the most effective answers based on the learner's questions.

[1842] "Learning resources" refer to educational content such as textbooks, videos, and audio guides that learners use to study efficiently.

[1843] A "learning plan" is a schedule or guideline that shows how learners should proceed with their studies within a certain period of time.

[1844] This invention relates to an AI-powered online learning system that tracks learners' progress and provides individually optimized educational resources within a virtual environment based on that progress. Specific embodiments are described below.

[1845] Tracking learning progress

[1846] server

[1847] The server collects data such as learners' learning activities, test results, and assignment submission status, and records it in a database. It also uses a generative AI model to analyze this data and understand the learners' progress. This allows for the provision of instruction tailored to each individual learner.

[1848] terminal

[1849] When a learner logs into the virtual environment, the device sends authentication information to the server. If authentication is successful, the learning dashboard sent from the server is displayed within the virtual environment. This dashboard displays the learner's progress, the status of submitted assignments, and other information.

[1850] Generation and distribution of course content

[1851] server

[1852] The server uses a generative AI model to generate individually optimized lesson content based on the learner's progress data. This lesson content is visually easy to understand using avatars and includes voice explanations using an artificial speech generation engine. Once the lesson content is generated, the server delivers it to terminals within the virtual environment.

[1853] terminal

[1854] The terminal displays lesson content received from the server to the learner within a virtual environment. By viewing this content, learners can receive individually optimized learning guidance.

[1855] Generation and grading of quizzes and regular tests

[1856] server

[1857] The server automatically generates appropriate quizzes or periodic tests based on learning progress data. This allows for effective assessment of learners' understanding. When learners take a test, the server processes the submitted answers using optical character recognition and automatically grades them using a generative AI model.

[1858] terminal

[1859] The terminal displays test questions sent from the server within a virtual environment and provides an interface for learners to input their answers. Once the learner completes the test, the terminal sends the answers to the server, and the server displays the graded results in real time.

[1860] Question and Answer Session

[1861] server

[1862] The server has a generative AI model installed for question and answering. When a learner submits a question, the server analyzes the question and generates the best possible answer.

[1863] terminal

[1864] When a learner enters a question using the terminal's question-and-answer interface, the terminal sends the question to the server. The answer provided by the server is then displayed to the learner within the virtual environment via the terminal.

[1865] Provision of teaching materials and learning plans

[1866] server

[1867] The server generates personalized learning materials and study plans based on the learner's progress data. These plans include which subjects to focus on and recommended study times.

[1868] terminal

[1869] The terminal displays learning materials and study plans received from the server to the learner within a virtual environment. This allows learners to study efficiently.

[1870] Specific example

[1871] Example of a learning progress tracking prompt message

[1872] Tracking learning progress

[1873] When a learner watches math lesson content and then submits an assignment, the device automatically sends this information to the server. The server uses this data to update the learner's progress and saves it in a database.

[1874] Generation and distribution of course content

[1875] If a learner struggles with the topic of "quadratic equations," the server generates specialized lesson content based on past data, adds an artificial voice explanation, and then delivers it to the terminal in the virtual environment.

[1876] Examples of prompt messages for generating quizzes and periodic tests.

[1877] Please generate a short quiz on "quadratic equations" for learner ID: 12345.

[1878] Examples of question-and-answer prompts

[1879] Learner ID: 12345 has a question about "How to solve equations". Please generate the best answer.

[1880] Provision of teaching materials and learning plans

[1881] The server analyzes each learner's progress data and generates a study plan that includes subjects to focus on in the following week and recommended study time. The terminal displays this plan to the learner within the virtual environment.

[1882] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[1883] Step 1:

[1884] The user logs into the virtual environment.

[1885] Input: User ID, Password

[1886] Operation: Enter your user ID and password, and the terminal will send them to the server.

[1887] Output: Authentication result

[1888] Specific operation: The server compares user information with the database and returns whether authentication was successful or not.

[1889] Step 2:

[1890] The device displays the learner's learning dashboard.

[1891] Input: Authentication result, User ID

[1892] Operation: If authentication is successful, learner progress data is retrieved from the server and a learning dashboard is generated.

[1893] Output: Learning Dashboard

[1894] Specific operation: The dashboard displays learning progress, viewed content, submitted assignments, and more.

[1895] Step 3:

[1896] The server generates optimized lesson content.

[1897] Input: Learner progress data

[1898] Operation: Uses a generative AI model to analyze learner progress data and generate optimal lesson content.

[1899] Output: Course content

[1900] Specific actions: The lesson content will include avatars and artificial voice explanations.

[1901] Step 4:

[1902] The device displays the lesson content within a virtual environment.

[1903] Input: Lesson content

[1904] Operation: Displays lecture content received from the server through a screen or avatar in a virtual environment.

[1905] Output: Lesson content viewable by learners

[1906] Specific actions: Learners watch lesson content and complete related assignments.

[1907] Step 5:

[1908] The server automatically generates short quizzes or periodic tests.

[1909] Input: Learner progress data, test requests

[1910] Operation: Generates appropriate test questions using a generative AI model.

[1911] Output: Test questions

[1912] Specific operation: The AI ​​automatically generates the most suitable problems based on progress data and provides them in the form of tests.

[1913] Step 6:

[1914] The terminal displays the test questions within the virtual environment.

[1915] Input: Test Questions

[1916] Operation: Displays test questions received from the server on the interface and accepts answer input.

[1917] Output: Learner's answer

[1918] Specific actions: The learner takes a test and enters their answers.

[1919] Step 7:

[1920] The server processes the learner's answers using optical character recognition (OCR).

[1921] Input: Learner's answer

[1922] Operation: Performs optical character recognition processing to convert answers from handwritten text or images into digital data.

[1923] Output: Digital answer data

[1924] Specific operation: The AI ​​model recognizes the answer and converts it into a format that can be stored in the database.

[1925] Step 8:

[1926] The server automatically scores the answers that have been processed using optical character recognition.

[1927] Input: Digital answer data

[1928] Operation: Use a generative AI model to score the answers and calculate the results.

[1929] Output: Scoring results

[1930] Specific operation: A score is assigned to each answer based on the scoring algorithm.

[1931] Step 9:

[1932] The device displays the scoring results in real time.

[1933] Input: Scoring result

[1934] Operation: Displays the scoring results received from the server on the dashboard.

[1935] Output: Scoring results displayed to learners

[1936] Specific operation: Scoring results are displayed on the screen in real time, allowing learners to check them.

[1937] Step 10:

[1938] The server manages the Q&A session.

[1939] Input: Learner's question

[1940] Operation: Uses a generative AI model to analyze the question and generate an appropriate answer.

[1941] Output: Answer text

[1942] Specific operation: The AI ​​receives a question from the learner, analyzes it, generates an answer, and sends it back.

[1943] Step 11:

[1944] The device displays the answers to the questions and answers.

[1945] Input: Answer text

[1946] Operation: Displays the response provided by the server within the virtual environment.

[1947] Output: Answers that learners can see.

[1948] Specific action: Learners check their answers via their devices and deepen their understanding.

[1949] Step 12:

[1950] The server generates individually optimized learning materials and study plans for each learner.

[1951] Input: Learner progress data

[1952] Operation: Uses a generative AI model to analyze data and create new learning plans and materials.

[1953] Output: Individually optimized learning materials and study plans

[1954] Specific action: A study plan is created, including recommended subjects and study time for the following week.

[1955] Step 13:

[1956] The device displays individually optimized learning materials and study plans.

[1957] Input: Course materials and study plans

[1958] Operation: Displays the learning plan and materials received from the server within the virtual environment.

[1959] Output: Learning plan and materials presented to the learner

[1960] Specific operation: The learning dashboard displays the learning plan and recommended materials for the following week, allowing learners to proceed with their studies accordingly.

[1961] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[1962] This invention combines an AI-powered online learning system that tracks learners' progress and provides individually optimized educational resources with an emotion engine that recognizes learners' emotions. The system consists of a server, a terminal, and a user (learner), each performing a specific function.

[1963] Tracking learning progress

[1964] server

[1965] The server collects data such as learners' learning activities, test results, and assignment submission status, and records it in a database. An AI model is used to analyze this data and understand the learners' progress. The system is configured to provide optimal instruction based on the learners' progress.

[1966] terminal

[1967] When a learner logs in, the device sends authentication information to the server. If authentication is successful, the learning dashboard sent from the server is displayed. This dashboard shows the learner's progress, the status of submitted assignments, and other information.

[1968] Generation and distribution of lesson videos

[1969] server

[1970] The server uses an AI model to generate individually optimized lesson videos based on the learner's progress data. The lesson videos are visually easy to understand using avatars, and audio explanations are added using a speech generation engine. The generated lesson videos are stored on the server and delivered to the user's device.

[1971] terminal

[1972] The terminal displays lesson videos received from the server to the learner. Learners can watch these videos and receive personalized instruction.

[1973] Learners' perception and response to emotions

[1974] server

[1975] The server is equipped with an emotion engine that recognizes not only the learner's progress data but also their emotional state. The emotion engine analyzes the learner's facial expression data and voice data to recognize their emotional state in real time.

[1976] terminal

[1977] The device collects the learner's facial expressions and voice through its camera and microphone and transmits them to the server. The server analyzes this data to understand the learner's emotional state.

[1978] server

[1979] The server adjusts the content and difficulty level of the lesson videos in real time based on the recognized emotional state. For example, if a learner is feeling stressed, the server can increase the amount of explanations in a gentle tone or change the content to a lower difficulty level.

[1980] Generation and grading of quizzes and regular tests

[1981] server

[1982] The server automatically generates appropriate test questions using an AI model based on learning progress data and sentiment data. When a learner takes the test, the server processes the submitted answers using OCR and automatically scores them using the AI ​​model.

[1983] terminal

[1984] The terminal displays test questions sent from the server, and the learner enters their answers. Once the test is completed, the answers are sent to the server, and the scoring results are immediately fed back.

[1985] Question and Answer Session

[1986] server

[1987] The server is equipped with an AI model for question-and-answer sessions, which generates the most appropriate answers to learners' questions. An emotion engine allows it to understand the learner's emotional state and provide answers at the right time.

[1988] terminal

[1989] When a learner enters a question using the terminal's question-and-answer interface, the terminal sends the question to the server. The server generates an answer, which is then displayed to the learner through the terminal.

[1990] Provision of teaching materials and learning plans

[1991] server

[1992] The server generates individually optimized learning materials and study plans based on the learner's progress and sentiment data. An AI model calculates the next learning objectives and recommended study time, and then creates the learning materials and study plans.

[1993] terminal

[1994] The terminal supports efficient learning by displaying learning materials and study plans received from the server to the learner.

[1995] Specific example

[1996] Tracking learning progress

[1997] When a user (learner) watches a math lesson video and then submits an assignment, the device automatically sends this information to the server. The server uses this data to update the learner's progress and saves it in a database.

[1998] Generation and distribution of lesson videos

[1999] If a user (learner) struggles with the topic of "quadratic equations," the server generates a specialized lesson video on quadratic equations based on past data, adds audio explanations, and then delivers it to the user's device.

[2000] Learners' perception and response to emotions

[2001] If a user (learner) is experiencing stress during a lesson, the device's camera detects this and notifies the server. The server's emotion engine confirms the high level of stress and adjusts the lesson video content to help the learner relax.

[2002] Generation and grading of quizzes and regular tests

[2003] If a user (learner) wants to take a test on quadratic equations, the server uses an AI model to automatically generate questions and delivers them to the device. When the learner enters their answers and the device sends them to the server, the server uses OCR to digitize the answers and the AI ​​model to grade them.

[2004] Question and Answer Session

[2005] When a user (learner) enters a question about "how to solve an equation," the device sends the question to the server. The server analyzes the question using an AI model, generates an appropriate answer, and provides it to the learner through the device.

[2006] Provision of teaching materials and learning plans

[2007] The server analyzes each learner's progress and sentiment data to generate a study plan that includes subjects to focus on in the following week and recommended study time. The terminal displays this plan to the learner.

[2008] The above describes the embodiment of this invention. This system makes it possible to provide an optimal learning experience for each individual learner. By combining it with an emotional engine, the efficiency of learning can be further improved.

[2009] The following describes the processing flow.

[2010] Tracking learning progress

[2011] Step 1:

[2012] The user (learner) logs into the online learning platform using their device.

[2013] The terminal sends the user's authentication information to the server.

[2014] Step 2:

[2015] The server verifies the authentication information, and if authentication is successful, it generates a learning dashboard and sends it to the device.

[2016] The device displays the received dashboard, showing the user their current learning progress.

[2017] Step 3:

[2018] Users (learners) watch lesson videos, take tests, and submit assignments.

[2019] The device records these learning activities and sends them to the server in real time.

[2020] Step 4:

[2021] The server records the received training data in a database and performs analysis using an AI model.

[2022] The learner's progress is updated based on the analysis results.

[2023] Generation and distribution of lesson videos

[2024] Step 1:

[2025] The server starts generating individually optimized lesson videos based on the learner's progress data.

[2026] The AI ​​model selects the appropriate content and activates the avatar and voice generation engine.

[2027] Step 2:

[2028] The server generates visually easy-to-understand videos using avatars and adds narration using a voice generation engine.

[2029] The generated lesson videos are saved on the server.

[2030] Step 3:

[2031] The server sends the generated lesson videos to the terminal.

[2032] The device displays the received video to the user (learner) and begins playback.

[2033] Learners' perception and response to emotions

[2034] Step 1:

[2035] While the user (learner) is watching the lesson video, the device uses its camera and microphone to collect the learner's facial expression data and voice data.

[2036] Step 2:

[2037] The device sends the collected emotional data to the server in real time.

[2038] The server analyzes this data to recognize the learner's emotional state.

[2039] Step 3:

[2040] The server adjusts the content and difficulty level of the lesson videos in real time based on the emotional state it recognizes.

[2041] For example, if a learner is experiencing stress, the content can be changed to a less difficult version.

[2042] Generation and grading of quizzes and regular tests

[2043] Step 1:

[2044] The server automatically generates appropriate test questions using an AI model based on learning progress data and sentiment data.

[2045] The generated tests are saved in digital format.

[2046] Step 2:

[2047] The server sends the generated test questions to the terminal.

[2048] The user (learner) takes the test through their device.

[2049] Step 3:

[2050] The user (learner) enters their answer and sends it to the server using their device.

[2051] The answers will be formatted appropriately for OCR processing.

[2052] Step 4:

[2053] The server digitizes the answers using an OCR engine and performs automatic scoring using an AI model.

[2054] The scoring results are saved in a database and immediately sent to the device as feedback.

[2055] Step 5:

[2056] The device displays the scoring results to the user (learner) and suggests the next learning steps.

[2057] Question and Answer Session

[2058] Step 1:

[2059] The user (learner) enters their question using the terminal's question-and-answer interface.

[2060] The terminal sends the question to the server.

[2061] Step 2:

[2062] The server receives the question and analyzes its content using an AI model.

[2063] The system generates the optimal response based on the analysis results.

[2064] Step 3:

[2065] The server sends the generated response to the terminal.

[2066] The device displays the answer to the user (learner).

[2067] Provision of teaching materials and learning plans

[2068] Step 1:

[2069] The server generates individually optimized learning materials and study plans based on the learner's progress data and emotional data.

[2070] The AI ​​model calculates the next learning goal and recommended learning time.

[2071] Step 2:

[2072] The server sends the generated learning materials and study plans to the terminal.

[2073] The device displays the received plan and learning materials to the user (learner).

[2074] The above describes the specific operations at each processing step of the system. This system makes it possible to provide an optimal learning experience for each individual learner. By combining it with an emotion engine, the efficiency of learning can be further improved.

[2075] (Example 2)

[2076] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[2077] Traditional online learning systems have struggled to monitor learners' progress and emotional states in real time and provide personalized learning resources accordingly. Furthermore, the lack of features to adjust instruction based on learners' emotional states made it difficult to maintain motivation and maximize learning effectiveness. There were also challenges with the generation of quizzes and periodic tests, automated grading, and the quality of question-and-answer sessions, highlighting the need for a system that provides optimal education for each individual learner.

[2078] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for tracking learner progress data, means for generating optimized lesson content based on the progress data, means for delivering the lesson content to learners, means for collecting learner facial expression data and voice data, means for analyzing the facial expression data and voice data to recognize the learner's emotional state, means for adjusting the lesson content based on the emotional state, means for automatically generating quizzes or periodic tests for learners, means for character recognition processing of the test answers, means for automatically scoring the character recognition processed answers, means for generating appropriate answers to learners' questions, and means for providing learners with optimal learning materials and learning plans. This makes it possible to grasp the learner's progress and emotional state in real time and provide optimal learning resources.

[2079] A "learner" refers to a person who uses an online learning system to study educational content.

[2080] "Progress data" refers to information recorded in digital format, such as learners' learning activities, test results, and assignment submission status.

[2081] "Lesson content" refers to educational materials provided to learners, including in the form of text, videos, and audio.

[2082] "Distribution" refers to the act of sending data in digital format from a server to a learner's device.

[2083] "Facial expression data" refers to digital information about a learner's facial expressions acquired through a camera or other recording device.

[2084] "Audio data" refers to digital information about learners' voices acquired through audio collection devices such as microphones.

[2085] "Emotional state" refers to the learner's psychological and emotional state as identified as a result of the analysis of facial expression data and voice data.

[2086] "Character recognition processing" refers to the process of analyzing characters written on paper or in image data as digital data and converting them into text information.

[2087] "Automated scoring" refers to the process of using artificial intelligence models to mechanically evaluate test answers and assign scores.

[2088] "Question answering" refers to the act of a system generating and providing appropriate answers to questions from learners.

[2089] "Educational materials" refer to educational resources provided to learners, and include forms such as textbooks, videos, and practice exercises.

[2090] A "learning plan" refers to a plan created to present learners with the most suitable learning schedule and achievement goals.

[2091] A "virtual character" is an animated character created in a digital environment, and its role is to visually explain and instruct on lesson content.

[2092] "Speech generation means" refers to technology that converts text information into speech and uses it for narration and explanations.

[2093] An "artificial intelligence model" refers to an algorithm that uses machine learning or deep learning to perform data analysis and prediction.

[2094] This invention combines an AI-powered online learning system that tracks learners' progress and provides individually optimized educational resources with an emotion engine that recognizes learners' emotions. The system consists of a server, terminals, and users (learners), each performing a specific function.

[2095] Tracking learning progress

[2096] server

[2097] The server collects data such as learners' learning activities, test results, and assignment submission status, and records it in a database. An AI model implemented in Python is used to analyze this data and evaluate the learners' progress. Based on the results of this analysis, individually optimized instructional content is generated.

[2098] terminal

[2099] When a learner logs in, the device sends authentication information to the server. If authentication is successful, the learning dashboard sent from the server is displayed. This dashboard displays learning progress, the status of submitted assignments, and other information in real time.

[2100] Generation and distribution of lesson videos

[2101] server

[2102] The server generates individually optimized lesson videos using a generative AI model based on the learner's progress data. These lesson videos are created to be visually and aurally easy to understand using virtual characters and voice generation methods (e.g., Google Text-to-Speech API). The generated videos are stored on the server and delivered to the device.

[2103] terminal

[2104] The terminal displays lesson videos received from the server to the learner. The learner can watch these videos and receive individually optimized instruction.

[2105] Learners' perception and response to emotions

[2106] terminal

[2107] While learners are watching the lesson videos, the device's camera and microphone collect facial expression and voice data from the learners and send it to the server.

[2108] server

[2109] The server uses the Microsoft Azure Face API and Amazon Alexa Voice Service to analyze this data with an emotion engine and recognize the learner's emotional state. Based on the recognition results, it adjusts the content and difficulty level of the lesson videos in real time.

[2110] Generation and grading of quizzes and regular tests

[2111] server

[2112] The server automatically generates appropriate test questions using a generative AI model based on learning progress data and sentiment data. Once a test is submitted, the answers are digitized using an OCR tool (e.g., Tesseract OCR), and the AI ​​model automatically scores them.

[2113] terminal

[2114] The terminal displays test questions sent from the server, and the learner enters their answers. After the test is completed, the answers are sent to the server, and the results are immediately fed back.

[2115] Question and Answer Session

[2116] terminal

[2117] The learner enters a question using a question-and-answer interface, and the device sends the question to the server.

[2118] server

[2119] The server uses OpenAI GPT-4 to analyze the question and generate an appropriate answer. The generated answer is sent to the terminal and displayed to the learner.

[2120] Provision of teaching materials and learning plans

[2121] server

[2122] The server analyzes progress and sentiment data to generate personalized learning materials and study plans, including next learning goals and recommended study times. An AI model then suggests the optimal learning strategy.

[2123] terminal

[2124] The terminal displays learning materials and study plans received from the server to the learner, supporting efficient learning progress.

[2125] Specific example

[2126] Tracking learning progress

[2127] Specific example: When a user (learner) watches a math lesson video and then submits an assignment, the device automatically sends this information to the server. The server updates the learner's progress based on this data and saves it in a database.

[2128] Generation and distribution of lesson videos

[2129] Specific example: If a user (learner) struggles with the topic of "quadratic equations," the server generates a video lesson specifically focused on quadratic equations based on past data, adds audio explanations, and then delivers it to the user's device.

[2130] Learners' perception and response to emotions

[2131] Specific example: If a user (learner) experiences stress during a lesson, the device's camera detects this and notifies the server. The server then uses an emotion engine to analyze the stress and adjust the lesson video and teaching content accordingly.

[2132] Generation and grading of quizzes and regular tests

[2133] Specific example: If a user (learner) wants to take a test on "quadratic equations," the server automatically generates questions using a generative AI model and delivers them to the device. When the learner enters their answer and the device sends it to the server, the server uses OCR to digitize the answer and the AI ​​model scores it.

[2134] Question and Answer Session

[2135] Specific example: When a user (learner) inputs a question about "how to solve an equation," the device sends the question to the server. The server analyzes the question using an AI model, generates an appropriate answer, and sends it back to the device.

[2136] Provision of teaching materials and learning plans

[2137] Specific example: The server analyzes each learner's progress and sentiment data and generates a study plan that includes subjects to focus on in the following week and recommended study time. The terminal displays this plan to the learner.

[2138] The flow of the specific processing in Example 2 will be explained using Figure 13.

[2139] Step 1: Learner Authentication and Login

[2140] ---

[2141] Input: Username, Password

[2142] Output: Authentication token (success), error message (failure)

[2143] terminal

[2144] The user accesses the system and enters their username and password on the login screen. The entered authentication information is sent from the terminal to the server.

[2145] server

[2146] The server compares the received authentication information with the database. If authentication is successful, it generates a session ID and sends the learning dashboard to the device. If authentication fails, it sends an error message to the device.

[2147] Specific operation: The server checks if the username and password are correct. If they are correct, it returns the learning dashboard; otherwise, it returns an error message.

[2148] Step 2: Tracking Learning Progress

[2149] ---

[2150] Input: Learning activity data, test results, assignment submission status

[2151] Output: Updated progress status, analysis results

[2152] terminal

[2153] The user opens the learning dashboard to check their progress and the status of submitted assignments. The learning progress data is then sent to the server.

[2154] server

[2155] The server receives new learning progress data and records it in the database. An AI model implemented in Python is used to analyze the data and evaluate the learner's progress. The analysis results are reflected in the learning dashboard in real time.

[2156] Specific operation: Learning activities and test results are periodically sent to the server, analyzed by an AI model, and the user's learning progress is evaluated and recorded.

[2157] Step 3: Generate and distribute lesson videos

[2158] ---

[2159] Input: Progress data

[2160] Output: Lecture video URL, video content

[2161] server

[2162] The server generates individually optimized lesson videos using a generative AI model based on the learner's progress data. It uses virtual characters and voice generation methods (e.g., Google Text-to-Speech API) to create visually and aurally easy-to-understand lesson videos. The generated videos are saved on the server, and the video URL is sent to the user's device.

[2163] terminal

[2164] The device displays the lesson video using a video player based on the URL received from the server. The user then watches this video to progress with their learning.

[2165] Specific operation: Input learning progress data into the AI ​​model and deliver the generated video to the device.

[2166] Step 4: Learner's emotional recognition and response

[2167] ---

[2168] Input: Facial expression data, audio data

[2169] Output: Emotion recognition results, adjusted lesson content

[2170] terminal

[2171] While learners are watching the lesson videos, the device's camera and microphone collect facial expression and voice data from the learners and send it to the server.

[2172] server

[2173] The server uses the Microsoft Azure Face API and Amazon Alexa Voice Service to analyze this data with an emotion engine and recognize the learner's emotional state. Based on the recognition results, it adjusts the content and difficulty level of the lesson videos in real time.

[2174] Specific operation: Collected facial expressions and audio data are sent to a server, and the lesson content is adaptively modified based on the recognition results.

[2175] Step 5: Generating and grading quizzes and periodic tests

[2176] ---

[2177] Input: Progress data, sentiment data

[2178] Output: Test questions, scoring results

[2179] server

[2180] The server automatically generates appropriate test questions using an AI model based on learning progress data and sentiment data. The generated test questions are then sent to the device.

[2181] terminal

[2182] The terminal displays the test questions, and the learner enters their answers. The answers are then sent to the server.

[2183] server

[2184] The server uses an OCR tool (e.g., Tesseract OCR) to digitize the answers and an AI model to score them. The results are then fed back to the terminal.

[2185] Specific operation: Generate test questions based on data, grade learners' answers using an AI model, and return the results.

[2186] Step 6: Handling questions and answers

[2187] ---

[2188] Input: Learner's question

[2189] Output: Answer

[2190] terminal

[2191] The user enters a question using a question-and-answer interface, and the terminal sends it to the server.

[2192] server

[2193] The server uses OpenAI GPT-4 to analyze the question and generate an appropriate answer. The generated answer is then sent to the terminal.

[2194] Specific operation: The system receives a question, sends it to the server, generates an answer using a generative AI model, and sends it back to the terminal.

[2195] Step 7: Providing learning materials and study plans

[2196] ---

[2197] Input: Progress data, sentiment data

[2198] Output: Individually optimized learning materials and study plans

[2199] server

[2200] The server analyzes progress and sentiment data to generate personalized learning materials and study plans, including next learning goals and recommended study times. An AI model suggests the optimal learning strategy. The generated learning materials and study plans are then sent to the user's device.

[2201] terminal

[2202] The device displays received learning materials and study plans on a learning dashboard, supporting users in efficiently progressing through their studies.

[2203] Specific actions: Analyze data to create optimal learning materials and study plans, and present them to the user on a learning dashboard.

[2204] (Application Example 2)

[2205] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[2206] Traditional online learning systems struggle to track learners' progress and provide optimal educational resources. Furthermore, the lack of features to recognize learners' emotional states and adjust instruction in real time limits the potential for reducing learner stress and improving learning effectiveness. Similarly, in factory settings, the absence of systems to track work progress and provide optimal training resources results in insufficient work efficiency and mental well-being.

[2207] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[2208] In this invention, the server includes means for tracking learner (or worker) progress data, means for generating optimized lesson videos (or training videos) based on the progress data, and means for delivering the lesson videos (or training videos) to the learner (or worker). This makes it possible to grasp the learner's (or worker's) progress in real time and provide individually optimized educational resources (or training resources). Furthermore, the server includes means for recognizing the learner's (or worker's) emotional state and means for adjusting the instructional content (or training content) based on the emotional state, thereby reducing learner's (or worker's) stress and improving learning (or work) efficiency.

[2209] "Progress data" refers to data that shows the progress of learners' or workers' learning or work.

[2210] "Lesson videos" refer to educational video content provided to learners.

[2211] "Training videos" refer to video content provided to workers for training purposes.

[2212] "Automatic generation" means that the system automatically generates processes or results without requiring manual operation.

[2213] "OCR processing" is the process of digitizing handwritten or printed text using optical character recognition technology.

[2214] "Automated evaluation" refers to a system that automatically scores test answers and learning results.

[2215] "Generating an answer" means that the system creates an appropriate answer to the learner's question.

[2216] "Educational materials" refer to educational resources and content used by learners.

[2217] A "learning plan" is a plan or schedule designed to help learners progress through their studies efficiently.

[2218] "Emotional state" refers to the psychological feelings and moods of learners and workers.

[2219] "Instructional content" refers to the educational material and teaching methods provided to learners.

[2220] "Training content" refers to the training materials and instructional policies provided to workers.

[2221] This invention is a system that tracks the progress of learners and workers and provides individually optimized educational and training resources. Furthermore, it includes a function to recognize the emotional state of learners and workers in real time and adjust the instructional and training content based on that state.

[2222] System Configuration

[2223] The system primarily consists of servers, terminals, and users. Each element performs a specific function, enabling the provision of efficient and effective education and training.

[2224] server

[2225] The server forms the core of the system and implements the following functions:

[2226] 1. Tracking progress data

[2227] The server tracks the learning and work progress of learners and workers. The collected progress data is recorded in a database and analyzed using an AI model.

[2228] 2. Generation of optimized lesson videos and training videos

[2229] Using AI models, individually optimized video content is generated based on progress data. This includes the use of avatars and a voice generation engine to provide content that is easy to understand both visually and aurally.

[2230] 3. Recognition of emotional state

[2231] Equipped with an emotion recognition engine, it analyzes the user's facial expression and voice data to recognize their emotional state in real time. This information is sent to a server and used to adjust instruction and training content.

[2232] 4. Automatic generation and evaluation

[2233] Quiz and periodic tests for learners are automatically generated. Furthermore, submitted test answers are digitized using OCR processing and automatically graded using an AI model.

[2234] 5. Question Answering Function

[2235] It also features an AI model for question answering, which generates appropriate answers to questions entered by learners.

[2236] 6. Provision of teaching materials and learning plans

[2237] Based on progress data and sentiment data, the system generates and provides users with optimal learning materials and study plans.

[2238] terminal

[2239] A terminal is a device that the user directly operates and has the following functions:

[2240] 1. Login and Authentication

[2241] This is used when learners or workers log in to the system and sends authentication information to the server.

[2242] 2. Displaying the dashboard

[2243] You can view the dashboard sent from the server and check the progress, status of submitted assignments, and more.

[2244] 3. Playback of video content

[2245] The system displays lecture and training videos received from the server, allowing users to view them.

[2246] 4. Collection of facial expression and voice data

[2247] The system collects the user's facial expressions and voice through cameras and microphones and transmits them to a server.

[2248] User

[2249] Users utilize the system as learners or workers and perform the following activities:

[2250] 1. Progress of learning and work

[2251] Students progress by watching individually optimized lesson and training videos.

[2252] 2. Taking the test and answering questions

[2253] Students take quizzes and regular tests, and use a question-answering interface to enter questions.

[2254] Hardware and software used

[2255] Hardware: Webcam, smartphone, tablet, PC

[2256] Software: OpenCV (image processing library), TensorFlow / Keras (AI model implementation)

[2257] Specific example

[2258] For example, when a factory worker begins training, a terminal tracks the worker's progress. The server analyzes the collected data and generates and distributes optimized training videos. It also collects the worker's facial expressions and voice through the terminal's camera and microphone, and recognizes their emotional state in real time using an emotion recognition engine. If the worker is experiencing stress, the server adjusts the training content based on that emotional state to reduce the worker's stress.

[2259] Example of a prompt

[2260] Example: "Create a program that recognizes the emotional state of workers in real time and provides the optimal training resources. Use facial expression data captured by a camera to recognize emotions and analyze them with an AI model."

[2261] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[2262] Step 1: User login and authentication

[2263] The user logs into the system using a device (smartphone, tablet, or PC). The device sends the user's authentication information to the server. The server verifies the authentication information and, if the login is successful, sends the learning dashboard to the device.

[2264] Input: User authentication information (username, password)

[2265] Output: Authentication result (learning dashboard if authentication is successful, error message if it fails)

[2266] Specific operation: When a user enters their authentication information on the login screen and clicks the "Login" button, that information is sent to the server. The server then checks the database to verify the authentication information.

[2267] Step 2: Display the learning dashboard

[2268] After successful authentication, the server generates a learning dashboard based on the user's progress data and sends it to the device. The device then displays this dashboard.

[2269] Input: User progress data

[2270] Output: Learning Dashboard

[2271] Specific operation: The server retrieves user progress data from the database and analyzes the user's learning status using an AI model. Based on the results, it generates a dashboard that displays progress, submitted assignments, next assignments, etc.

[2272] Step 3: Generate and distribute lesson videos (or training videos).

[2273] The server uses an AI model to generate individually optimized lesson videos (or training videos) based on the user's progress data. The generated videos are then delivered to the user's device.

[2274] Input: User progress data

[2275] Output: Individually optimized lesson videos (or training videos)

[2276] Specific operation: The server combines learning content, training content, avatars, and a voice generation engine to create video content. After video generation, the server sends the video data to the terminal.

[2277] Step 4: Play video content

[2278] The terminal plays lecture videos (or training videos) received from the server for the user. The user watches these videos and engages in learning or training.

[2279] Input: Individually optimized lesson videos (or training videos)

[2280] Output: The video the user will watch.

[2281] Specific operation: The device displays the received video data using a playback application. The user watches the video and proceeds with learning or training.

[2282] Step 5: Recognizing your emotional state

[2283] The device's camera and microphone are used to collect the user's facial expressions and voice data, which are then sent to a server. The server uses an emotion recognition engine to analyze the user's emotional state.

[2284] Input: User facial expression data and voice data

[2285] Output: User's emotional state

[2286] Specific operation: The device periodically activates its camera and microphone to capture facial expressions and audio data. This data is sent to a server in real time, and the server uses an emotion recognition engine to analyze the emotional state.

[2287] Step 6: Adjusting the instruction and training content

[2288] The server adjusts the content and difficulty level of lecture and training videos in real time based on the user's recognized emotional state. If the user is feeling stressed, it will increase the amount of gentle explanations and adjust the difficulty level of the content.

[2289] Input: User's emotional state

[2290] Output: Adjusted lesson video (or training video)

[2291] Specific operation: Based on the emotional state, the server performs processes such as regenerating video content or replacing parts of already delivered content. The adjusted video data is then resent to the terminal.

[2292] Step 7: Automated test generation and evaluation

[2293] The server uses an AI model to automatically generate appropriate test questions based on the user's progress and sentiment data, and delivers them to the device. When the user takes the test, the device sends the answers to the server, which performs OCR processing and scores them using the AI ​​model.

[2294] Input: User progress data, sentiment data, test answers

[2295] Output: Automated test questions, scoring results

[2296] Specific operation: The server generates appropriate test questions and delivers them to the terminal. Once the user completes the test, the terminal sends the answers to the server, which then scores them using OCR processing and an AI model.

[2297] Step 8: The World of Question Answering Functions

[2298] When a user enters a question using their device, the device sends the question to the server. The server uses an AI model to analyze the question, generates an appropriate answer, and provides it to the user through the device.

[2299] Input: User's question

[2300] Output: Response from the server

[2301] Specific operation: The user enters a question using the terminal's question-answering interface. The terminal sends this question to the server, which generates an answer based on the results of analysis by an AI model, and sends it back to the terminal for display.

[2302] Step 9: Provide learning materials and study plans.

[2303] The server generates appropriate learning materials and study plans based on learner progress data and sentiment data, and provides them to the user through the terminal.

[2304] Input: Progress data, sentiment data

[2305] Output: Individually optimized learning materials and study plans

[2306] Specific operation: Based on the collected data, the server generates a learning plan including the next learning objectives and recommended learning time, and sends it to the terminal. The terminal then displays this plan to the user.

[2307] The above outlines the specific processing steps for implementing this invention. In each step, it is possible to take appropriate action in real time based on the collected data.

[2308] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[2309] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[2310] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[2311] [Fourth Embodiment]

[2312] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[2313] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[2314] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[2315] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[2316] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[2317] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[2318] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[2319] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[2320] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[2321] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[2322] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[2323] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[2324] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[2325] This invention relates to an AI-powered online learning system that tracks learners' progress and provides individually optimized educational resources. The system consists of a server, a terminal, and a user (learner), each performing a specific function.

[2326] Tracking learning progress

[2327] server

[2328] The server collects data such as learners' learning activities, test results, and assignment submission status, and records it in a database. It also uses an AI model to analyze this data and understand the learners' progress. This allows for the provision of instruction tailored to each individual learner.

[2329] terminal

[2330] When a learner logs in, their device sends authentication information to the server. Upon successful authentication, the learning dashboard sent from the server is displayed. This dashboard shows the learner's progress, the status of submitted assignments, and other relevant information.

[2331] Generation and distribution of lesson videos

[2332] server

[2333] The server uses an AI model to generate individually optimized lesson videos based on the learner's progress data. These lesson videos are visually easy to understand using avatars and include audio explanations using a speech generation engine. Once the lesson videos are generated, the server delivers them to the user's device.

[2334] terminal

[2335] The terminal displays lesson videos received from the server to the learner. By watching these videos, learners can receive personalized instruction.

[2336] Generation and grading of quizzes and regular tests

[2337] server

[2338] The server automatically generates appropriate quizzes or periodic tests based on learning progress data. This allows for effective assessment of learners' understanding. When learners take a test, the server processes the submitted answers using OCR and automatically grades them using an AI model.

[2339] terminal

[2340] The terminal displays test questions sent from the server and provides an interface for learners to input their answers. Once the learner completes the test, the terminal sends the answers to the server, and the server displays the graded results in real time.

[2341] Question and Answer Session

[2342] server

[2343] The server has an AI model installed for question-and-answer sessions. When a learner submits a question, the server analyzes the question and generates the best possible answer.

[2344] terminal

[2345] When a learner enters a question using the terminal's question-and-answer interface, the terminal sends the question to the server. The answer provided by the server is then displayed to the learner through the terminal.

[2346] Provision of teaching materials and learning plans

[2347] server

[2348] The server generates personalized learning materials and study plans based on the learner's progress data. These plans include which subjects to focus on and recommended study times.

[2349] terminal

[2350] The terminal displays learning materials and study plans received from the server to the learner. This allows the learner to study efficiently.

[2351] Specific example

[2352] Tracking learning progress

[2353] When a user (learner) watches a math lesson video and then submits an assignment, the device automatically sends this information to the server. The server uses this data to update the learner's progress and saves it in a database.

[2354] Generation and distribution of lesson videos

[2355] If a user (learner) struggles with the topic of "quadratic equations," the server generates a specialized lesson video on quadratic equations based on past data, adds audio explanations, and then delivers it to the user's device.

[2356] Generation and grading of quizzes and regular tests

[2357] If a user (learner) wants to take a test on quadratic equations, the server uses an AI model to automatically generate questions and delivers them to the device. When the learner enters their answers and the device sends them to the server, the server uses OCR to digitize the answers and the AI ​​model to grade them.

[2358] Question and Answer Session

[2359] When a user (learner) enters a question about "how to solve an equation," the device sends the question to the server. The server analyzes the question using an AI model, generates an appropriate answer, and provides it to the learner through the device.

[2360] Provision of teaching materials and learning plans

[2361] The server analyzes each learner's progress data and generates a study plan that includes subjects to focus on in the following week and recommended study time. The terminal displays this plan to the learner.

[2362] The above describes the embodiment of this invention. This system makes it possible to efficiently and effectively provide individually optimized educational resources.

[2363] The following describes the processing flow.

[2364] Tracking learning progress

[2365] Step 1:

[2366] The user (student) logs into the online learning platform using their device.

[2367] The terminal sends the user's authentication information to the server.

[2368] Step 2:

[2369] The server verifies the authentication information, and if authentication is successful, it generates a learning dashboard and sends it to the device.

[2370] The device displays the received dashboard, showing the user their current learning progress.

[2371] Step 3:

[2372] Users (students) watch lesson videos or take tests.

[2373] The device records these learning activities and sends them to the server in real time.

[2374] Step 4:

[2375] The server records the received training data in a database and performs analysis using an AI model.

[2376] The learner's progress is updated based on the analysis results.

[2377] Generation and distribution of lesson videos

[2378] Step 1:

[2379] The server starts generating individually optimized lesson videos based on the learner's progress data.

[2380] The AI ​​model selects the appropriate content and activates the avatar and voice generation engine.

[2381] Step 2:

[2382] The server generates visually easy-to-understand videos using avatars and adds narration using a voice generation engine.

[2383] The generated lesson videos are saved on the server.

[2384] Step 3:

[2385] The server sends the generated lesson videos to the terminal.

[2386] The device displays the received video to the user (student) and begins playback.

[2387] Generation and grading of quizzes and regular tests

[2388] Step 1:

[2389] The server automatically generates appropriate test questions using an AI model based on the learning progress data.

[2390] The generated tests are saved in digital format.

[2391] Step 2:

[2392] The server sends the generated test questions to the terminal.

[2393] Users (students) take the test through their devices.

[2394] Step 3:

[2395] The user (student) enters their answer and sends it to the server using their device.

[2396] The answers will be formatted appropriately for OCR processing.

[2397] Step 4:

[2398] The server digitizes the answers using an OCR engine and performs automatic scoring using an AI model.

[2399] The scoring results are saved in a database and immediately sent to the device as feedback.

[2400] Step 5:

[2401] The device displays the scoring results to the user (student) and suggests the next learning steps.

[2402] Question and Answer Session

[2403] Step 1:

[2404] The user (student) enters their question using the terminal's question-and-answer interface.

[2405] The terminal sends the question to the server.

[2406] Step 2:

[2407] The server receives the question and analyzes its content using an AI model.

[2408] The system generates the optimal response based on the analysis results.

[2409] Step 3:

[2410] The server sends the generated response to the terminal.

[2411] The device displays the answer to the user (student).

[2412] Provision of teaching materials and learning plans

[2413] Step 1:

[2414] The server generates individually optimized learning materials and study plans based on the learner's progress data.

[2415] The AI ​​model calculates the next learning goal and recommended learning time.

[2416] Step 2:

[2417] The server sends the generated learning materials and study plans to the terminal.

[2418] The terminal displays the received plan and learning materials to the user (student).

[2419] The above describes the specific operations at each processing step of the system. This system makes it possible to provide an optimal learning experience for each individual learner.

[2420] (Example 1)

[2421] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[2422] Traditional online learning systems have difficulty individually tracking the progress of each learner and providing them with the most suitable learning materials and teaching methods. Furthermore, manual test creation and grading are time-consuming, making it difficult to quickly assess learners' understanding. Additionally, real-time question-and-answer sessions are not possible, preventing immediate resolution of learners' questions.

[2423] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[2424] In this invention, the server includes means for tracking learner progress data, means for generating optimized lesson information based on the progress data, means for distributing the lesson information to learners, means for automatically generating quizzes or periodic tests for learners, means for performing optical character recognition (OCR) processing on the answers to the tests, means for automatically scoring the OCR-processed answers, means for generating appropriate answers to learners' questions, means for providing learners with optimal learning materials and learning plans, means for learners to engage in learning activities using a terminal, means for saving the learners' progress in a database using the progress data and analyzing it using an AI model, means for using an avatar and a voice generation engine to generate the lesson videos, and means for performing OCR processing on the answers and analyzing and displaying the scoring results using an AI model. This makes it possible to efficiently provide individually optimized educational resources and realize flexible instruction according to the learners' progress. Furthermore, the automation of test generation and scoring enables rapid and accurate learning assessment. Real-time question and answer allows learners' questions to be resolved immediately.

[2425] "Learner progress data" refers to data that includes information such as the learning activities undertaken by learners, test results, and assignment submission status.

[2426] "Lesson information" refers to educational content optimized based on learner progress data, and includes visual and audio lesson videos.

[2427] "Automatic generation of quizzes or periodic tests" refers to the process of automatically creating appropriate test questions based on learner progress data.

[2428] Optical character recognition (OCR) is a technology that converts handwritten or printed text into digital data.

[2429] "Automatic scoring" is a process that automatically evaluates and scores test answers converted through optical character recognition using an algorithm.

[2430] "Generating appropriate answers" means that the AI ​​model generates the best possible answer to a question submitted by the learner via their device.

[2431] "Providing optimal learning materials and study plans" means proposing and providing individually optimized learning materials and study plans based on the learner's progress data.

[2432] "Device" refers to a device used by learners to conduct learning activities, and includes, for example, computers, tablets, and smartphones.

[2433] An "avatar" is a visual character used in lesson videos to convey educational content in a visually easy-to-understand way.

[2434] A "speech generation engine" is a technology that converts text into natural-sounding speech and uses it to provide audio explanations in educational videos.

[2435] An "AI model" is a technology that uses machine learning algorithms to perform tasks such as analyzing progress data, automatically grading tests, and generating answers to questions.

[2436] This invention relates to an AI-powered online learning system that tracks learners' progress and provides individually optimized educational resources. This system primarily consists of three components: a server, a terminal, and a user (learner). Specific embodiments are described below.

[2437] Tracking learning progress

[2438] server

[2439] The server collects data such as learners' learning activities, test results, and assignment submission status, and records it in a database. The server also uses AI models (such as PyTorch or TensorFlow) to analyze this data and understand the learners' progress. For example, when a learner watches a math lesson video and then submits an assignment, the device automatically sends this information to the server, which then updates the progress status and saves it in the database.

[2440] terminal

[2441] When a user (learner) logs in, the device sends authentication information to the server. If authentication is successful, the learning dashboard sent from the server is displayed. This dashboard displays the learner's progress, the status of submitted assignments, and other information.

[2442] Generation and distribution of lesson videos

[2443] server

[2444] The server uses an AI model based on the learner's progress data to generate individually optimized lesson videos. These lesson videos are visually easy to understand using avatars (e.g., Vyond) and include voice explanations using a speech generation engine (e.g., Google Text-to-Speech). For example, if a learner struggles with "quadratic equations," the server generates a lesson video specifically tailored to quadratic equations based on past data, adds voice explanations, and then delivers it to the device.

[2445] terminal

[2446] The terminal displays lesson videos received from the server to the learner. By watching these videos, learners can receive personalized instruction.

[2447] Generation and grading of quizzes and regular tests

[2448] server

[2449] The server automatically generates appropriate quizzes or periodic tests based on learning progress data. This allows for effective assessment of learners' understanding. For example, if a learner wants to take a test on "quadratic equations," the server uses an AI model to automatically generate the questions and delivers them to the device. Once the learner enters their answers and the device sends them to the server, the server uses an OCR tool (e.g., Tesseract) to convert the handwritten answers into digital data and automatically grades them.

[2450] terminal

[2451] The terminal displays test questions sent from the server and provides an interface for learners to input their answers. Once the learner completes the test, the terminal sends the answers to the server, and the server displays the graded results in real time.

[2452] Question and Answer Session

[2453] server

[2454] The server has an AI model (e.g., OpenAI GPT-3) installed for question-and-answer sessions. When a learner submits a question, the server analyzes the question and generates the most appropriate answer. For example, if a learner enters a question about "how to solve equations," the server uses the AI ​​model to analyze the question, generates an appropriate answer, and provides it to the learner via their terminal.

[2455] terminal

[2456] When a learner enters a question using the terminal's question-and-answer interface, the terminal sends the question to the server. The answer provided by the server is then displayed to the learner through the terminal.

[2457] Provision of teaching materials and learning plans

[2458] server

[2459] The server generates individually optimized learning materials and study plans based on learners' progress data. These study plans include which subjects to focus on and recommended study times. For example, the server analyzes each learner's progress data and generates a study plan that includes subjects to focus on and recommended study times for the following week.

[2460] terminal

[2461] The terminal displays learning materials and study plans received from the server to the learner. This allows the learner to study efficiently.

[2462] Example of a prompt

[2463] "Generate answers to questions from learners about how to solve equations."

[2464] Please explain the process for creating instructional videos on quadratic equations.

[2465] "Please explain how to generate a learning plan based on learner progress data."

[2466] The above describes the embodiment of this invention. This system efficiently provides individually optimized educational resources and enables flexible instruction tailored to the learner's progress. Furthermore, the automation of test generation and scoring allows for rapid and accurate learning assessment. Real-time question and answer allows learners' questions to be resolved immediately.

[2467] The flow of the specific processing in Example 1 will be explained using Figure 11.

[2468] Step 1:

[2469] The user (learner) logs into the device.

[2470] Input: Enter your User ID and password on the device's login screen.

[2471] Action: The terminal sends the entered authentication information to the server.

[2472] Output: The server performs login authentication and returns the authentication result to the terminal.

[2473] Step 2:

[2474] The server authenticates the learner.

[2475] Input: User ID and password sent from the terminal.

[2476] Operation: The server compares the user information in the database and performs authentication.

[2477] Output: If authentication is successful, a login success response is sent to the device, and data is sent to display the learning dashboard.

[2478] Step 3:

[2479] Learners engage in learning activities (watching videos, submitting assignments, etc.).

[2480] Input: The user (learner) selects and watches a lesson video, or enters their answers to an assignment.

[2481] Operation: The device collects learner activity data (viewing history, assignment submission information, etc.).

[2482] Output: The device sends the collected learning activity data to the server.

[2483] Step 4:

[2484] The device sends the training data to the server.

[2485] Input: Learner's video viewing history and assignment submission information.

[2486] Operation: The device automatically sends this data to the server.

[2487] Output: The server receives the learning activity data.

[2488] Step 5:

[2489] The server saves the data to the database.

[2490] Input: Learning activity data sent from the device.

[2491] Operation: The server records and updates data in the database.

[2492] Output: Progress data in the database is updated.

[2493] Step 6:

[2494] The server uses an AI model to analyze the data and update the progress status.

[2495] Input: Learning activity data and progress data from the database.

[2496] Operation: The server uses an AI model (e.g., PyTorch or TensorFlow) to perform data analysis.

[2497] Output: As a result of the analysis, the individual learner's progress is updated and reflected in the dashboard.

[2498] Step 7:

[2499] The server optimizes the content of the lesson videos based on the learner's progress data.

[2500] Input: Learner progress data from the database.

[2501] Operation: The server uses an AI model (e.g., TensorFlow) to determine the optimal content for the lesson videos.

[2502] Output: The content of the generated lesson video (for example, content specifically focused on quadratic equations).

[2503] Step 8:

[2504] The server generates lesson videos using avatars and a voice generation engine.

[2505] Input: Optimized lesson video content.

[2506] Operation: The server generates a visual avatar using an animation tool (e.g., Vyond) and adds a voice d...

Claims

1. A means of tracking learners' progress data, A means for generating optimized lesson videos based on the aforementioned progress data, A means of distributing the aforementioned lesson videos to learners, A means for automatically generating quizzes or periodic tests for learners, A means for performing OCR processing on the answers to the aforementioned test, A means for automatically scoring the OCR-processed answers, A means of generating appropriate answers to learners' questions, A means of providing learners with the most suitable learning materials and study plans, A system that includes this.

2. The system according to claim 1, wherein an avatar and a voice generation engine are used to generate the aforementioned lesson videos.

3. The system according to claim 1, characterized in that the OCR processing is performed on digitized test answers using an AI model.

Images