system

Abstract

Provide a system. The present invention includes a means for receiving and analyzing textbook data from a user terminal that uploads the textbook data; A means to automatically generate learning curricula based on the analyzed textbook data, and A means for providing easy-to-understand lectures based on the generated learning curriculum; a means for generating and providing quizzes and questions to test comprehension related to the lecture; A means for receiving and analyzing the results of the comprehension check from the user terminal; A means to provide additional lectures or follow-ups based on the results of the checks; A means for managing the overall progress of the system and proposing appropriate learning units; A system including:

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In the past, many parents wanted to teach their children, but were unsure of their own comprehension or teaching methods, leaving their children to do all the studying themselves. Furthermore, when children continue to study on their own, it is often difficult to maintain motivation, and their level of understanding is often limited. For parents and children to study together and for parents to support their children, an effective and easy-to-understand learning support system is needed, but existing systems have not provided sufficient support. [Means for solving the problem]

[0005] The present invention provides a means for receiving and analyzing textbook data from a user terminal that uploads the textbook data. It also incorporates a means for automatically generating a learning curriculum based on the analyzed textbook data. It also provides a means for providing easy-to-understand lectures based on the generated learning curriculum, and includes a means for generating and providing quizzes and questions to check comprehension related to the lecture. The system also includes a means for receiving and analyzing the results of the comprehension check from the user terminal, and a means for providing additional lectures or follow-ups based on the check results. The system also includes a means for recording the user's learning history and comprehension and reflecting this in the next lesson, and a means for providing a user interface for parents and children to study together, thereby enabling parents and children to study together effectively.

[0006] "Textbook data" is text data containing the contents of teaching materials used in educational institutions such as schools.

[0007] A "user terminal" is a device such as a computer, tablet, or smartphone that a child or parent uses to access the learning system.

[0008] "Analysis means" refers to technology or algorithms that decode uploaded data and extract structural information and content.

[0009] A "curriculum" is a set of topics or units of study planned for educational purposes.

[0010] "Automatic generation" refers to technology in which a system, through a program, mechanically generates data or content based on a set of instructions or algorithms.

[0011] A "lecture" is a lecture-style content that provides educational information to learners, and includes text, video, audio, etc.

[0012] A "comprehension check" is a test or quiz that assesses how well a learner has understood the content they have learned.

[0013] "Check results" refers to the scores and feedback obtained in the comprehension check.

[0014] "Following up" is the act of supplementing learning by providing additional explanations or learning materials for areas where understanding is insufficient.

[0015] "Progress" is information that indicates the progress of learning and the state of achievement of learning goals.

[0016] A "learning unit" is a group of learning content related to a single theme or topic within the learning curriculum.

[0017] A "user interface" is the means or method by which a user interacts with a computer system. [Brief explanation of the drawings]

[0018] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0019] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0020] First, the terms used in the following description will be explained.

[0021] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0022] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0023] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0024] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0025] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0026] [First embodiment]

[0027] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0028] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0029] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0030] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0031] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0032] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0033] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0034] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0035] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0036] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0037] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0038] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0039] This invention is a support system for parents and children to work together to effectively advance learning. This system automatically generates learning curricula based on textbook data, and manages and supports learning progress through lectures and comprehension checks.

[0040] System configuration

[0041] Textbook data entry

[0042] The user uploads a PDF file of the textbook to the system, and the device sends the file to the server.

[0043] Textbook data analysis

[0044] The server analyzes the received textbook PDF file, extracts text data, and uses natural language processing (NLP) techniques to categorize the textbook content by chapter, section, and topic.

[0045] Learning curriculum generation

[0046] The server automatically generates a learning curriculum based on the analyzed data. Specifically, it determines learning units such as "fraction calculations" and "solving equations," and generates text lectures, videos, and quizzes for each unit.

[0047] Start a study session

[0048] Users (parents and children) log in to the system and take the initial comprehension test. The device sends the test results to the server, which analyzes the results and sets the appropriate difficulty level and starting point.

[0049] Lecture provision

[0050] The server sequentially distributes lectures created for each learning unit, and the devices display the lectures in text or video format, allowing parents and children to study together.

[0051] Check your understanding

[0052] The server generates quizzes and questions to check comprehension as the learning progresses and distributes them to the device. The user (child) answers the questions, and the device sends the results to the server. The server analyzes the results and provides additional lectures for areas where understanding is insufficient.

[0053] Follow-up and continuous learning

[0054] The server records the user's learning history and level of understanding and reflects this in the next lesson.The server also analyzes the user's learning progress and suggests the next appropriate learning unit for the user.

[0055] Specific examples

[0056] For example, if a user wants to study mathematics with his or her child, the learning process will proceed as follows:

[0057] 1. A user uploads a mathematics textbook PDF to the system.

[0058] 2. The server analyzes the textbook PDF and automatically generates units such as "Fraction Fundamentals," "Applications of Fractions," and "Solving Equations."

[0059] 3. Users (parents and children) take the initial comprehension test and send the results to the server.

[0060] 4. Based on the results, the server recommends starting with the first unit, "Fractions Basics."

[0061] 5. The device displays a video lecture on "Fraction Basics" that parents and children watch together.

[0062] 6. The server distributes a quiz to check comprehension after the lecture.

[0063] 7. The user (child) answers the quiz and sends the results to the server.

[0064] 8. The server analyzes the quiz results and provides additional lectures on areas where understanding is insufficient.

[0065] 9. The server will record progress and suggest the next unit, "Applications of Fractions."

[0066] In this way, parents and children can effectively advance their learning through the system. The present invention can reduce parental anxiety, which has been a problem in the past, and improve children's learning effectiveness.

[0067] The processing flow will be explained below.

[0068] Specific processing flow of the program

[0069] Step 1:

[0070] Uploading textbook data

[0071] Users select the textbook PDF file and drag and drop it into the system's upload form.

[0072] The terminal sends the uploaded PDF file to the server.

[0073] Step 2:

[0074] Textbook data analysis

[0075] The server analyzes the received textbook PDF file and extracts the text data.

[0076] The server uses natural language processing (NLP) techniques to categorize the textbook content by chapter, section, and topic.

[0077] Step 3:

[0078] Learning curriculum generation

[0079] Based on the analyzed data, the server generates learning units for each chapter and section.

[0080] The server associates each unit with the appropriate text lectures, video content, and quizzes.

[0081] Step 4:

[0082] User Login

[0083] Users (parents and children) log into the system.

[0084] The terminal sends the user's login information to the server for authentication.

[0085] Step 5:

[0086] First comprehension test

[0087] The server provides comprehension tests to parents and children as a preliminary step in learning.

[0088] Users (parents and children) take the test and send the results from their device to the server.

[0089] The server analyzes the received results and evaluates the user's current level of understanding.

[0090] Step 6:

[0091] Setting appropriate learning units

[0092] Based on the results of the comprehension test, the server sets the initial learning unit according to the user's level.

[0093] The server delivers the lecture for the first learning unit to the terminal.

[0094] Step 7:

[0095] Lecture provision

[0096] The terminal displays the lecture of the learning unit received from the server (e.g., text, video).

[0097] Users (parents and children) watch lectures together and progress through their learning.

[0098] Step 8:

[0099] Comprehension check

[0100] After the lecture ends, the server generates quizzes and questions to check comprehension and distributes them to the device.

[0101] The user (child) answers questions and quizzes and sends the results from the terminal to the server.

[0102] Step 9:

[0103] Analysis of check results and feedback

[0104] The server analyzes the answers to the quizzes and questions and evaluates the user's level of understanding.

[0105] The server generates and delivers additional lectures and follow-ups for areas where understanding is deemed insufficient.

[0106] Step 10:

[0107] Suggestions for the next learning unit

[0108] The server records the user's progress and level of understanding and suggests the next unit to study.

[0109] The device will display the lecture for the next learning unit and notify the user.

[0110] In this way, the system provides support at each stage of the process to help parents and children learn effectively.

[0111] Example 1

[0112] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0113] In recent years, there has been a demand for support systems that allow parents and children to study together, but conventional systems have the following issues.

[0114] 1. It is difficult to efficiently analyze textbook data and automatically generate appropriate learning programs.

[0115] 2. There is insufficient checking of the level of understanding of the learning content and follow-up on any deficiencies.

[0116] 3. Lack of interfaces for parents and children to learn together.

[0117] 4. There is a lack of functionality to administer an initial comprehension test and set a starting point for learning based on the results.

[0118] 5. It is not possible to continuously record learning history and comprehension and reflect this in the next learning session.

[0119] This makes it difficult to alleviate parents' anxiety and improve children's learning outcomes.

[0120] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0121] In this invention, the server includes means for receiving and analyzing textbook data from a user device that uploads the textbook data, means for automatically generating a study program based on the analyzed textbook data, and means for providing explanations based on the generated study program. This makes it possible to efficiently analyze complex textbook data and automatically generate an optimal study program.

[0122] In addition, in this invention, the server includes means for generating and providing questions to check the level of understanding related to the explanation, means for receiving and analyzing the results of the comprehension check from the user device, means for providing additional explanations and support based on the check results, and means for managing the overall learning progress of the system and suggesting appropriate learning units. This makes it possible to appropriately manage the user's learning status and effectively follow up on areas where understanding is lacking.

[0123] Furthermore, in this invention, the server includes means for analyzing textbook content using natural language processing technology, means for generating text explanations and videos using a generation AI module, means for conducting a comprehension test before the start of learning and setting a starting point based on the results, means for providing a user interface for parents and children to learn together, and means for recording the user's learning history and comprehension and reflecting this in the next learning session. This makes it possible to efficiently manage the entire learning process and support parent-child collaborative learning.

[0124] "Textbook data" refers to digital information on books and teaching materials used for education.

[0125] A "user device" is a terminal device operated by a user, and includes, for example, a personal computer, a tablet, a smartphone, etc.

[0126] "Analysis" refers to the act of breaking down complex data and information into an easy-to-understand form and classifying and organizing each element.

[0127] "Program of Study" refers to a set of educational plans or curricula designed to achieve specific learning objectives.

[0128] "Explanation" refers to explanations and answers that make the learning content easy to understand.

[0129] A "comprehension check" refers to a test or quiz that assesses how well the material has been understood.

[0130] "Questions" refer to questions or tasks that are used to assess comprehension of specific learning content.

[0131] "Support" refers to assistance and guidance provided to help users learn more efficiently.

[0132] "Learning progress" refers to data and records that show the progress a user is making at each stage.

[0133] "Natural language processing technology" refers to computer technology for understanding and processing human language.

[0134] "Generative AI Module" means an artificial intelligence program for automatically generating text or other content.

[0135] "Study history" refers to data that records what a user has learned and their progress to date.

[0136] "User interface" refers to the screens and operation methods designed to make it easy for users to operate the system.

[0137] "Comprehension test" refers to a test to assess a user's current level of knowledge before learning.

[0138] This invention relates to a support system for parents and children to work together to effectively advance their learning. The system of this invention automatically generates individual learning curricula using digital data from textbooks, and effectively manages and supports learning progress.

[0139] The user begins the learning process by uploading a textbook PDF file to the system. The textbook data is uploaded to the system by the user via a device, which can be a PC, tablet, or smartphone. The uploaded textbook data is then sent to the server via communication.

[0140] The server analyzes the received textbook PDF file. This analysis uses natural language processing (NLP) techniques. Specifically, it uses Tesseract OCR to extract text from PDF pages, and then uses Tensorflow (registered trademark) or spaCy to analyze the text data. This categorizes the textbook content by chapter, section, and topic.

[0141] The server automatically generates a learning curriculum based on the analysis results. The generated curriculum is divided into learning units such as "fraction calculation" and "solving equations," and for each unit, text lectures, videos, and quizzes to check comprehension are generated. OpenAI's (registered trademark) GPT-3 (registered trademark) is used to generate the text lectures, and DALL-E is used to generate the videos.

[0142] Users (parents and children) log in to the system and take an initial comprehension test. The test results are sent from the device to the server, where they are analyzed. Based on the analysis results, the starting point for learning and the optimal level of difficulty are set. The comprehension test is conducted by answering questions, and the answer data is sent to the server.

[0143] Once learning begins, the server sequentially distributes lectures created for each learning unit. The lectures are displayed on the device in text or video format, allowing parents and children to study together.

[0144] As the learning progresses, the server generates quizzes and questions to check comprehension and distributes them to the device. The user (child) answers the questions and sends the results from the device to the server. The server analyzes the results and provides additional lectures or support for areas where understanding is insufficient.

[0145] The server records the user's learning history and level of understanding and reflects this in the next lesson. This allows for detailed management of learning progress and suggests appropriate learning units to tackle next. In particular, by utilizing natural language processing technology, it is possible to effectively analyze complex textbook data and generate optimal learning curricula.

[0146] As a specific example, when a user is studying mathematics with his / her child, the learning process will proceed as follows:

[0147] 1. A user uploads a mathematics textbook PDF to the system.

[0148] 2. The server analyzes the textbook PDF and automatically generates units such as "Fraction Fundamentals," "Applications of Fractions," and "Solving Equations."

[0149] 3. Users (parents and children) take the initial comprehension test and send the results to the server.

[0150] 4. Based on the results, the server suggests starting the lesson with the first unit, "Fractions Basics."

[0151] 5. The device displays a video lecture on "Fraction Basics" that parents and children watch together.

[0152] 6. The server will deliver a quiz to check comprehension after the lecture.

[0153] 7. The user (child) answers the quiz and sends the results to the server.

[0154] 8. The server analyzes the quiz results and provides additional lectures on areas where understanding is insufficient.

[0155] 9. The server will record progress and suggest the next unit, "Applications of Fractions."

[0156] Example prompt sentence:

[0157] "Please upload the PDF file of your textbook."

[0158] "Please take the comprehension test."

[0159] "Let's begin the next unit of study."

[0160] This system can reduce parental anxiety and improve children's learning outcomes.

[0161] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0162] Step 1:

[0163] Users upload PDF files of textbooks to the system.

[0164] Input: Textbook PDF file

[0165] Output: PDF file data sent to the server

[0166] Specific operation: The user opens the browser on the terminal and clicks the "Upload textbook" button on the system. A file dialog opens, selects the textbook PDF file, and presses the "Open" button. The terminal then sends this file to the server.

[0167] Step 2:

[0168] The server analyzes the received textbook PDF file.

[0169] Input: PDF file data sent to the server

[0170] Output: Text data and data categorized into chapters, sections, and topics

[0171] What it does: The server runs a Python script that first reads each page of the PDF file as an image, then uses Tesseract OCR to convert the pages to text, then uses the spaCy library to parse the extracted text and categorize it by chapter, section, and topic.

[0172] Step 3:

[0173] The server automatically generates a learning curriculum based on the analyzed data.

[0174] Input: Parsed text data and classification data

[0175] Output: Learning curriculum and text lectures, videos, quiz data

[0176] Specific operation: The server runs an algorithm that automatically divides learning units based on the analyzed text data. Then, for each unit, it calls OpenAI's GPT-3 to generate a text lecture. It also uses DALL-E to generate related videos and a set of quiz questions.

[0177] Step 4:

[0178] Users (parents and children) log into the system and take an initial comprehension test.

[0179] Input: User ID and password, comprehension test answer data

[0180] Output: Test result data

[0181] Specific operation: The user enters their ID and password on the login screen to log in to the system. After logging in, a comprehension test screen appears, and several questions are asked. Parents and children answer the quiz together, and when they are done, the results are sent from the device to the server.

[0182] Step 5:

[0183] The server analyzes the test results and sets the appropriate difficulty level and starting point.

[0184] Input: Comprehension test result data

[0185] Output: User-specific learning starting points and difficulty settings

[0186] How it works: The server analyzes the test results and runs algorithms to determine the starting point and difficulty level. Based on these results, it creates a personalized learning plan for the user.

[0187] Step 6:

[0188] The server sequentially distributes lectures created for each learning unit.

[0189] Input: Learning curriculum data, user-specific learning plans

[0190] Output: Text lecture, video lecture

[0191] Specific operation: The server determines the next learning unit and sends the URL of the corresponding text lecture or video file to the device. The device displays the lecture in the browser, and if it is a video lecture, it will automatically play it.

[0192] Step 7:

[0193] The server generates quizzes and questions to check comprehension as the learning progresses and distributes them to the device.

[0194] Input: The user's current learning progress data

[0195] Output: Quizzes and questions to check comprehension

[0196] Specific operation: The server tracks the user's learning progress data and generates quizzes to check their comprehension. The generated quizzes are sent to the device and displayed to the user.

[0197] Step 8:

[0198] The user (child) answers the questions and sends the results from the terminal to the server.

[0199] Input: User's quiz answer data

[0200] Output: Quiz result data

[0201] Specific operation: The user (child) answers the quiz displayed on the device. When the answer is complete, the device sends the result data to the server.

[0202] Step 9:

[0203] The server analyzes the quiz results and provides additional lectures and assistance for areas where understanding is lacking.

[0204] Input: Quiz result data

[0205] Output: Additional lectures and support

[0206] How it works: The server analyzes the quiz results, runs an algorithm to identify areas of insufficient understanding, and generates additional lectures based on the results and delivers them to the device.

[0207] Step 10:

[0208] The server records the user's learning history and level of understanding and reflects this in the next learning session.

[0209] Input: Learning history data and comprehension data

[0210] Output: Next lesson plan

[0211] Specific operation: The server stores the user's learning history and level of understanding in a database and executes an algorithm to reflect this in the next learning session. It also sets the learning curriculum to be tackled next.

[0212] (Application example 1)

[0213] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0214] As part of education, there is a lack of support for parents and children to study together effectively and efficiently.In addition, it is difficult to provide an appropriate curriculum and follow-up that is tailored to each child's learning progress and level of understanding, which results in poor learning outcomes.

[0215] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0216] In this invention, the server includes means for receiving and analyzing textbook data from a user terminal that uploads the textbook data, means for automatically generating a learning curriculum based on the analyzed textbook data, and means for providing easy-to-understand lectures based on the generated learning curriculum, thereby enabling parents and children to study together effectively.

[0217] "Textbook Data" means digital data of textbooks used for educational purposes that is uploaded by a User to the System.

[0218] A "user terminal" is a device used to upload textbook data, check learning progress, and check comprehension, and typically refers to a smartphone, tablet, or PC.

[0219] "Analysis" is the process of converting uploaded textbook data into text data and categorizing the content by chapter, section, and topic.

[0220] A "learning curriculum" is a learning progress plan that is automatically generated based on analyzed textbook data, and includes specific learning units and assignments.

[0221] A "lecture" is educational content provided based on a learning curriculum and displayed to users in text or video format.

[0222] "Comprehension check" refers to quizzes and questions to check the level of understanding of the learning content after the lecture.

[0223] "Quiz generation" is the process of automatically creating quizzes and questions based on learning content using a generative AI model.

[0224] The "check result" is the result data of the comprehension check sent from the user terminal, which is analyzed by the server and reflected in the next learning content.

[0225] "Additional lectures" are educational content that is provided in addition to the parts of the lecture that are not fully understood based on the results of the comprehension check.

[0226] "Follow-up" refers to ongoing support and advice provided to users based on their learning history and level of understanding.

[0227] A "generative AI model" is an artificial intelligence model that uses machine learning technology to automatically generate quizzes and questions from text data.

[0228] "Progress" is data that indicates the progress of the user's learning, and is managed by the server.

[0229] A "learning unit" is a unit of study that is classified into a specific theme or topic within a learning curriculum.

[0230] This invention is a support system for parents and children to work together to effectively advance learning. This system automatically generates a learning curriculum based on textbook data, automatically generates quizzes to check comprehension using a generative AI model, and manages and supports learning progress through lectures and comprehension checks.

[0231] System configuration

[0232] 1. Enter textbook data

[0233] The user uploads a PDF file of the textbook to the system, and the device sends the file to the server.

[0234] 2. Analysis of textbook data

[0235] The server analyzes the received textbook PDF file, extracts text data, and uses natural language processing (NLP) technology to categorize the textbook content by chapter, section, and topic.

[0236] 3. Generating learning curriculum

[0237] The server automatically generates a learning curriculum based on the analyzed data. Specifically, it determines learning units such as "fraction calculations" and "solving equations," and generates text lectures, videos, and quizzes for each unit.

[0238] 4. Start your study session

[0239] Users (parents and children) log in to the system and take the initial comprehension test. The device sends the test results to the server, which analyzes the results and sets the appropriate difficulty level and starting point.

[0240] 5. Providing lectures

[0241] The server sequentially distributes lectures created for each learning unit, and the devices display the lectures in text or video format, allowing parents and children to study together.

[0242] 6. Generating and providing quizzes to check comprehension using generative AI models

[0243] The server automatically generates quizzes based on the learning content using a generative AI model. The device provides the generated quizzes to the user, who then answers the questions.

[0244] 7. Checking understanding and providing feedback

[0245] The user (child) answers the quiz and sends the results from the device to the server. The server analyzes the quiz results and provides additional lectures on areas where the child did not fully understand.

[0246] 8. Follow-up and continuous learning

[0247] The server records the user's learning history and level of understanding and reflects this in the next lesson.The server also analyzes the user's learning progress and suggests the next appropriate learning unit for the user.

[0248] Hardware and software:

[0249] This system runs on standard PCs and smart devices (smartphones, tablets). The software and libraries used are as follows:

[0250] PyMuPDF (for text extraction)

[0251] NLTK (for text tokenization)

[0252] Transformers (for model inference, especially Facebook's BART model)

[0253] Examples:

[0254] If a user uploads a "sixth grade math textbook," the learning process will proceed as follows:

[0255] 1. A user uploads a "6th grade mathematics textbook PDF."

[0256] 2. The server analyzes the PDF and automatically generates the units "Fundamentals of Arithmetic," "Understanding Fractions," and "Fundamentals of Equations."

[0257] 3. Users (parents and children) take the initial comprehension test and send the results to the server.

[0258] 4. Based on the results, the server recommends starting with the first unit, "Fundamentals of Mathematics."

[0259] 5. The device displays a video lecture on "Mathematics Basics" that parents and children watch together.

[0260] 6. The server automatically generates a quiz using a generative AI model to check comprehension after the lecture and distributes it to the device.

[0261] 7. The user (child) answers the quiz and sends the results to the server.

[0262] 8. The server analyzes the quiz results and provides additional lectures on areas where understanding is insufficient.

[0263] 9. The server will record progress and suggest the next unit, "Understanding Fractions."

[0264] Example prompt sentence:

[0265] Generated curriculum text:

[0266] "Understanding fractions involves the following steps: First, understand the basic concept of fractions. Then practice adding and subtracting fractions. Finally, learn how to multiply and divide fractions."

[0267] Quiz generation prompt:

[0268] "Based on this text, please formulate the following questions:"

[0269] The question that is output is:

[0270] 1. What is the basic concept of fractions?

[0271] 2. Explain adding and subtracting fractions.

[0272] 3. How do you multiply and divide fractions?

[0273] This allows parents and children to study together effectively.

[0274] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0275] Step 1:

[0276] The user uploads a PDF file of a textbook to the terminal. The input is the PDF file of the textbook, and the output is the textbook PDF data sent to the server.

[0277] Step 2:

[0278] The server analyzes the received textbook PDF file and extracts the text data. Specifically, it uses the PyMuPDF library to extract text from the PDF. In this process, the input is the PDF file and the output is the text data.

[0279] Step 3:

[0280] The server uses natural language processing techniques to classify the extracted text data into chapters, sections, and topics. It uses the NLTK library to tokenize sentences and identify each section. The input of this process is the extracted text data, and the output is the classified text data.

[0281] Step 4:

[0282] The server automatically generates a learning curriculum based on the analyzed data. Specifically, it generates learning units such as "fraction calculation" and "solving equations" from the analyzed data. The input for this process is classified text data, and the output is a learning curriculum.

[0283] Step 5:

[0284] Users (parents and children) log in to the system and take the initial comprehension test. The terminal sends the test results to the server. The input is the comprehension test questions and their answers, and the output is the answer data sent to the server.

[0285] Step 6:

[0286] The server analyzes the results of the comprehension test and sets the appropriate difficulty level and starting point. The analysis uses the received answer data to determine the initial learning unit. The input of this process is the answer data, and the output is the initial learning unit and its difficulty setting.

[0287] Step 7:

[0288] The server sequentially distributes lectures created for each learning unit. The terminal displays them to the user in text or video format. The content of the lecture is based on the learning curriculum determined in the previous step. The input of this process is the learning curriculum, and the output is the lecture content to be distributed.

[0289] Step 8:

[0290] The server uses a generative AI model to automatically generate quizzes based on the learning content. The model uses the BART model from the Transformers library and prompts to generate quizzes. The inputs to this process are the learning content and prompts, and the output is an automatically generated quiz.

[0291] Step 9:

[0292] The user (child) answers the generated quiz and sends it from the terminal to the server. The input is the answer to the quiz, and the output is the answer data sent to the server.

[0293] Step 10:

[0294] The server analyzes the quiz results and provides additional lectures for areas where the student did not fully understand the quiz. The input to this process is the answer data, and the output is additional lectures as needed.

[0295] Step 11:

[0296] The server records the user's learning history and level of understanding and suggests the next learning unit. Based on the recorded data, the server analyzes the learning history and level of understanding and provides the next learning curriculum. The input for this process is the learning history and quiz results, and the output is a suggestion for the next learning unit.

[0297] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0298] This invention is an educational support system that enables parents and children to work together to effectively advance learning, and it also includes an emotion engine that recognizes the user's emotions and customizes the learning experience. This system analyzes textbook data, automatically generates learning curricula, conducts lectures and comprehension checks, and records the user's learning history and emotional state to reflect in the next lesson.

[0299] System configuration

[0300] Textbook data entry

[0301] Users upload textbook PDF files to the system. The device sends the files to the server, which then analyzes the received textbook PDF files to extract text data. Natural language processing (NLP) techniques are used to categorize the textbook content by chapter, section, and topic.

[0302] Learning curriculum generation

[0303] The server generates learning units from the analyzed textbook data. These units include "fraction calculations" and "solving equations." Each unit is associated with a text lecture, video content, and quizzes.

[0304] Start a study session

[0305] Users (parents and children) log in to the system and take the initial comprehension test. The device sends the test results to the server, which analyzes the results and sets the appropriate difficulty level and starting point.

[0306] Lecture provision

[0307] The server delivers lectures for the designated learning unit to the device, which displays the lectures in text or video format, allowing parents and children to study together.

[0308] Check your understanding

[0309] As the learning progresses, the server generates quizzes and questions to check comprehension and distributes them to the device. The user (child) answers the questions, and the device sends the results to the server. The server analyzes the results and provides additional lectures or follow-ups for areas where understanding is insufficient.

[0310] Follow-up and continuous learning

[0311] The server records the user's learning history and level of understanding and reflects this in the next lesson. The server also analyzes the user's learning progress and suggests the appropriate next learning unit. The device displays the lecture for the next learning unit and notifies the user.

[0312] User Emotion Recognition

[0313] The server contains an emotion engine and analyzes emotions from the user's facial expressions and voice. The device's built-in camera and microphone are used to capture users' (parents and children's) emotional data in real time and send it to the server. Based on the emotion analysis results, the server adjusts the system's behavior as follows:

[0314] If the user feels stressed while learning, the server will adjust the difficulty level and provide additional support to promote understanding.

[0315] If the user shows excitement or interest, progress the learning content to maintain that state.

[0316] Motivational feedback

[0317] Based on the analysis results of the emotion engine, the server generates encouragement and feedback that matches the user's emotional state. For example, if it determines that the user is tired, it will suggest taking a break, and if they are concentrating, it will display a message encouraging them to continue. This makes the learning experience for parents and children smoother and more comfortable.

[0318] Specific examples

[0319] For example, when a user is learning mathematics with his / her child, the user goes through the following process.

[0320] 1. The user uploads a PDF of a mathematics textbook, and the server analyzes the data and generates a learning curriculum.

[0321] 2. Users (parents and children) log in, take the initial test, and send the results to the server.

[0322] 3. Based on the analysis results, the server sets the first learning unit, "Fraction Basics," and delivers the lecture to the device.

[0323] 4. The device displays the lecture, and parents and children can learn together.

[0324] 5. The server sends a quiz to check comprehension, and the child sends the answers to the server.

[0325] 6. The server provides additional lectures based on comprehension analysis and adjusts the difficulty level as needed.

[0326] 7. The server records the learning history and level of understanding and suggests the next learning unit.

[0327] 8. The device will display the lecture for the next learning unit and you can continue learning.

[0328] 9. The server analyzes the user's emotions through an emotion engine and adjusts the learning content and lecture method as needed.

[0329] 10. Based on the results of the emotion analysis, the server generates encouraging and motivating feedback and delivers it to the device.

[0330] In this way, it is possible to effectively support parents and children in their learning and improve learning efficiency by customizing the learning experience for each individual. This system can reduce parental anxiety, which has traditionally been a challenge, and improve children's learning outcomes.

[0331] The processing flow will be explained below.

[0332] Specific processing flow of the program

[0333] Step 1:

[0334] Uploading textbook data

[0335] Users select the textbook PDF file and drag and drop it into the system's upload form.

[0336] The terminal sends the uploaded PDF file to the server.

[0337] Step 2:

[0338] Textbook data analysis

[0339] The server analyzes the received textbook PDF file and extracts the text data.

[0340] The server uses natural language processing (NLP) techniques to categorize the textbook content by chapter, section, and topic.

[0341] Step 3:

[0342] Learning curriculum generation

[0343] Based on the analyzed data, the server generates learning units for each chapter and section.

[0344] The server associates each unit with the appropriate text lectures, video content, and quizzes.

[0345] Step 4:

[0346] User Login

[0347] Users (parents and children) log into the system.

[0348] The terminal sends the user's login information to the server for authentication.

[0349] Step 5:

[0350] First comprehension test

[0351] The server provides comprehension tests to parents and children as a preliminary step in learning.

[0352] Users (parents and children) take the test and send the results from their device to the server.

[0353] The server analyzes the received results and evaluates the user's current level of understanding.

[0354] Step 6:

[0355] Setting appropriate learning units

[0356] Based on the results of the comprehension test, the server sets the initial learning unit according to the user's level.

[0357] The server delivers the lecture for the first learning unit to the terminal.

[0358] Step 7:

[0359] Lecture provision

[0360] The terminal displays the lecture of the learning unit received from the server (e.g., text, video).

[0361] Users (parents and children) watch lectures together and progress through their learning.

[0362] Step 8:

[0363] Comprehension check

[0364] After the lecture ends, the server generates quizzes and questions to check comprehension and distributes them to the device.

[0365] The user (child) answers questions and quizzes and sends the results from the terminal to the server.

[0366] Step 9:

[0367] Analysis of check results and feedback

[0368] The server analyzes the answers to the quizzes and questions and evaluates the user's level of understanding.

[0369] The server generates and delivers additional lectures and follow-ups for areas where understanding is deemed insufficient.

[0370] Step 10:

[0371] Suggestions for the next learning unit

[0372] The server records the user's progress and level of understanding and suggests the next unit to study.

[0373] The device will display the lecture for the next learning unit and notify the user.

[0374] Step 11:

[0375] Using the Emotion Engine

[0376] The device uses its built-in camera and microphone to collect the user's facial expressions and voice in real time.

[0377] The terminal transmits the collected data to the server.

[0378] The server uses an emotion engine to analyze the received data and assess the user's emotional state.

[0379] Step 12:

[0380] Adjustment based on emotional state

[0381] Based on the analysis results of the emotion engine, the server adjusts the difficulty of the task if the user is feeling stressed and provides additional support to promote understanding.

[0382] If the user shows excitement or interest, the server will progress the learning content to maintain that emotion.

[0383] Step 13:

[0384] Motivational feedback

[0385] Based on the analysis results of the emotion engine, the server generates encouragement and feedback that is in line with the user's emotional state.

[0386] The device displays generated feedback messages, such as suggesting a break if you're tired, or encouraging you to keep going if you're concentrating.

[0387] In this way, the system of the present invention responds not only to the learning progress but also to the emotional state of the user, making it possible to provide an effective learning experience.

[0388] Example 2

[0389] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0390] While conventional learning support systems can measure a user's learning situation and level of understanding, they have the problem of being unable to customize the system to take into account the user's emotional state. This reduces learning efficiency and makes it difficult to maintain user motivation. Furthermore, when parents and children study together, the lack of appropriate support increases parental anxiety and makes it difficult to improve children's learning outcomes.

[0391] The specific processing by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes: means for receiving and analyzing textbook data from a user terminal that uploads the textbook data; means for automatically generating a learning curriculum based on the analyzed textbook data; means for providing easy-to-understand lectures based on the generated learning curriculum; means for generating and providing quizzes and questions to check the level of understanding related to the lecture; means for receiving and analyzing the results of the comprehension check from the user terminal; means for providing additional lectures and follow-ups based on the check results; means for managing the overall progress of the system and suggesting appropriate learning units; and means for analyzing the emotional state of the user during learning in real time and adjusting the system operation based on the results. This provides an optimal learning experience that takes into account not only the user's learning situation but also their emotional state, thereby improving learning efficiency and motivation.

[0392] "Textbook data" refers to digital document data containing content used for educational purposes.

[0393] A "user terminal" is an electronic device used for learning activities such as uploading textbook data, viewing lectures, and answering quizzes.

[0394] "Means of analysis" refers to the technology and equipment used to analyze textbook data, extract the necessary information, and classify it.

[0395] A "learning curriculum" is a set of learning content and activity plans that are established to help users effectively advance their learning.

[0396] "Means for automatic generation" refers to technology or devices for automatically creating a learning curriculum from analyzed textbook data.

[0397] A "lecture" is an explanation or instruction of the learning content provided based on the learning curriculum.

[0398] "Quizzes and questions to check comprehension" are questions provided to assess the user's level of comprehension of the learning content.

[0399] The "means for providing additional lectures or follow-up based on the check results" refers to a technology or device that analyzes the results of the check on the user's comprehension and provides further guidance or supplementary explanations as necessary.

[0400] "Means for managing progress and suggesting appropriate learning units" refers to technologies and devices for tracking a user's learning progress and suggesting what content to learn next.

[0401] "Means for analyzing emotional states in real time" refers to technology or devices for detecting and analyzing emotions in real time from a user's facial expressions, voice, etc.

[0402] "Means for adjusting the system's operation" refers to technology or devices for appropriately changing the learning content or lecture method based on the results of emotion analysis.

[0403] The present invention relates to an educational support system for parents and children to effectively collaborate in learning, and in particular, to a system including an emotion engine that recognizes the user's emotions and customizes the learning experience. This system is realized through the interaction between a server, a terminal, and a user.

[0404] Hardware and software used

[0405] The hardware in this system is as follows:

[0406] Device: A personal computer (PC) or tablet device used at home or in an educational institution.

[0407] Server: A central management device that analyzes and manages data.

[0408] Camera and microphone: Equipped on the device, they capture the user's facial expressions and voice in real time.

[0409] The software used is as follows:

[0410] Natural Language Processing (NLP) techniques: used to analyze textbook data.

[0411] Sentiment Analysis Engine: Used to analyze the user's emotional state.

[0412] OCR (Optical Character Recognition) technology: Used to convert PDF files into text data.

[0413] Various modules: Examples include PDFReceiverService, NLPAnalyzer, TextClassifier, CurriculumGenerator, TestEvaluator, LectureDistributor, LectureViewer, QuizGenerator, AnswerAnalyzer, LearningHistoryRecorder, NextUnitRecommender, EmotionAnalyzer, EmotionCapture, FeedbackGenerator, and FeedbackDisplay.

[0414] System Description

[0415] 1. Enter textbook data

[0416] A user logs into the system and uploads a textbook PDF file from their device. The device sends this file to the server, which uses a PDFReceiverService to receive, store, and prepare the file for processing.

[0417] 2. Textbook data analysis and curriculum generation

[0418] The server uses the OCRProcessor to scan PDF files and extract text data. The extracted text data is then analyzed by the NLPAnalyzer and classified by chapter, section, and topic. The CurriculumGenerator automatically generates a learning curriculum based on this data, associating each unit with text lectures, video content, and quizzes.

[0419] 3. First comprehension test

[0420] Users (parents and children) log in to the system and take the initial comprehension test on their device. The device sends the test results to the server, which analyzes them using the TestEvaluator and sets the appropriate difficulty level and starting point.

[0421] 4. Providing lectures

[0422] The server distributes text and video lectures to devices using LectureDistributor based on the set learning units. The devices display these lectures using LectureViewer, allowing parents and children to study together.

[0423] 5. Check your understanding

[0424] As the learning progresses, the server uses QuizGenerator to generate quizzes and questions to check comprehension and distributes them to the device. The user (child) answers the quiz on the device and sends the results to the server. The server analyzes the answers using AnswerAnalyzer and provides additional lectures or follow-ups for areas where understanding is insufficient.

[0425] 6. Recording learning history and suggesting next learning units

[0426] The server uses LearningHistoryRecorder to record the user's learning history and level of understanding. NextUnitRecommender then uses this information to suggest the next learning unit. The device then displays the lecture for the suggested next learning unit and notifies the user.

[0427] 7. User Emotion Recognition

[0428] The server uses the emotion engine to analyze the user's facial expressions and voice captured by the device's camera and microphone in real time. The data is acquired through EmotionCapture and analyzed by EmotionAnalyzer to adjust the system's behavior.

[0429] 8. Feedback and Motivation

[0430] The server uses the FeedbackGenerator to provide encouragement and feedback to the user based on the sentiment analysis results, and the device uses the FeedbackDisplay to display this feedback and improve the quality of the learning experience.

[0431] Specific examples

[0432] For example, when a user is learning mathematics with a child, the following process is carried out.

[0433] 1. The user uploads a PDF of a mathematics textbook, and the server analyzes the data and generates a learning curriculum.

[0434] 2. Users (parents and children) log in, take the initial test, and send the results to the server.

[0435] 3. Based on the analysis results, the server sets the first learning unit, "Fraction Basics," and delivers the lecture to the device.

[0436] 4. The device displays the lecture, and parents and children can learn together.

[0437] 5. The server sends a quiz to check comprehension, and the child sends the answers to the server.

[0438] 6. The server provides additional lectures based on comprehension analysis and adjusts the difficulty level as needed.

[0439] 7. The server records the learning history and level of understanding and suggests the next learning unit.

[0440] 8. The device will display the lecture for the next learning unit and you can continue learning.

[0441] 9. The server analyzes the user's emotions through an emotion engine and adjusts the learning content and lecture method as needed.

[0442] 10. Based on the results of the emotion analysis, the server generates encouraging and motivating feedback and delivers it to the device.

[0443] Prompt Sentence Examples

[0444] "Generate learning units from the following PDF textbooks and customize lecture content based on comprehension checks and sentiment analysis."

[0445] As a result, parents and children can study effectively, and the learning effect and motivation of users can be improved.

[0446] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0447] Step 1: Enter textbook data

[0448] Input: Textbook PDF file

[0449] Specific operation: The user clicks the "Upload textbook" button on the terminal, selects and uploads a textbook PDF file from the local file system, and the terminal uses the FileUploader module to send the selected PDF file to the server.

[0450] Output: PDF file saved on the server

[0451] Data processing: The server starts the PDFReceiverService and saves the received file in a temporary folder.

[0452] Step 2: Analyzing textbook data

[0453] Input: PDF file stored on the server

[0454] How it works: The server uses OCRProcessor to scan PDF files and extract text data, which is then analyzed by NLPAnalyzer and classified into chapters, sections, and topics.

[0455] Output: Analyzed textbook data

[0456] Data processing: The server scans the PDF data as an image and converts it into text data. The converted text data is then analyzed using natural language processing technology to classify each piece of content.

[0457] Step 3: Generate a learning curriculum

[0458] Input: Parsed textbook data

[0459] Specific operation: The server uses CurriculumGenerator to automatically generate learning units from the parsed data, and associates text lectures, video content, and quizzes with the generated learning units.

[0460] Output: Generated learning curriculum

[0461] Data processing: Based on each extracted topic, the server generates appropriate learning units and adds related educational resources (text, videos, quizzes).

[0462] Step 4: First assessment

[0463] Input: User (parent and child) login information

[0464] Specific operation: A user logs in to the system and takes the initial comprehension test on their device. The device displays the test questions using QuizPresenter and the user enters their answers. The answer data is sent to the server via AnswerCollector.

[0465] Output: Test result data

[0466] Data processing: The server uses TestEvaluator to analyze the submitted answers and set appropriate difficulty levels and starting points.

[0467] Step 5: Deliver the lecture

[0468] Input: Learning curriculum and initial comprehension test results

[0469] Specific operation: The server distributes the lectures of the set learning unit to the device using LectureDistributor. The device uses LectureViewer to display the received lectures in text or video format.

[0470] Output: Visual and audio lecture data

[0471] Data processing: The server selects and sends the appropriate lecture data (text, video), which is then displayed on the device.

[0472] Step 6: Check your understanding

[0473] Input: Learning status data after the lecture

[0474] Specific operation: The server uses QuizGenerator to generate quizzes and questions for comprehension checks and distributes them to the device. The user (child) answers the quiz on the device and sends the results to the server via AnswerCollector.

[0475] Output: Comprehension check result data

[0476] Data processing: The server uses AnswerAnalyzer to analyze the answer data and identify any missing parts of understanding.

[0477] Step 7: Follow-up and learning history

[0478] Input: Comprehension check result data

[0479] Specific operation: The server uses LearningHistoryRecorder to record the user's learning history and recommends the best learning unit for the next lesson. NextUnitRecommender selects the next learning unit and sends it from the server to the device. The device then displays the next lecture and notifies the user.

[0480] Output: Updated learning history data and next learning unit

[0481] Data processing: The server identifies the next learning step based on the recorded learning history and comprehension data, and updates the optimal learning plan.

[0482] Step 8: Recognizing User Emotions

[0483] Input: User facial and voice data

[0484] How it works: The device uses a camera and microphone to capture the user's facial expressions and voice in real time. This data is sent to the server via EmotionCapture, and the server uses EmotionAnalyzer to analyze the user's emotional state.

[0485] Output: Sentiment analysis data

[0486] Data processing: The server applies facial expression recognition and voice analysis technologies to identify the user's emotional state, such as stress, excitement, or concentration, in real time.

[0487] Step 9: Feedback and motivation

[0488] Input: Sentiment analysis data

[0489] Specific operation: The server uses FeedbackGenerator to automatically generate appropriate encouragement and feedback for the user based on the results of sentiment analysis, and the device notifies the user using FeedbackDisplay.

[0490] Output: An informative feedback message

[0491] Data processing: The server generates feedback messages that reflect the user's emotional state and presents them to the user to personalize the learning experience and increase motivation.

[0492] This detailed process flow enables the system to provide a personalized educational experience tailored to the user's learning needs.

[0493] (Application example 2)

[0494] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0495] Conventional educational support systems simply provide learning curricula and lack functionality to respond to users' emotions and individual progress. This makes it difficult to respond appropriately to declines in motivation or differences in understanding, especially in children's learning. Furthermore, when parents and children study together, the burden on parents increases, making it difficult to provide effective support.

[0496] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving and analyzing textbook data from a user terminal that uploads the textbook data, means for automatically generating a learning curriculum based on the analyzed textbook data, means for providing easy-to-understand lectures based on the generated learning curriculum, means for generating and providing quizzes and questions to check the level of understanding related to the lecture, means for receiving and analyzing the results of the comprehension check from the user terminal, means for providing additional lectures and follow-ups based on the check results, means for managing overall progress and suggesting appropriate learning units, and means for recognizing the user's emotions and customizing the learning experience. This enables flexible response according to the user's learning status, allowing parents and children to study together effectively.

[0497] "Textbook data" refers to digital data of books and teaching materials used in education.

[0498] A "user device" is an electronic device such as a computer, tablet, or smartphone used by a learner.

[0499] "Analysis" refers to the process of understanding and classifying the content of textbook data using techniques such as natural language processing.

[0500] A "curriculum" is a set of learning content and activities planned in accordance with educational objectives.

[0501] A "lecture" is educational content that provides explanations and instruction on a certain topic.

[0502] A "quiz" is a question or problem that allows you to check what you have learned.

[0503] A "comprehension check" is a process of assessing how well a learner has understood the learning content.

[0504] "Following up" means providing additional instruction or materials to reinforce areas of incomplete understanding.

[0505] "Progress" refers to the degree to which a learner has progressed in the learning curriculum.

[0506] A "learning unit" is a piece of learning organized around a specific educational goal.

[0507] "Emotion recognition" is a technology that analyzes emotions from a user's facial expressions, voice, etc.

[0508] A "learning experience" is any experience or sensation a learner has in the process of learning.

[0509] The present invention is an educational support system for parents and children to effectively collaborate in learning, and further includes an emotion engine that recognizes the user's emotions and customizes the learning experience.

[0510] System configuration

[0511] Textbook data entry

[0512] Users upload textbook PDF files to the system. The user's device sends the files to the server, which then analyzes the received textbook PDF files and extracts text data. Natural language processing (NLP) techniques are used to categorize the textbook content by chapter, section, and topic. For this purpose, Python and NLP libraries (e.g., spaCy) are used.

[0513] Learning curriculum generation

[0514] The server generates a learning curriculum from the analyzed textbook data. The generated units include "fraction calculation" and "solving equations." Each unit is associated with a text lecture, video content, and quizzes.

[0515] Start a study session

[0516] Users (parents and children) log in to the system and take the initial comprehension test. The user device sends the test results to the server, which analyzes the results and sets an appropriate difficulty level and starting point.

[0517] Lecture provision

[0518] The server delivers lectures for the selected learning unit to the user's device, which displays the lecture in text or video format, allowing parents and children to study together.

[0519] Check your understanding

[0520] As the learning progresses, the server generates quizzes and questions to check comprehension and distributes them to the user's device. The user (child) answers the questions, and the user's device sends the results to the server. The server analyzes the results and provides additional lectures or follow-ups for areas where understanding is insufficient.

[0521] Follow-up and continuous learning

[0522] The server records the user's learning history and level of understanding and reflects this in the next lesson. The server also analyzes the user's learning progress and suggests the appropriate next learning unit. The user's device displays the lecture for the next learning unit and notifies the user.

[0523] User Emotion Recognition

[0524] The server contains an emotion engine and analyzes emotions from the user's facial expressions and voice. The camera and microphone installed on the user's device are used to capture the user's (parent and child) emotional data in real time and send it to the server. Based on the emotion analysis results, the server adjusts the system's operation as follows:

[0525] If the user feels stressed while learning, the server will adjust the difficulty level and provide additional support to promote understanding.

[0526] If the user shows excitement or interest, progress the learning content to maintain that state.

[0527] Motivational feedback

[0528] Based on the analysis results of the emotion engine, the server generates encouragement and feedback that matches the user's emotional state. For example, if it determines that the user is tired, it will suggest taking a break, and if they are concentrating, it will display a message encouraging them to continue. This makes the learning experience for parents and children smoother and more comfortable.

[0529] Specific examples

[0530] For example, when a user and his / her child are learning mathematics in a virtual store, the user goes through the following process.

[0531] 1. The user uploads a PDF of a mathematics textbook, and the server analyzes the data and generates a learning curriculum.

[0532] 2. Users (parents and children) log in, take the initial test, and send the results to the server.

[0533] 3. Based on the analysis results, the server sets the first learning unit, "Fraction Basics," and delivers the lecture to the user's device.

[0534] 4. The user device displays the lecture, and the parent and child learn together.

[0535] 5. The server sends a quiz to check comprehension, and the child sends the answers to the server.

[0536] 6. The server provides additional lectures based on comprehension analysis and adjusts the difficulty level as needed.

[0537] 7. The server records the learning history and level of understanding and suggests the next learning unit.

[0538] 8. The user's device will display the lecture for the next learning unit and continue learning.

[0539] 9. The server analyzes the user's emotions through an emotion engine and adjusts the learning content and lecture method as needed.

[0540] 10. Based on the results of the emotion analysis, the server generates encouraging and motivating feedback and delivers it to the user's device.

[0541] Example prompt for a generative AI model:

[0542] "Analyze the content of the fourth grade textbook "Calculating Fractions" and generate a lecture in the following format.

[0543] 1. Lecture Title

[0544] 2. Lecture Overview

[0545] 3. List of key points

[0546] 4. Quiz questions to check comprehension and sample answers

[0547] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0548] Step 1:

[0549] The user uploads a PDF file of a textbook to the system. The user's device sends this file to the server. The input is the PDF data of the textbook, and the output is the PDF file sent to the server. Specifically, a file selection dialog opens and a request is made to send the file selected by the user to the server.

[0550] Step 2:

[0551] The server analyzes the received textbook PDF file and extracts the text data. The input is the PDF file, and the output is the analyzed text data. Specifically, the PDFMiner library is used to convert the contents of the PDF file into text format.

[0552] Step 3:

[0553] The server uses natural language processing (NLP) techniques to classify the parsed text data into chapters, sections, and topics. The input is text data, and the output is classified text data. Specifically, the spaCy library is used to split the text and store it as structured data.

[0554] Step 4:

[0555] The server automatically generates a learning curriculum from structured text data. The input is classified text data, and the output is a learning curriculum. Specifically, it automatically generates related lectures and quizzes for each chapter and compiles them into learning units.

[0556] Step 5:

[0557] Users (parents and children) log in to the system and take the initial comprehension test. The user device sends the test results to the server. The input is the answer data to the comprehension test, and the output is the test results sent for analysis. In concrete terms, the user answers the test and sends the results to the server.

[0558] Step 6:

[0559] The server analyzes the initial test results and sets the appropriate difficulty level and starting point. The input is the test results, and the output is the appropriate difficulty setting. Specifically, it determines the optimal learning level based on the test results and selects the next learning unit.

[0560] Step 7:

[0561] The server delivers lectures for the set learning units to the user's device. The input is the learning unit data, and the output is the lecture data for the user's device. Specifically, the server sends the lecture in text or video format for display on the user's device.

[0562] Step 8:

[0563] The user device displays the lecture, and parents and children can study together. The input is lecture data, and the output is the lecture displayed on the user device. In concrete terms, the user device provides a UI that displays the lecture content.

[0564] Step 9:

[0565] The server generates quizzes and questions to check comprehension and distributes them to the user's device. The input is the learning unit data, and the output is the quizzes and questions sent to the user's device. Specifically, the server generates quizzes using a Python script and sends them to the user's device.

[0566] Step 10:

[0567] The user (child) answers quizzes and questions, and the user device sends the results to the server. The input is the answer data to the quizzes and questions, and the answer results sent to the server are obtained as output. In concrete terms, the user answers the quiz and sends the results to the server.

[0568] Step 11:

[0569] The server analyzes the answer results and provides additional lectures or follow-ups for areas where understanding is insufficient. The answer result data is input, and additional lecture data is obtained as output. Specifically, the server analyzes the answer results, generates necessary supplementary explanations, and sends them to the user's device.

[0570] Step 12:

[0571] The server records the user's learning history and level of understanding and reflects this in the next lesson. The input is comprehension level and learning history data, and the output is the next learning plan. Specific operations include saving the learning history and level of understanding in a database and planning the next learning unit.

[0572] Step 13:

[0573] The server analyzes emotions from the user's facial expressions and voice. Facial expressions and voice data are input, and emotion analysis results are obtained as output. Specifically, emotions are analyzed using OpenCV and Google (registered trademark) Cloud Speech API.

[0574] Step 14:

[0575] The server adjusts the learning content and lecture method based on the emotion analysis results. The emotion analysis results are input, and the adjusted learning content is obtained as output. Specifically, the server adjusts the difficulty and progress of the learning content according to the emotional state.

[0576] Step 15:

[0577] The server generates encouraging and motivating feedback based on the emotion analysis results and delivers it to the user's device. The emotion analysis results are input, and feedback messages are obtained as output. Specifically, the server generates messages of encouragement or suggestions for taking a break based on the user's emotional state and sends them to the user's device.

[0578] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0579] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0580] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0581] [Second embodiment]

[0582] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0583] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0584] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0585] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0586] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0587] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0588] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0589] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0590] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0591] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0592] In the smart glasses 214, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0593] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0594] This invention is a support system for parents and children to work together to effectively advance learning. This system automatically generates learning curricula based on textbook data, and manages and supports learning progress through lectures and comprehension checks.

[0595] System configuration

[0596] Textbook data entry

[0597] The user uploads a PDF file of the textbook to the system, and the device sends the file to the server.

[0598] Textbook data analysis

[0599] The server analyzes the received textbook PDF file, extracts text data, and uses natural language processing (NLP) techniques to categorize the textbook content by chapter, section, and topic.

[0600] Learning curriculum generation

[0601] The server automatically generates a learning curriculum based on the analyzed data. Specifically, it determines learning units such as "fraction calculations" and "solving equations," and generates text lectures, videos, and quizzes for each unit.

[0602] Start a study session

[0603] Users (parents and children) log in to the system and take the initial comprehension test. The device sends the test results to the server, which analyzes the results and sets the appropriate difficulty level and starting point.

[0604] Lecture provision

[0605] The server sequentially distributes lectures created for each learning unit, and the devices display the lectures in text or video format, allowing parents and children to study together.

[0606] Check your understanding

[0607] The server generates quizzes and questions to check comprehension as the learning progresses and distributes them to the device. The user (child) answers the questions, and the device sends the results to the server. The server analyzes the results and provides additional lectures for areas where understanding is insufficient.

[0608] Follow-up and continuous learning

[0609] The server records the user's learning history and level of understanding and reflects this in the next lesson.The server also analyzes the user's learning progress and suggests the next appropriate learning unit for the user.

[0610] Specific examples

[0611] For example, if a user wants to study mathematics with his or her child, the learning process will proceed as follows:

[0612] 1. A user uploads a mathematics textbook PDF to the system.

[0613] 2. The server analyzes the textbook PDF and automatically generates units such as "Fraction Fundamentals," "Applications of Fractions," and "Solving Equations."

[0614] 3. Users (parents and children) take the initial comprehension test and send the results to the server.

[0615] 4. Based on the results, the server recommends starting with the first unit, "Fractions Basics."

[0616] 5. The device displays a video lecture on "Fraction Basics" that parents and children watch together.

[0617] 6. The server distributes a quiz to check comprehension after the lecture.

[0618] 7. The user (child) answers the quiz and sends the results to the server.

[0619] 8. The server analyzes the quiz results and provides additional lectures on areas where understanding is insufficient.

[0620] 9. The server will record progress and suggest the next unit, "Applications of Fractions."

[0621] In this way, parents and children can effectively advance their learning through the system. The present invention can reduce parental anxiety, which has been a problem in the past, and improve children's learning effectiveness.

[0622] The processing flow will be explained below.

[0623] Specific processing flow of the program

[0624] Step 1:

[0625] Uploading textbook data

[0626] Users select the textbook PDF file and drag and drop it into the system's upload form.

[0627] The terminal sends the uploaded PDF file to the server.

[0628] Step 2:

[0629] Textbook data analysis

[0630] The server analyzes the received textbook PDF file and extracts the text data.

[0631] The server uses natural language processing (NLP) techniques to categorize the textbook content by chapter, section, and topic.

[0632] Step 3:

[0633] Learning curriculum generation

[0634] Based on the analyzed data, the server generates learning units for each chapter and section.

[0635] The server associates each unit with the appropriate text lectures, video content, and quizzes.

[0636] Step 4:

[0637] User Login

[0638] Users (parents and children) log into the system.

[0639] The terminal sends the user's login information to the server for authentication.

[0640] Step 5:

[0641] First comprehension test

[0642] The server provides comprehension tests to parents and children as a preliminary step in learning.

[0643] Users (parents and children) take the test and send the results from their device to the server.

[0644] The server analyzes the received results and evaluates the user's current level of understanding.

[0645] Step 6:

[0646] Setting appropriate learning units

[0647] Based on the results of the comprehension test, the server sets the initial learning unit according to the user's level.

[0648] The server delivers the lecture for the first learning unit to the terminal.

[0649] Step 7:

[0650] Lecture provision

[0651] The terminal displays the lecture of the learning unit received from the server (e.g., text, video).

[0652] Users (parents and children) watch lectures together and progress through their learning.

[0653] Step 8:

[0654] Comprehension check

[0655] After the lecture ends, the server generates quizzes and questions to check comprehension and distributes them to the device.

[0656] The user (child) answers questions and quizzes and sends the results from the terminal to the server.

[0657] Step 9:

[0658] Analysis of check results and feedback

[0659] The server analyzes the answers to the quizzes and questions and evaluates the user's level of understanding.

[0660] The server generates and delivers additional lectures and follow-ups for areas where understanding is deemed insufficient.

[0661] Step 10:

[0662] Suggestions for the next learning unit

[0663] The server records the user's progress and level of understanding and suggests the next unit to study.

[0664] The device will display the lecture for the next learning unit and notify the user.

[0665] In this way, the system provides support at each stage of the process to help parents and children learn effectively.

[0666] Example 1

[0667] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0668] In recent years, there has been a demand for support systems that allow parents and children to study together, but conventional systems have the following issues.

[0669] 1. It is difficult to efficiently analyze textbook data and automatically generate appropriate learning programs.

[0670] 2. There is insufficient checking of the level of understanding of the learning content and follow-up on any deficiencies.

[0671] 3. Lack of interfaces for parents and children to learn together.

[0672] 4. There is a lack of functionality to administer an initial comprehension test and set a starting point for learning based on the results.

[0673] 5. It is not possible to continuously record learning history and comprehension and reflect this in the next learning session.

[0674] This makes it difficult to alleviate parents' anxiety and improve children's learning outcomes.

[0675] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0676] In this invention, the server includes means for receiving and analyzing textbook data from a user device that uploads the textbook data, means for automatically generating a study program based on the analyzed textbook data, and means for providing explanations based on the generated study program. This makes it possible to efficiently analyze complex textbook data and automatically generate an optimal study program.

[0677] In addition, in this invention, the server includes means for generating and providing questions to check the level of understanding related to the explanation, means for receiving and analyzing the results of the comprehension check from the user device, means for providing additional explanations and support based on the check results, and means for managing the overall learning progress of the system and suggesting appropriate learning units. This makes it possible to appropriately manage the user's learning status and effectively follow up on areas where understanding is lacking.

[0678] Furthermore, in this invention, the server includes means for analyzing textbook content using natural language processing technology, means for generating text explanations and videos using a generation AI module, means for conducting a comprehension test before the start of learning and setting a starting point based on the results, means for providing a user interface for parents and children to learn together, and means for recording the user's learning history and comprehension and reflecting this in the next learning session. This makes it possible to efficiently manage the entire learning process and support parent-child collaborative learning.

[0679] "Textbook data" refers to digital information on books and teaching materials used for education.

[0680] A "user device" is a terminal device operated by a user, and includes, for example, a personal computer, a tablet, a smartphone, etc.

[0681] "Analysis" refers to the act of breaking down complex data and information into an easy-to-understand form and classifying and organizing each element.

[0682] "Program of Study" refers to a set of educational plans or curricula designed to achieve specific learning objectives.

[0683] "Explanation" refers to explanations and answers that make the learning content easy to understand.

[0684] A "comprehension check" refers to a test or quiz that assesses how well the material has been understood.

[0685] "Questions" refer to questions or tasks that are used to assess comprehension of specific learning content.

[0686] "Support" refers to assistance and guidance provided to help users learn more efficiently.

[0687] "Learning progress" refers to data and records that show the progress a user is making at each stage.

[0688] "Natural language processing technology" refers to computer technology for understanding and processing human language.

[0689] "Generative AI Module" means an artificial intelligence program for automatically generating text or other content.

[0690] "Study history" refers to data that records what a user has learned and their progress to date.

[0691] "User interface" refers to the screens and operation methods designed to make it easy for users to operate the system.

[0692] "Comprehension test" refers to a test to assess a user's current level of knowledge before learning.

[0693] This invention relates to a support system for parents and children to work together to effectively advance their learning. The system of this invention automatically generates individual learning curricula using digital data from textbooks, and effectively manages and supports learning progress.

[0694] The user begins the learning process by uploading a textbook PDF file to the system. The textbook data is uploaded to the system by the user via a device, which can be a PC, tablet, or smartphone. The uploaded textbook data is then sent to the server via communication.

[0695] The server analyzes the received textbook PDF file using natural language processing (NLP) techniques. Specifically, it uses Tesseract OCR to extract text from PDF pages, and then uses TensorFlow and spaCy to analyze the text data. This categorizes the textbook content by chapter, section, and topic.

[0696] The server automatically generates a learning curriculum based on the analysis results. The generated curriculum is divided into learning units such as "Calculating Fractions" and "Solving Equations," and for each unit, text lectures, videos, and quizzes to check comprehension are generated. OpenAI's GPT-3 is used to generate the text lectures, and DALL-E is used to generate the videos.

[0697] Users (parents and children) log in to the system and take an initial comprehension test. The test results are sent from the device to the server, where they are analyzed. Based on the analysis results, the starting point for learning and the optimal level of difficulty are set. The comprehension test is conducted by answering questions, and the answer data is sent to the server.

[0698] Once learning begins, the server sequentially distributes lectures created for each learning unit. The lectures are displayed on the device in text or video format, allowing parents and children to study together.

[0699] As the learning progresses, the server generates quizzes and questions to check comprehension and distributes them to the device. The user (child) answers the questions and sends the results from the device to the server. The server analyzes the results and provides additional lectures or support for areas where understanding is insufficient.

[0700] The server records the user's learning history and level of understanding and reflects this in the next lesson. This allows for detailed management of learning progress and suggests appropriate learning units to tackle next. In particular, by utilizing natural language processing technology, it is possible to effectively analyze complex textbook data and generate optimal learning curricula.

[0701] As a specific example, when a user is studying mathematics with his / her child, the learning process will proceed as follows:

[0702] 1. A user uploads a mathematics textbook PDF to the system.

[0703] 2. The server analyzes the textbook PDF and automatically generates units such as "Fraction Fundamentals," "Applications of Fractions," and "Solving Equations."

[0704] 3. Users (parents and children) take the initial comprehension test and send the results to the server.

[0705] 4. Based on the results, the server suggests starting the lesson with the first unit, "Fractions Basics."

[0706] 5. The device displays a video lecture on "Fraction Basics" that parents and children watch together.

[0707] 6. The server will deliver a quiz to check comprehension after the lecture.

[0708] 7. The user (child) answers the quiz and sends the results to the server.

[0709] 8. The server analyzes the quiz results and provides additional lectures on areas where understanding is insufficient.

[0710] 9. The server will record progress and suggest the next unit, "Applications of Fractions."

[0711] Example prompt sentence:

[0712] "Please upload the PDF file of your textbook."

[0713] "Please take the comprehension test."

[0714] "Let's begin the next unit of study."

[0715] This system can reduce parental anxiety and improve children's learning outcomes.

[0716] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0717] Step 1:

[0718] Users upload PDF files of textbooks to the system.

[0719] Input: Textbook PDF file

[0720] Output: PDF file data sent to the server

[0721] Specific operation: The user opens the browser on the terminal and clicks the "Upload textbook" button on the system. A file dialog opens, selects the textbook PDF file, and presses the "Open" button. The terminal then sends this file to the server.

[0722] Step 2:

[0723] The server analyzes the received textbook PDF file.

[0724] Input: PDF file data sent to the server

[0725] Output: Text data and data categorized into chapters, sections, and topics

[0726] What it does: The server runs a Python script that first reads each page of the PDF file as an image, then uses Tesseract OCR to convert the pages to text, then uses the spaCy library to parse the extracted text and categorize it by chapter, section, and topic.

[0727] Step 3:

[0728] The server automatically generates a learning curriculum based on the analyzed data.

[0729] Input: Parsed text data and classification data

[0730] Output: Learning curriculum and text lectures, videos, quiz data

[0731] Specific operation: The server runs an algorithm that automatically divides learning units based on the analyzed text data. Then, for each unit, it calls OpenAI's GPT-3 to generate a text lecture. It also uses DALL-E to generate related videos and a set of quiz questions.

[0732] Step 4:

[0733] Users (parents and children) log into the system and take an initial comprehension test.

[0734] Input: User ID and password, comprehension test answer data

[0735] Output: Test result data

[0736] Specific operation: The user enters their ID and password on the login screen to log in to the system. After logging in, a comprehension test screen appears, and several questions are asked. Parents and children answer the quiz together, and when they are done, the results are sent from the device to the server.

[0737] Step 5:

[0738] The server analyzes the test results and sets the appropriate difficulty level and starting point.

[0739] Input: Comprehension test result data

[0740] Output: User-specific learning starting points and difficulty settings

[0741] How it works: The server analyzes the test results and runs algorithms to determine the starting point and difficulty level. Based on these results, it creates a personalized learning plan for the user.

[0742] Step 6:

[0743] The server sequentially distributes lectures created for each learning unit.

[0744] Input: Learning curriculum data, user-specific learning plans

[0745] Output: Text lecture, video lecture

[0746] Specific operation: The server determines the next learning unit and sends the URL of the corresponding text lecture or video file to the device. The device displays the lecture in the browser, and if it is a video lecture, it will automatically play it.

[0747] Step 7:

[0748] The server generates quizzes and questions to check comprehension as the learning progresses and distributes them to the device.

[0749] Input: The user's current learning progress data

[0750] Output: Quizzes and questions to check comprehension

[0751] Specific operation: The server tracks the user's learning progress data and generates quizzes to check their comprehension. The generated quizzes are sent to the device and displayed to the user.

[0752] Step 8:

[0753] The user (child) answers the questions and sends the results from the terminal to the server.

[0754] Input: User's quiz answer data

[0755] Output: Quiz result data

[0756] Specific operation: The user (child) answers the quiz displayed on the device. When the answer is complete, the device sends the result data to the server.

[0757] Step 9:

[0758] The server analyzes the quiz results and provides additional lectures and assistance for areas where understanding is lacking.

[0759] Input: Quiz result data

[0760] Output: Additional lectures and support

[0761] How it works: The server analyzes the quiz results, runs an algorithm to identify areas of insufficient understanding, and generates additional lectures based on the results and delivers them to the device.

[0762] Step 10:

[0763] The server records the user's learning history and level of understanding and reflects this in the next learning session.

[0764] Input: Learning history data and comprehension data

[0765] Output: Next lesson plan

[0766] Specific operation: The server stores the user's learning history and level of understanding in a database and executes an algorithm to reflect this in the next learning session. It also sets the learning curriculum to be tackled next.

[0767] (Application example 1)

[0768] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0769] As part of education, there is a lack of support for parents and children to study together effectively and efficiently.In addition, it is difficult to provide an appropriate curriculum and follow-up that is tailored to each child's learning progress and level of understanding, which results in poor learning outcomes.

[0770] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0771] In this invention, the server includes means for receiving and analyzing textbook data from a user terminal that uploads the textbook data, means for automatically generating a learning curriculum based on the analyzed textbook data, and means for providing easy-to-understand lectures based on the generated learning curriculum, thereby enabling parents and children to study together effectively.

[0772] "Textbook Data" means digital data of textbooks used for educational purposes that is uploaded by a User to the System.

[0773] A "user terminal" is a device used to upload textbook data, check learning progress, and check comprehension, and typically refers to a smartphone, tablet, or PC.

[0774] "Analysis" is the process of converting uploaded textbook data into text data and categorizing the content by chapter, section, and topic.

[0775] A "learning curriculum" is a learning progress plan that is automatically generated based on analyzed textbook data, and includes specific learning units and assignments.

[0776] A "lecture" is educational content provided based on a learning curriculum and displayed to users in text or video format.

[0777] "Comprehension check" refers to quizzes and questions to check the level of understanding of the learning content after the lecture.

[0778] "Quiz generation" is the process of automatically creating quizzes and questions based on learning content using a generative AI model.

[0779] The "check result" is the result data of the comprehension check sent from the user terminal, which is analyzed by the server and reflected in the next learning content.

[0780] "Additional lectures" are educational content that is provided in addition to the parts of the lecture that are not fully understood based on the results of the comprehension check.

[0781] "Follow-up" refers to ongoing support and advice provided to users based on their learning history and level of understanding.

[0782] A "generative AI model" is an artificial intelligence model that uses machine learning technology to automatically generate quizzes and questions from text data.

[0783] "Progress" is data that indicates the progress of the user's learning, and is managed by the server.

[0784] A "learning unit" is a unit of study that is classified into a specific theme or topic within a learning curriculum.

[0785] This invention is a support system for parents and children to work together to effectively advance learning. This system automatically generates a learning curriculum based on textbook data, automatically generates quizzes to check comprehension using a generative AI model, and manages and supports learning progress through lectures and comprehension checks.

[0786] System configuration

[0787] 1. Enter textbook data

[0788] The user uploads a PDF file of the textbook to the system, and the device sends the file to the server.

[0789] 2. Analysis of textbook data

[0790] The server analyzes the received textbook PDF file, extracts text data, and uses natural language processing (NLP) technology to categorize the textbook content by chapter, section, and topic.

[0791] 3. Generating learning curriculum

[0792] The server automatically generates a learning curriculum based on the analyzed data. Specifically, it determines learning units such as "fraction calculations" and "solving equations," and generates text lectures, videos, and quizzes for each unit.

[0793] 4. Start your study session

[0794] Users (parents and children) log in to the system and take the initial comprehension test. The device sends the test results to the server, which analyzes the results and sets the appropriate difficulty level and starting point.

[0795] 5. Providing lectures

[0796] The server sequentially distributes lectures created for each learning unit, and the devices display the lectures in text or video format, allowing parents and children to study together.

[0797] 6. Generating and providing quizzes to check comprehension using generative AI models

[0798] The server automatically generates quizzes based on the learning content using a generative AI model. The device provides the generated quizzes to the user, who then answers the questions.

[0799] 7. Checking understanding and providing feedback

[0800] The user (child) answers the quiz and sends the results from the device to the server. The server analyzes the quiz results and provides additional lectures on areas where the child did not fully understand.

[0801] 8. Follow-up and continuous learning

[0802] The server records the user's learning history and level of understanding and reflects this in the next lesson.The server also analyzes the user's learning progress and suggests the next appropriate learning unit for the user.

[0803] Hardware and software:

[0804] This system runs on standard PCs and smart devices (smartphones, tablets). The software and libraries used are as follows:

[0805] PyMuPDF (for text extraction)

[0806] NLTK (for text tokenization)

[0807] Transformers (for model inference, especially Facebook's BART model)

[0808] Examples:

[0809] If a user uploads a "sixth grade math textbook," the learning process will proceed as follows:

[0810] 1. A user uploads a "6th grade mathematics textbook PDF."

[0811] 2. The server analyzes the PDF and automatically generates the units "Fundamentals of Arithmetic," "Understanding Fractions," and "Fundamentals of Equations."

[0812] 3. Users (parents and children) take the initial comprehension test and send the results to the server.

[0813] 4. Based on the results, the server recommends starting with the first unit, "Fundamentals of Mathematics."

[0814] 5. The device displays a video lecture on "Mathematics Basics" that parents and children watch together.

[0815] 6. The server automatically generates a quiz using a generative AI model to check comprehension after the lecture and distributes it to the device.

[0816] 7. The user (child) answers the quiz and sends the results to the server.

[0817] 8. The server analyzes the quiz results and provides additional lectures on areas where understanding is insufficient.

[0818] 9. The server will record progress and suggest the next unit, "Understanding Fractions."

[0819] Example prompt sentence:

[0820] Generated curriculum text:

[0821] "Understanding fractions involves the following steps: First, understand the basic concept of fractions. Then practice adding and subtracting fractions. Finally, learn how to multiply and divide fractions."

[0822] Quiz generation prompt:

[0823] "Based on this text, please formulate the following questions:"

[0824] The question that is output is:

[0825] 1. What is the basic concept of fractions?

[0826] 2. Explain adding and subtracting fractions.

[0827] 3. How do you multiply and divide fractions?

[0828] This allows parents and children to study together effectively.

[0829] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0830] Step 1:

[0831] The user uploads a PDF file of a textbook to the terminal. The input is the PDF file of the textbook, and the output is the textbook PDF data sent to the server.

[0832] Step 2:

[0833] The server analyzes the received textbook PDF file and extracts the text data. Specifically, it uses the PyMuPDF library to extract text from the PDF. In this process, the input is the PDF file and the output is the text data.

[0834] Step 3:

[0835] The server uses natural language processing techniques to classify the extracted text data into chapters, sections, and topics. It uses the NLTK library to tokenize sentences and identify each section. The input of this process is the extracted text data, and the output is the classified text data.

[0836] Step 4:

[0837] The server automatically generates a learning curriculum based on the analyzed data. Specifically, it generates learning units such as "fraction calculation" and "solving equations" from the analyzed data. The input for this process is classified text data, and the output is a learning curriculum.

[0838] Step 5:

[0839] Users (parents and children) log in to the system and take the initial comprehension test. The terminal sends the test results to the server. The input is the comprehension test questions and their answers, and the output is the answer data sent to the server.

[0840] Step 6:

[0841] The server analyzes the results of the comprehension test and sets the appropriate difficulty level and starting point. The analysis uses the received answer data to determine the initial learning unit. The input of this process is the answer data, and the output is the initial learning unit and its difficulty setting.

[0842] Step 7:

[0843] The server sequentially distributes lectures created for each learning unit. The terminal displays them to the user in text or video format. The content of the lecture is based on the learning curriculum determined in the previous step. The input of this process is the learning curriculum, and the output is the lecture content to be distributed.

[0844] Step 8:

[0845] The server uses a generative AI model to automatically generate quizzes based on the learning content. The model uses the BART model from the Transformers library and prompts to generate quizzes. The inputs to this process are the learning content and prompts, and the output is an automatically generated quiz.

[0846] Step 9:

[0847] The user (child) answers the generated quiz and sends it from the terminal to the server. The input is the answer to the quiz, and the output is the answer data sent to the server.

[0848] Step 10:

[0849] The server analyzes the quiz results and provides additional lectures for areas where the student did not fully understand the quiz. The input to this process is the answer data, and the output is additional lectures as needed.

[0850] Step 11:

[0851] The server records the user's learning history and level of understanding and suggests the next learning unit. Based on the recorded data, the server analyzes the learning history and level of understanding and provides the next learning curriculum. The input for this process is the learning history and quiz results, and the output is a suggestion for the next learning unit.

[0852] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0853] This invention is an educational support system that enables parents and children to work together to effectively advance learning, and it also includes an emotion engine that recognizes the user's emotions and customizes the learning experience. This system analyzes textbook data, automatically generates learning curricula, conducts lectures and comprehension checks, and records the user's learning history and emotional state to reflect in the next lesson.

[0854] System configuration

[0855] Textbook data entry

[0856] Users upload textbook PDF files to the system. The device sends the files to the server, which then analyzes the received textbook PDF files to extract text data. Natural language processing (NLP) techniques are used to categorize the textbook content by chapter, section, and topic.

[0857] Learning curriculum generation

[0858] The server generates learning units from the analyzed textbook data. These units include "fraction calculations" and "solving equations." Each unit is associated with a text lecture, video content, and quizzes.

[0859] Start a study session

[0860] Users (parents and children) log in to the system and take the initial comprehension test. The device sends the test results to the server, which analyzes the results and sets the appropriate difficulty level and starting point.

[0861] Lecture provision

[0862] The server delivers lectures for the designated learning unit to the device, which displays the lectures in text or video format, allowing parents and children to study together.

[0863] Check your understanding

[0864] As the learning progresses, the server generates quizzes and questions to check comprehension and distributes them to the device. The user (child) answers the questions, and the device sends the results to the server. The server analyzes the results and provides additional lectures or follow-ups for areas where understanding is insufficient.

[0865] Follow-up and continuous learning

[0866] The server records the user's learning history and level of understanding and reflects this in the next lesson. The server also analyzes the user's learning progress and suggests the appropriate next learning unit. The device displays the lecture for the next learning unit and notifies the user.

[0867] User Emotion Recognition

[0868] The server contains an emotion engine and analyzes emotions from the user's facial expressions and voice. The device's built-in camera and microphone are used to capture users' (parents and children's) emotional data in real time and send it to the server. Based on the emotion analysis results, the server adjusts the system's behavior as follows:

[0869] If the user feels stressed while learning, the server will adjust the difficulty level and provide additional support to promote understanding.

[0870] If the user shows excitement or interest, progress the learning content to maintain that state.

[0871] Motivational feedback

[0872] Based on the analysis results of the emotion engine, the server generates encouragement and feedback that matches the user's emotional state. For example, if it determines that the user is tired, it will suggest taking a break, and if they are concentrating, it will display a message encouraging them to continue. This makes the learning experience for parents and children smoother and more comfortable.

[0873] Specific examples

[0874] For example, when a user is learning mathematics with his / her child, the user goes through the following process.

[0875] 1. The user uploads a PDF of a mathematics textbook, and the server analyzes the data and generates a learning curriculum.

[0876] 2. Users (parents and children) log in, take the initial test, and send the results to the server.

[0877] 3. Based on the analysis results, the server sets the first learning unit, "Fraction Basics," and delivers the lecture to the device.

[0878] 4. The device displays the lecture, and parents and children can learn together.

[0879] 5. The server sends a quiz to check comprehension, and the child sends the answers to the server.

[0880] 6. The server provides additional lectures based on comprehension analysis and adjusts the difficulty level as needed.

[0881] 7. The server records the learning history and level of understanding and suggests the next learning unit.

[0882] 8. The device will display the lecture for the next learning unit and you can continue learning.

[0883] 9. The server analyzes the user's emotions through an emotion engine and adjusts the learning content and lecture method as needed.

[0884] 10. Based on the results of the emotion analysis, the server generates encouraging and motivating feedback and delivers it to the device.

[0885] In this way, it is possible to effectively support parents and children in their learning and improve learning efficiency by customizing the learning experience for each individual. This system can reduce parental anxiety, which has traditionally been a challenge, and improve children's learning outcomes.

[0886] The processing flow will be explained below.

[0887] Specific processing flow of the program

[0888] Step 1:

[0889] Uploading textbook data

[0890] Users select the textbook PDF file and drag and drop it into the system's upload form.

[0891] The terminal sends the uploaded PDF file to the server.

[0892] Step 2:

[0893] Textbook data analysis

[0894] The server analyzes the received textbook PDF file and extracts the text data.

[0895] The server uses natural language processing (NLP) techniques to categorize the textbook content by chapter, section, and topic.

[0896] Step 3:

[0897] Learning curriculum generation

[0898] Based on the analyzed data, the server generates learning units for each chapter and section.

[0899] The server associates each unit with the appropriate text lectures, video content, and quizzes.

[0900] Step 4:

[0901] User Login

[0902] Users (parents and children) log into the system.

[0903] The terminal sends the user's login information to the server for authentication.

[0904] Step 5:

[0905] First comprehension test

[0906] The server provides comprehension tests to parents and children as a preliminary step in learning.

[0907] Users (parents and children) take the test and send the results from their device to the server.

[0908] The server analyzes the received results and evaluates the user's current level of understanding.

[0909] Step 6:

[0910] Setting appropriate learning units

[0911] Based on the results of the comprehension test, the server sets the initial learning unit according to the user's level.

[0912] The server delivers the lecture for the first learning unit to the terminal.

[0913] Step 7:

[0914] Lecture provision

[0915] The terminal displays the lecture of the learning unit received from the server (e.g., text, video).

[0916] Users (parents and children) watch lectures together and progress through their learning.

[0917] Step 8:

[0918] Comprehension check

[0919] After the lecture ends, the server generates quizzes and questions to check comprehension and distributes them to the device.

[0920] The user (child) answers questions and quizzes and sends the results from the terminal to the server.

[0921] Step 9:

[0922] Analysis of check results and feedback

[0923] The server analyzes the answers to the quizzes and questions and evaluates the user's level of understanding.

[0924] The server generates and delivers additional lectures and follow-ups for areas where understanding is deemed insufficient.

[0925] Step 10:

[0926] Suggestions for the next learning unit

[0927] The server records the user's progress and level of understanding and suggests the next unit to study.

[0928] The device will display the lecture for the next learning unit and notify the user.

[0929] Step 11:

[0930] Using the Emotion Engine

[0931] The device uses its built-in camera and microphone to collect the user's facial expressions and voice in real time.

[0932] The terminal transmits the collected data to the server.

[0933] The server uses an emotion engine to analyze the received data and assess the user's emotional state.

[0934] Step 12:

[0935] Adjustment based on emotional state

[0936] Based on the analysis results of the emotion engine, the server adjusts the difficulty of the task if the user is feeling stressed and provides additional support to promote understanding.

[0937] If the user shows excitement or interest, the server will progress the learning content to maintain that emotion.

[0938] Step 13:

[0939] Motivational feedback

[0940] Based on the analysis results of the emotion engine, the server generates encouragement and feedback that is in line with the user's emotional state.

[0941] The device displays generated feedback messages, such as suggesting a break if you're tired, or encouraging you to keep going if you're concentrating.

[0942] In this way, the system of the present invention responds not only to the learning progress but also to the emotional state of the user, making it possible to provide an effective learning experience.

[0943] Example 2

[0944] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0945] While conventional learning support systems can measure a user's learning situation and level of understanding, they have the problem of being unable to customize the system to take into account the user's emotional state. This reduces learning efficiency and makes it difficult to maintain user motivation. Furthermore, when parents and children study together, the lack of appropriate support increases parental anxiety and makes it difficult to improve children's learning outcomes.

[0946] The specific processing by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes: means for receiving and analyzing textbook data from a user terminal that uploads the textbook data; means for automatically generating a learning curriculum based on the analyzed textbook data; means for providing easy-to-understand lectures based on the generated learning curriculum; means for generating and providing quizzes and questions to check the level of understanding related to the lecture; means for receiving and analyzing the results of the comprehension check from the user terminal; means for providing additional lectures and follow-ups based on the check results; means for managing the overall progress of the system and suggesting appropriate learning units; and means for analyzing the emotional state of the user during learning in real time and adjusting the system operation based on the results. This provides an optimal learning experience that takes into account not only the user's learning situation but also their emotional state, thereby improving learning efficiency and motivation.

[0947] "Textbook data" refers to digital document data containing content used for educational purposes.

[0948] A "user terminal" is an electronic device used for learning activities such as uploading textbook data, viewing lectures, and answering quizzes.

[0949] "Means of analysis" refers to the technology and equipment used to analyze textbook data, extract the necessary information, and classify it.

[0950] A "learning curriculum" is a set of learning content and activity plans that are established to help users effectively advance their learning.

[0951] "Means for automatic generation" refers to technology or devices for automatically creating a learning curriculum from analyzed textbook data.

[0952] A "lecture" is an explanation or instruction of the learning content provided based on the learning curriculum.

[0953] "Quizzes and questions to check comprehension" are questions provided to assess the user's level of comprehension of the learning content.

[0954] The "means for providing additional lectures or follow-up based on the check results" refers to a technology or device that analyzes the results of the check on the user's comprehension and provides further guidance or supplementary explanations as necessary.

[0955] "Means for managing progress and suggesting appropriate learning units" refers to technologies and devices for tracking a user's learning progress and suggesting what content to learn next.

[0956] "Means for analyzing emotional states in real time" refers to technology or devices for detecting and analyzing emotions in real time from a user's facial expressions, voice, etc.

[0957] "Means for adjusting the system's operation" refers to technology or devices for appropriately changing the learning content or lecture method based on the results of emotion analysis.

[0958] The present invention relates to an educational support system for parents and children to effectively collaborate in learning, and in particular, to a system including an emotion engine that recognizes the user's emotions and customizes the learning experience. This system is realized through the interaction between a server, a terminal, and a user.

[0959] Hardware and software used

[0960] The hardware in this system is as follows:

[0961] Device: A personal computer (PC) or tablet device used at home or in an educational institution.

[0962] Server: A central management device that analyzes and manages data.

[0963] Camera and microphone: Equipped on the device, they capture the user's facial expressions and voice in real time.

[0964] The software used is as follows:

[0965] Natural Language Processing (NLP) techniques: used to analyze textbook data.

[0966] Sentiment Analysis Engine: Used to analyze the user's emotional state.

[0967] OCR (Optical Character Recognition) technology: Used to convert PDF files into text data.

[0968] Various modules: Examples include PDFReceiverService, NLPAnalyzer, TextClassifier, CurriculumGenerator, TestEvaluator, LectureDistributor, LectureViewer, QuizGenerator, AnswerAnalyzer, LearningHistoryRecorder, NextUnitRecommender, EmotionAnalyzer, EmotionCapture, FeedbackGenerator, and FeedbackDisplay.

[0969] System Description

[0970] 1. Enter textbook data

[0971] A user logs into the system and uploads a textbook PDF file from their device. The device sends this file to the server, which uses a PDFReceiverService to receive, store, and prepare the file for processing.

[0972] 2. Textbook data analysis and curriculum generation

[0973] The server uses the OCRProcessor to scan PDF files and extract text data. The extracted text data is then analyzed by the NLPAnalyzer and classified by chapter, section, and topic. The CurriculumGenerator automatically generates a learning curriculum based on this data, associating each unit with text lectures, video content, and quizzes.

[0974] 3. First comprehension test

[0975] Users (parents and children) log in to the system and take the initial comprehension test on their device. The device sends the test results to the server, which analyzes them using the TestEvaluator and sets the appropriate difficulty level and starting point.

[0976] 4. Providing lectures

[0977] The server distributes text and video lectures to devices using LectureDistributor based on the set learning units. The devices display these lectures using LectureViewer, allowing parents and children to study together.

[0978] 5. Check your understanding

[0979] As the learning progresses, the server uses QuizGenerator to generate quizzes and questions to check comprehension and distributes them to the device. The user (child) answers the quiz on the device and sends the results to the server. The server analyzes the answers using AnswerAnalyzer and provides additional lectures or follow-ups for areas where understanding is insufficient.

[0980] 6. Recording learning history and suggesting next learning units

[0981] The server uses LearningHistoryRecorder to record the user's learning history and level of understanding. NextUnitRecommender then uses this information to suggest the next learning unit. The device then displays the lecture for the suggested next learning unit and notifies the user.

[0982] 7. User Emotion Recognition

[0983] The server uses the emotion engine to analyze the user's facial expressions and voice captured by the device's camera and microphone in real time. The data is acquired through EmotionCapture and analyzed by EmotionAnalyzer to adjust the system's behavior.

[0984] 8. Feedback and Motivation

[0985] The server uses the FeedbackGenerator to provide encouragement and feedback to the user based on the sentiment analysis results, and the device uses the FeedbackDisplay to display this feedback and improve the quality of the learning experience.

[0986] Specific examples

[0987] For example, when a user is learning mathematics with a child, the following process is carried out.

[0988] 1. The user uploads a PDF of a mathematics textbook, and the server analyzes the data and generates a learning curriculum.

[0989] 2. Users (parents and children) log in, take the initial test, and send the results to the server.

[0990] 3. Based on the analysis results, the server sets the first learning unit, "Fraction Basics," and delivers the lecture to the device.

[0991] 4. The device displays the lecture, and parents and children can learn together.

[0992] 5. The server sends a quiz to check comprehension, and the child sends the answers to the server.

[0993] 6. The server provides additional lectures based on comprehension analysis and adjusts the difficulty level as needed.

[0994] 7. The server records the learning history and level of understanding and suggests the next learning unit.

[0995] 8. The device will display the lecture for the next learning unit and you can continue learning.

[0996] 9. The server analyzes the user's emotions through an emotion engine and adjusts the learning content and lecture method as needed.

[0997] 10. Based on the results of the emotion analysis, the server generates encouraging and motivating feedback and delivers it to the device.

[0998] Prompt Sentence Examples

[0999] "Generate learning units from the following PDF textbooks and customize lecture content based on comprehension checks and sentiment analysis."

[1000] As a result, parents and children can study effectively, and the learning effect and motivation of users can be improved.

[1001] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1002] Step 1: Enter textbook data

[1003] Input: Textbook PDF file

[1004] Specific operation: The user clicks the "Upload textbook" button on the terminal, selects and uploads a textbook PDF file from the local file system, and the terminal uses the FileUploader module to send the selected PDF file to the server.

[1005] Output: PDF file saved on the server

[1006] Data processing: The server starts the PDFReceiverService and saves the received file in a temporary folder.

[1007] Step 2: Analyzing textbook data

[1008] Input: PDF file stored on the server

[1009] How it works: The server uses OCRProcessor to scan PDF files and extract text data, which is then analyzed by NLPAnalyzer and classified into chapters, sections, and topics.

[1010] Output: Analyzed textbook data

[1011] Data processing: The server scans the PDF data as an image and converts it into text data. The converted text data is then analyzed using natural language processing technology to classify each piece of content.

[1012] Step 3: Generate a learning curriculum

[1013] Input: Parsed textbook data

[1014] Specific operation: The server uses CurriculumGenerator to automatically generate learning units from the parsed data, and associates text lectures, video content, and quizzes with the generated learning units.

[1015] Output: Generated learning curriculum

[1016] Data processing: Based on each extracted topic, the server generates appropriate learning units and adds related educational resources (text, videos, quizzes).

[1017] Step 4: First assessment

[1018] Input: User (parent and child) login information

[1019] Specific operation: A user logs in to the system and takes the initial comprehension test on their device. The device displays the test questions using QuizPresenter and the user enters their answers. The answer data is sent to the server via AnswerCollector.

[1020] Output: Test result data

[1021] Data processing: The server uses TestEvaluator to analyze the submitted answers and set appropriate difficulty levels and starting points.

[1022] Step 5: Deliver the lecture

[1023] Input: Learning curriculum and initial comprehension test results

[1024] Specific operation: The server distributes the lectures of the set learning unit to the device using LectureDistributor. The device uses LectureViewer to display the received lectures in text or video format.

[1025] Output: Visual and audio lecture data

[1026] Data processing: The server selects and sends the appropriate lecture data (text, video), which is then displayed on the device.

[1027] Step 6: Check your understanding

[1028] Input: Learning status data after the lecture

[1029] Specific operation: The server uses QuizGenerator to generate quizzes and questions for comprehension checks and distributes them to the device. The user (child) answers the quiz on the device and sends the results to the server via AnswerCollector.

[1030] Output: Comprehension check result data

[1031] Data processing: The server uses AnswerAnalyzer to analyze the answer data and identify any missing parts of understanding.

[1032] Step 7: Follow-up and learning history

[1033] Input: Comprehension check result data

[1034] Specific operation: The server uses LearningHistoryRecorder to record the user's learning history and recommends the best learning unit for the next lesson. NextUnitRecommender selects the next learning unit and sends it from the server to the device. The device then displays the next lecture and notifies the user.

[1035] Output: Updated learning history data and next learning unit

[1036] Data processing: The server identifies the next learning step based on the recorded learning history and comprehension data, and updates the optimal learning plan.

[1037] Step 8: Recognizing User Emotions

[1038] Input: User facial and voice data

[1039] How it works: The device uses a camera and microphone to capture the user's facial expressions and voice in real time. This data is sent to the server via EmotionCapture, and the server uses EmotionAnalyzer to analyze the user's emotional state.

[1040] Output: Sentiment analysis data

[1041] Data processing: The server applies facial expression recognition and voice analysis technologies to identify the user's emotional state, such as stress, excitement, or concentration, in real time.

[1042] Step 9: Feedback and motivation

[1043] Input: Sentiment analysis data

[1044] Specific operation: The server uses FeedbackGenerator to automatically generate appropriate encouragement and feedback for the user based on the results of sentiment analysis, and the device notifies the user using FeedbackDisplay.

[1045] Output: An informative feedback message

[1046] Data processing: The server generates feedback messages that reflect the user's emotional state and presents them to the user to personalize the learning experience and increase motivation.

[1047] This detailed process flow enables the system to provide a personalized educational experience tailored to the user's learning needs.

[1048] (Application example 2)

[1049] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1050] Conventional educational support systems simply provide learning curricula and lack functionality to respond to users' emotions and individual progress. This makes it difficult to respond appropriately to declines in motivation or differences in understanding, especially in children's learning. Furthermore, when parents and children study together, the burden on parents increases, making it difficult to provide effective support.

[1051] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving and analyzing textbook data from a user terminal that uploads the textbook data, means for automatically generating a learning curriculum based on the analyzed textbook data, means for providing easy-to-understand lectures based on the generated learning curriculum, means for generating and providing quizzes and questions to check the level of understanding related to the lecture, means for receiving and analyzing the results of the comprehension check from the user terminal, means for providing additional lectures and follow-ups based on the check results, means for managing overall progress and suggesting appropriate learning units, and means for recognizing the user's emotions and customizing the learning experience. This enables flexible response according to the user's learning status, allowing parents and children to study together effectively.

[1052] "Textbook data" refers to digital data of books and teaching materials used in education.

[1053] A "user device" is an electronic device such as a computer, tablet, or smartphone used by a learner.

[1054] "Analysis" refers to the process of understanding and classifying the content of textbook data using techniques such as natural language processing.

[1055] A "curriculum" is a set of learning content and activities planned in accordance with educational objectives.

[1056] A "lecture" is educational content that provides explanations and instruction on a certain topic.

[1057] A "quiz" is a question or problem that allows you to check what you have learned.

[1058] A "comprehension check" is a process of assessing how well a learner has understood the learning content.

[1059] "Following up" means providing additional instruction or materials to reinforce areas of incomplete understanding.

[1060] "Progress" refers to the degree to which a learner has progressed in the learning curriculum.

[1061] A "learning unit" is a piece of learning organized around a specific educational goal.

[1062] "Emotion recognition" is a technology that analyzes emotions from a user's facial expressions, voice, etc.

[1063] A "learning experience" is any experience or sensation a learner has in the process of learning.

[1064] The present invention is an educational support system for parents and children to effectively collaborate in learning, and further includes an emotion engine that recognizes the user's emotions and customizes the learning experience.

[1065] System configuration

[1066] Textbook data entry

[1067] Users upload textbook PDF files to the system. The user's device sends the files to the server, which then analyzes the received textbook PDF files and extracts text data. Natural language processing (NLP) techniques are used to categorize the textbook content by chapter, section, and topic. For this purpose, Python and NLP libraries (e.g., spaCy) are used.

[1068] Learning curriculum generation

[1069] The server generates a learning curriculum from the analyzed textbook data. The generated units include "fraction calculation" and "solving equations." Each unit is associated with a text lecture, video content, and quizzes.

[1070] Start a study session

[1071] Users (parents and children) log in to the system and take the initial comprehension test. The user device sends the test results to the server, which analyzes the results and sets an appropriate difficulty level and starting point.

[1072] Lecture provision

[1073] The server delivers lectures for the selected learning unit to the user's device, which displays the lecture in text or video format, allowing parents and children to study together.

[1074] Check your understanding

[1075] As the learning progresses, the server generates quizzes and questions to check comprehension and distributes them to the user's device. The user (child) answers the questions, and the user's device sends the results to the server. The server analyzes the results and provides additional lectures or follow-ups for areas where understanding is insufficient.

[1076] Follow-up and continuous learning

[1077] The server records the user's learning history and level of understanding and reflects this in the next lesson. The server also analyzes the user's learning progress and suggests the appropriate next learning unit. The user's device displays the lecture for the next learning unit and notifies the user.

[1078] User Emotion Recognition

[1079] The server contains an emotion engine and analyzes emotions from the user's facial expressions and voice. The camera and microphone installed on the user's device are used to capture the user's (parent and child) emotional data in real time and send it to the server. Based on the emotion analysis results, the server adjusts the system's operation as follows:

[1080] If the user feels stressed while learning, the server will adjust the difficulty level and provide additional support to promote understanding.

[1081] If the user shows excitement or interest, progress the learning content to maintain that state.

[1082] Motivational feedback

[1083] Based on the analysis results of the emotion engine, the server generates encouragement and feedback that matches the user's emotional state. For example, if it determines that the user is tired, it will suggest taking a break, and if they are concentrating, it will display a message encouraging them to continue. This makes the learning experience for parents and children smoother and more comfortable.

[1084] Specific examples

[1085] For example, when a user and his / her child are learning mathematics in a virtual store, the user goes through the following process.

[1086] 1. The user uploads a PDF of a mathematics textbook, and the server analyzes the data and generates a learning curriculum.

[1087] 2. Users (parents and children) log in, take the initial test, and send the results to the server.

[1088] 3. Based on the analysis results, the server sets the first learning unit, "Fraction Basics," and delivers the lecture to the user's device.

[1089] 4. The user device displays the lecture, and the parent and child learn together.

[1090] 5. The server sends a quiz to check comprehension, and the child sends the answers to the server.

[1091] 6. The server provides additional lectures based on comprehension analysis and adjusts the difficulty level as needed.

[1092] 7. The server records the learning history and level of understanding and suggests the next learning unit.

[1093] 8. The user's device will display the lecture for the next learning unit and continue learning.

[1094] 9. The server analyzes the user's emotions through an emotion engine and adjusts the learning content and lecture method as needed.

[1095] 10. Based on the results of the emotion analysis, the server generates encouraging and motivating feedback and delivers it to the user's device.

[1096] Example prompt for a generative AI model:

[1097] "Analyze the content of the fourth grade textbook "Calculating Fractions" and generate a lecture in the following format.

[1098] 1. Lecture Title

[1099] 2. Lecture Overview

[1100] 3. List of key points

[1101] 4. Quiz questions to check comprehension and sample answers

[1102] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1103] Step 1:

[1104] The user uploads a PDF file of a textbook to the system. The user's device sends this file to the server. The input is the PDF data of the textbook, and the output is the PDF file sent to the server. Specifically, a file selection dialog opens and a request is made to send the file selected by the user to the server.

[1105] Step 2:

[1106] The server analyzes the received textbook PDF file and extracts the text data. The input is the PDF file, and the output is the analyzed text data. Specifically, the PDFMiner library is used to convert the contents of the PDF file into text format.

[1107] Step 3:

[1108] The server uses natural language processing (NLP) techniques to classify the parsed text data into chapters, sections, and topics. The input is text data, and the output is classified text data. Specifically, the spaCy library is used to split the text and store it as structured data.

[1109] Step 4:

[1110] The server automatically generates a learning curriculum from structured text data. The input is classified text data, and the output is a learning curriculum. Specifically, it automatically generates related lectures and quizzes for each chapter and compiles them into learning units.

[1111] Step 5:

[1112] Users (parents and children) log in to the system and take the initial comprehension test. The user device sends the test results to the server. The input is the answer data to the comprehension test, and the output is the test results sent for analysis. In concrete terms, the user answers the test and sends the results to the server.

[1113] Step 6:

[1114] The server analyzes the initial test results and sets the appropriate difficulty level and starting point. The input is the test results, and the output is the appropriate difficulty setting. Specifically, it determines the optimal learning level based on the test results and selects the next learning unit.

[1115] Step 7:

[1116] The server delivers lectures for the set learning units to the user's device. The input is the learning unit data, and the output is the lecture data for the user's device. Specifically, the server sends the lecture in text or video format for display on the user's device.

[1117] Step 8:

[1118] The user device displays the lecture, and parents and children can study together. The input is lecture data, and the output is the lecture displayed on the user device. In concrete terms, the user device provides a UI that displays the lecture content.

[1119] Step 9:

[1120] The server generates quizzes and questions to check comprehension and distributes them to the user's device. The input is the learning unit data, and the output is the quizzes and questions sent to the user's device. Specifically, the server generates quizzes using a Python script and sends them to the user's device.

[1121] Step 10:

[1122] The user (child) answers quizzes and questions, and the user device sends the results to the server. The input is the answer data to the quizzes and questions, and the answer results sent to the server are obtained as output. In concrete terms, the user answers the quiz and sends the results to the server.

[1123] Step 11:

[1124] The server analyzes the answer results and provides additional lectures or follow-ups for areas where understanding is insufficient. The answer result data is input, and additional lecture data is obtained as output. Specifically, the server analyzes the answer results, generates necessary supplementary explanations, and sends them to the user's device.

[1125] Step 12:

[1126] The server records the user's learning history and level of understanding and reflects this in the next lesson. The input is comprehension level and learning history data, and the output is the next learning plan. Specific operations include saving the learning history and level of understanding in a database and planning the next learning unit.

[1127] Step 13:

[1128] The server analyzes emotions from the user's facial expressions and voice. Facial expressions and voice data are input, and emotion analysis results are obtained as output. Specifically, emotions are analyzed using OpenCV and the Google Cloud Speech API.

[1129] Step 14:

[1130] The server adjusts the learning content and lecture method based on the emotion analysis results. The emotion analysis results are input, and the adjusted learning content is obtained as output. Specifically, the server adjusts the difficulty and progress of the learning content according to the emotional state.

[1131] Step 15:

[1132] The server generates encouraging and motivating feedback based on the emotion analysis results and delivers it to the user's device. The emotion analysis results are input, and feedback messages are obtained as output. Specifically, the server generates messages of encouragement or suggestions for taking a break based on the user's emotional state and sends them to the user's device.

[1133] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1134] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1135] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1136] [Third embodiment]

[1137] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1138] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1139] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1140] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1141] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1142] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1143] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1144] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1145] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1146] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1147] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1148] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1149] This invention is a support system for parents and children to work together to effectively advance learning. This system automatically generates learning curricula based on textbook data, and manages and supports learning progress through lectures and comprehension checks.

[1150] System configuration

[1151] Textbook data entry

[1152] The user uploads a PDF file of the textbook to the system, and the device sends the file to the server.

[1153] Textbook data analysis

[1154] The server analyzes the received textbook PDF file, extracts text data, and uses natural language processing (NLP) techniques to categorize the textbook content by chapter, section, and topic.

[1155] Learning curriculum generation

[1156] The server automatically generates a learning curriculum based on the analyzed data. Specifically, it determines learning units such as "fraction calculations" and "solving equations," and generates text lectures, videos, and quizzes for each unit.

[1157] Start a study session

[1158] Users (parents and children) log in to the system and take the initial comprehension test. The device sends the test results to the server, which analyzes the results and sets the appropriate difficulty level and starting point.

[1159] Lecture provision

[1160] The server sequentially distributes lectures created for each learning unit, and the devices display the lectures in text or video format, allowing parents and children to study together.

[1161] Check your understanding

[1162] The server generates quizzes and questions to check comprehension as the learning progresses and distributes them to the device. The user (child) answers the questions, and the device sends the results to the server. The server analyzes the results and provides additional lectures for areas where understanding is insufficient.

[1163] Follow-up and continuous learning

[1164] The server records the user's learning history and level of understanding and reflects this in the next lesson.The server also analyzes the user's learning progress and suggests the next appropriate learning unit for the user.

[1165] Specific examples

[1166] For example, if a user wants to study mathematics with his or her child, the learning process will proceed as follows:

[1167] 1. A user uploads a mathematics textbook PDF to the system.

[1168] 2. The server analyzes the textbook PDF and automatically generates units such as "Fraction Fundamentals," "Applications of Fractions," and "Solving Equations."

[1169] 3. Users (parents and children) take the initial comprehension test and send the results to the server.

[1170] 4. Based on the results, the server recommends starting with the first unit, "Fractions Basics."

[1171] 5. The device displays a video lecture on "Fraction Basics" that parents and children watch together.

[1172] 6. The server distributes a quiz to check comprehension after the lecture.

[1173] 7. The user (child) answers the quiz and sends the results to the server.

[1174] 8. The server analyzes the quiz results and provides additional lectures on areas where understanding is insufficient.

[1175] 9. The server will record progress and suggest the next unit, "Applications of Fractions."

[1176] In this way, parents and children can effectively advance their learning through the system. The present invention can reduce parental anxiety, which has been a problem in the past, and improve children's learning effectiveness.

[1177] The processing flow will be explained below.

[1178] Specific processing flow of the program

[1179] Step 1:

[1180] Uploading textbook data

[1181] Users select the textbook PDF file and drag and drop it into the system's upload form.

[1182] The terminal sends the uploaded PDF file to the server.

[1183] Step 2:

[1184] Textbook data analysis

[1185] The server analyzes the received textbook PDF file and extracts the text data.

[1186] The server uses natural language processing (NLP) techniques to categorize the textbook content by chapter, section, and topic.

[1187] Step 3:

[1188] Learning curriculum generation

[1189] Based on the analyzed data, the server generates learning units for each chapter and section.

[1190] The server associates each unit with the appropriate text lectures, video content, and quizzes.

[1191] Step 4:

[1192] User Login

[1193] Users (parents and children) log into the system.

[1194] The terminal sends the user's login information to the server for authentication.

[1195] Step 5:

[1196] First comprehension test

[1197] The server provides comprehension tests to parents and children as a preliminary step in learning.

[1198] Users (parents and children) take the test and send the results from their device to the server.

[1199] The server analyzes the received results and evaluates the user's current level of understanding.

[1200] Step 6:

[1201] Setting appropriate learning units

[1202] Based on the results of the comprehension test, the server sets the initial learning unit according to the user's level.

[1203] The server delivers the lecture for the first learning unit to the terminal.

[1204] Step 7:

[1205] Lecture provision

[1206] The terminal displays the lecture of the learning unit received from the server (e.g., text, video).

[1207] Users (parents and children) watch lectures together and progress through their learning.

[1208] Step 8:

[1209] Comprehension check

[1210] After the lecture ends, the server generates quizzes and questions to check comprehension and distributes them to the device.

[1211] The user (child) answers questions and quizzes and sends the results from the terminal to the server.

[1212] Step 9:

[1213] Analysis of check results and feedback

[1214] The server analyzes the answers to the quizzes and questions and evaluates the user's level of understanding.

[1215] The server generates and delivers additional lectures and follow-ups for areas where understanding is deemed insufficient.

[1216] Step 10:

[1217] Suggestions for the next learning unit

[1218] The server records the user's progress and level of understanding and suggests the next unit to study.

[1219] The device will display the lecture for the next learning unit and notify the user.

[1220] In this way, the system provides support at each stage of the process to help parents and children learn effectively.

[1221] Example 1

[1222] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1223] In recent years, there has been a demand for support systems that allow parents and children to study together, but conventional systems have the following issues.

[1224] 1. It is difficult to efficiently analyze textbook data and automatically generate appropriate learning programs.

[1225] 2. There is insufficient checking of the level of understanding of the learning content and follow-up on any deficiencies.

[1226] 3. Lack of interfaces for parents and children to learn together.

[1227] 4. There is a lack of functionality to administer an initial comprehension test and set a starting point for learning based on the results.

[1228] 5. It is not possible to continuously record learning history and comprehension and reflect this in the next learning session.

[1229] This makes it difficult to alleviate parents' anxiety and improve children's learning outcomes.

[1230] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1231] In this invention, the server includes means for receiving and analyzing textbook data from a user device that uploads the textbook data, means for automatically generating a study program based on the analyzed textbook data, and means for providing explanations based on the generated study program. This makes it possible to efficiently analyze complex textbook data and automatically generate an optimal study program.

[1232] In addition, in this invention, the server includes means for generating and providing questions to check the level of understanding related to the explanation, means for receiving and analyzing the results of the comprehension check from the user device, means for providing additional explanations and support based on the check results, and means for managing the overall learning progress of the system and suggesting appropriate learning units. This makes it possible to appropriately manage the user's learning status and effectively follow up on areas where understanding is lacking.

[1233] Furthermore, in this invention, the server includes means for analyzing textbook content using natural language processing technology, means for generating text explanations and videos using a generation AI module, means for conducting a comprehension test before the start of learning and setting a starting point based on the results, means for providing a user interface for parents and children to learn together, and means for recording the user's learning history and comprehension and reflecting this in the next learning session. This makes it possible to efficiently manage the entire learning process and support parent-child collaborative learning.

[1234] "Textbook data" refers to digital information on books and teaching materials used for education.

[1235] A "user device" is a terminal device operated by a user, and includes, for example, a personal computer, a tablet, a smartphone, etc.

[1236] "Analysis" refers to the act of breaking down complex data and information into an easy-to-understand form and classifying and organizing each element.

[1237] "Program of Study" refers to a set of educational plans or curricula designed to achieve specific learning objectives.

[1238] "Explanation" refers to explanations and answers that make the learning content easy to understand.

[1239] A "comprehension check" refers to a test or quiz that assesses how well the material has been understood.

[1240] "Questions" refer to questions or tasks that are used to assess comprehension of specific learning content.

[1241] "Support" refers to assistance and guidance provided to help users learn more efficiently.

[1242] "Learning progress" refers to data and records that show the progress a user is making at each stage.

[1243] "Natural language processing technology" refers to computer technology for understanding and processing human language.

[1244] "Generative AI Module" means an artificial intelligence program for automatically generating text or other content.

[1245] "Study history" refers to data that records what a user has learned and their progress to date.

[1246] "User interface" refers to the screens and operation methods designed to make it easy for users to operate the system.

[1247] "Comprehension test" refers to a test to assess a user's current level of knowledge before learning.

[1248] This invention relates to a support system for parents and children to work together to effectively advance their learning. The system of this invention automatically generates individual learning curricula using digital data from textbooks, and effectively manages and supports learning progress.

[1249] The user begins the learning process by uploading a textbook PDF file to the system. The textbook data is uploaded to the system by the user via a device, which can be a PC, tablet, or smartphone. The uploaded textbook data is then sent to the server via communication.

[1250] The server analyzes the received textbook PDF file using natural language processing (NLP) techniques. Specifically, it uses Tesseract OCR to extract text from PDF pages, and then uses TensorFlow and spaCy to analyze the text data. This categorizes the textbook content by chapter, section, and topic.

[1251] The server automatically generates a learning curriculum based on the analysis results. The generated curriculum is divided into learning units such as "Calculating Fractions" and "Solving Equations," and for each unit, text lectures, videos, and quizzes to check comprehension are generated. OpenAI's GPT-3 is used to generate the text lectures, and DALL-E is used to generate the videos.

[1252] Users (parents and children) log in to the system and take an initial comprehension test. The test results are sent from the device to the server, where they are analyzed. Based on the analysis results, the starting point for learning and the optimal level of difficulty are set. The comprehension test is conducted by answering questions, and the answer data is sent to the server.

[1253] Once learning begins, the server sequentially distributes lectures created for each learning unit. The lectures are displayed on the device in text or video format, allowing parents and children to study together.

[1254] As the learning progresses, the server generates quizzes and questions to check comprehension and distributes them to the device. The user (child) answers the questions and sends the results from the device to the server. The server analyzes the results and provides additional lectures or support for areas where understanding is insufficient.

[1255] The server records the user's learning history and level of understanding and reflects this in the next lesson. This allows for detailed management of learning progress and suggests appropriate learning units to tackle next. In particular, by utilizing natural language processing technology, it is possible to effectively analyze complex textbook data and generate optimal learning curricula.

[1256] As a specific example, when a user is studying mathematics with his / her child, the learning process will proceed as follows:

[1257] 1. A user uploads a mathematics textbook PDF to the system.

[1258] 2. The server analyzes the textbook PDF and automatically generates units such as "Fraction Fundamentals," "Applications of Fractions," and "Solving Equations."

[1259] 3. Users (parents and children) take the initial comprehension test and send the results to the server.

[1260] 4. Based on the results, the server suggests starting the lesson with the first unit, "Fractions Basics."

[1261] 5. The device displays a video lecture on "Fraction Basics" that parents and children watch together.

[1262] 6. The server will deliver a quiz to check comprehension after the lecture.

[1263] 7. The user (child) answers the quiz and sends the results to the server.

[1264] 8. The server analyzes the quiz results and provides additional lectures on areas where understanding is insufficient.

[1265] 9. The server will record progress and suggest the next unit, "Applications of Fractions."

[1266] Example prompt sentence:

[1267] "Please upload the PDF file of your textbook."

[1268] "Please take the comprehension test."

[1269] "Let's begin the next unit of study."

[1270] This system can reduce parental anxiety and improve children's learning outcomes.

[1271] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1272] Step 1:

[1273] Users upload PDF files of textbooks to the system.

[1274] Input: Textbook PDF file

[1275] Output: PDF file data sent to the server

[1276] Specific operation: The user opens the browser on the terminal and clicks the "Upload textbook" button on the system. A file dialog opens, selects the textbook PDF file, and presses the "Open" button. The terminal then sends this file to the server.

[1277] Step 2:

[1278] The server analyzes the received textbook PDF file.

[1279] Input: PDF file data sent to the server

[1280] Output: Text data and data categorized into chapters, sections, and topics

[1281] What it does: The server runs a Python script that first reads each page of the PDF file as an image, then uses Tesseract OCR to convert the pages to text, then uses the spaCy library to parse the extracted text and categorize it by chapter, section, and topic.

[1282] Step 3:

[1283] The server automatically generates a learning curriculum based on the analyzed data.

[1284] Input: Parsed text data and classification data

[1285] Output: Learning curriculum and text lectures, videos, quiz data

[1286] Specific operation: The server runs an algorithm that automatically divides learning units based on the analyzed text data. Then, for each unit, it calls OpenAI's GPT-3 to generate a text lecture. It also uses DALL-E to generate related videos and a set of quiz questions.

[1287] Step 4:

[1288] Users (parents and children) log into the system and take an initial comprehension test.

[1289] Input: User ID and password, comprehension test answer data

[1290] Output: Test result data

[1291] Specific operation: The user enters their ID and password on the login screen to log in to the system. After logging in, a comprehension test screen appears, and several questions are asked. Parents and children answer the quiz together, and when they are done, the results are sent from the device to the server.

[1292] Step 5:

[1293] The server analyzes the test results and sets the appropriate difficulty level and starting point.

[1294] Input: Comprehension test result data

[1295] Output: User-specific learning starting points and difficulty settings

[1296] How it works: The server analyzes the test results and runs algorithms to determine the starting point and difficulty level. Based on these results, it creates a personalized learning plan for the user.

[1297] Step 6:

[1298] The server sequentially distributes lectures created for each learning unit.

[1299] Input: Learning curriculum data, user-specific learning plans

[1300] Output: Text lecture, video lecture

[1301] Specific operation: The server determines the next learning unit and sends the URL of the corresponding text lecture or video file to the device. The device displays the lecture in the browser, and if it is a video lecture, it will automatically play it.

[1302] Step 7:

[1303] The server generates quizzes and questions to check comprehension as the learning progresses and distributes them to the device.

[1304] Input: The user's current learning progress data

[1305] Output: Quizzes and questions to check comprehension

[1306] Specific operation: The server tracks the user's learning progress data and generates quizzes to check their comprehension. The generated quizzes are sent to the device and displayed to the user.

[1307] Step 8:

[1308] The user (child) answers the questions and sends the results from the terminal to the server.

[1309] Input: User's quiz answer data

[1310] Output: Quiz result data

[1311] Specific operation: The user (child) answers the quiz displayed on the device. When the answer is complete, the device sends the result data to the server.

[1312] Step 9:

[1313] The server analyzes the quiz results and provides additional lectures and assistance for areas where understanding is lacking.

[1314] Input: Quiz result data

[1315] Output: Additional lectures and support

[1316] How it works: The server analyzes the quiz results, runs an algorithm to identify areas of insufficient understanding, and generates additional lectures based on the results and delivers them to the device.

[1317] Step 10:

[1318] The server records the user's learning history and level of understanding and reflects this in the next learning session.

[1319] Input: Learning history data and comprehension data

[1320] Output: Next lesson plan

[1321] Specific operation: The server stores the user's learning history and level of understanding in a database and executes an algorithm to reflect this in the next learning session. It also sets the learning curriculum to be tackled next.

[1322] (Application example 1)

[1323] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1324] As part of education, there is a lack of support for parents and children to study together effectively and efficiently.In addition, it is difficult to provide an appropriate curriculum and follow-up that is tailored to each child's learning progress and level of understanding, which results in poor learning outcomes.

[1325] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1326] In this invention, the server includes means for receiving and analyzing textbook data from a user terminal that uploads the textbook data, means for automatically generating a learning curriculum based on the analyzed textbook data, and means for providing easy-to-understand lectures based on the generated learning curriculum, thereby enabling parents and children to study together effectively.

[1327] "Textbook Data" means digital data of textbooks used for educational purposes that is uploaded by a User to the System.

[1328] A "user terminal" is a device used to upload textbook data, check learning progress, and check comprehension, and typically refers to a smartphone, tablet, or PC.

[1329] "Analysis" is the process of converting uploaded textbook data into text data and categorizing the content by chapter, section, and topic.

[1330] A "learning curriculum" is a learning progress plan that is automatically generated based on analyzed textbook data, and includes specific learning units and assignments.

[1331] A "lecture" is educational content provided based on a learning curriculum and displayed to users in text or video format.

[1332] "Comprehension check" refers to quizzes and questions to check the level of understanding of the learning content after the lecture.

[1333] "Quiz generation" is the process of automatically creating quizzes and questions based on learning content using a generative AI model.

[1334] The "check result" is the result data of the comprehension check sent from the user terminal, which is analyzed by the server and reflected in the next learning content.

[1335] "Additional lectures" are educational content that is provided in addition to the parts of the lecture that are not fully understood based on the results of the comprehension check.

[1336] "Follow-up" refers to ongoing support and advice provided to users based on their learning history and level of understanding.

[1337] A "generative AI model" is an artificial intelligence model that uses machine learning technology to automatically generate quizzes and questions from text data.

[1338] "Progress" is data that indicates the progress of the user's learning, and is managed by the server.

[1339] A "learning unit" is a unit of study that is classified into a specific theme or topic within a learning curriculum.

[1340] This invention is a support system for parents and children to work together to effectively advance learning. This system automatically generates a learning curriculum based on textbook data, automatically generates quizzes to check comprehension using a generative AI model, and manages and supports learning progress through lectures and comprehension checks.

[1341] System configuration

[1342] 1. Enter textbook data

[1343] The user uploads a PDF file of the textbook to the system, and the device sends the file to the server.

[1344] 2. Analysis of textbook data

[1345] The server analyzes the received textbook PDF file, extracts text data, and uses natural language processing (NLP) technology to categorize the textbook content by chapter, section, and topic.

[1346] 3. Generating learning curriculum

[1347] The server automatically generates a learning curriculum based on the analyzed data. Specifically, it determines learning units such as "fraction calculations" and "solving equations," and generates text lectures, videos, and quizzes for each unit.

[1348] 4. Start your study session

[1349] Users (parents and children) log in to the system and take the initial comprehension test. The device sends the test results to the server, which analyzes the results and sets the appropriate difficulty level and starting point.

[1350] 5. Providing lectures

[1351] The server sequentially distributes lectures created for each learning unit, and the devices display the lectures in text or video format, allowing parents and children to study together.

[1352] 6. Generating and providing quizzes to check comprehension using generative AI models

[1353] The server automatically generates quizzes based on the learning content using a generative AI model. The device provides the generated quizzes to the user, who then answers the questions.

[1354] 7. Checking understanding and providing feedback

[1355] The user (child) answers the quiz and sends the results from the device to the server. The server analyzes the quiz results and provides additional lectures on areas where the child did not fully understand.

[1356] 8. Follow-up and continuous learning

[1357] The server records the user's learning history and level of understanding and reflects this in the next lesson.The server also analyzes the user's learning progress and suggests the next appropriate learning unit for the user.

[1358] Hardware and software:

[1359] This system runs on standard PCs and smart devices (smartphones, tablets). The software and libraries used are as follows:

[1360] PyMuPDF (for text extraction)

[1361] NLTK (for text tokenization)

[1362] Transformers (for model inference, especially Facebook's BART model)

[1363] Examples:

[1364] If a user uploads a "sixth grade math textbook," the learning process will proceed as follows:

[1365] 1. A user uploads a "6th grade mathematics textbook PDF."

[1366] 2. The server analyzes the PDF and automatically generates the units "Fundamentals of Arithmetic," "Understanding Fractions," and "Fundamentals of Equations."

[1367] 3. Users (parents and children) take the initial comprehension test and send the results to the server.

[1368] 4. Based on the results, the server recommends starting with the first unit, "Fundamentals of Mathematics."

[1369] 5. The device displays a video lecture on "Mathematics Basics" that parents and children watch together.

[1370] 6. The server automatically generates a quiz using a generative AI model to check comprehension after the lecture and distributes it to the device.

[1371] 7. The user (child) answers the quiz and sends the results to the server.

[1372] 8. The server analyzes the quiz results and provides additional lectures on areas where understanding is insufficient.

[1373] 9. The server will record progress and suggest the next unit, "Understanding Fractions."

[1374] Example prompt sentence:

[1375] Generated curriculum text:

[1376] "Understanding fractions involves the following steps: First, understand the basic concept of fractions. Then practice adding and subtracting fractions. Finally, learn how to multiply and divide fractions."

[1377] Quiz generation prompt:

[1378] "Based on this text, please formulate the following questions:"

[1379] The question that is output is:

[1380] 1. What is the basic concept of fractions?

[1381] 2. Explain adding and subtracting fractions.

[1382] 3. How do you multiply and divide fractions?

[1383] This allows parents and children to study together effectively.

[1384] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1385] Step 1:

[1386] The user uploads a PDF file of a textbook to the terminal. The input is the PDF file of the textbook, and the output is the textbook PDF data sent to the server.

[1387] Step 2:

[1388] The server analyzes the received textbook PDF file and extracts the text data. Specifically, it uses the PyMuPDF library to extract text from the PDF. In this process, the input is the PDF file and the output is the text data.

[1389] Step 3:

[1390] The server uses natural language processing techniques to classify the extracted text data into chapters, sections, and topics. It uses the NLTK library to tokenize sentences and identify each section. The input of this process is the extracted text data, and the output is the classified text data.

[1391] Step 4:

[1392] The server automatically generates a learning curriculum based on the analyzed data. Specifically, it generates learning units such as "fraction calculation" and "solving equations" from the analyzed data. The input for this process is classified text data, and the output is a learning curriculum.

[1393] Step 5:

[1394] Users (parents and children) log in to the system and take the initial comprehension test. The terminal sends the test results to the server. The input is the comprehension test questions and their answers, and the output is the answer data sent to the server.

[1395] Step 6:

[1396] The server analyzes the results of the comprehension test and sets the appropriate difficulty level and starting point. The analysis uses the received answer data to determine the initial learning unit. The input of this process is the answer data, and the output is the initial learning unit and its difficulty setting.

[1397] Step 7:

[1398] The server sequentially distributes lectures created for each learning unit. The terminal displays them to the user in text or video format. The content of the lecture is based on the learning curriculum determined in the previous step. The input of this process is the learning curriculum, and the output is the lecture content to be distributed.

[1399] Step 8:

[1400] The server uses a generative AI model to automatically generate quizzes based on the learning content. The model uses the BART model from the Transformers library and prompts to generate quizzes. The inputs to this process are the learning content and prompts, and the output is an automatically generated quiz.

[1401] Step 9:

[1402] The user (child) answers the generated quiz and sends it from the terminal to the server. The input is the answer to the quiz, and the output is the answer data sent to the server.

[1403] Step 10:

[1404] The server analyzes the quiz results and provides additional lectures for areas where the student did not fully understand the quiz. The input to this process is the answer data, and the output is additional lectures as needed.

[1405] Step 11:

[1406] The server records the user's learning history and level of understanding and suggests the next learning unit. Based on the recorded data, the server analyzes the learning history and level of understanding and provides the next learning curriculum. The input for this process is the learning history and quiz results, and the output is a suggestion for the next learning unit.

[1407] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1408] This invention is an educational support system that enables parents and children to work together to effectively advance learning, and it also includes an emotion engine that recognizes the user's emotions and customizes the learning experience. This system analyzes textbook data, automatically generates learning curricula, conducts lectures and comprehension checks, and records the user's learning history and emotional state to reflect in the next lesson.

[1409] System configuration

[1410] Textbook data entry

[1411] Users upload textbook PDF files to the system. The device sends the files to the server, which then analyzes the received textbook PDF files to extract text data. Natural language processing (NLP) techniques are used to categorize the textbook content by chapter, section, and topic.

[1412] Learning curriculum generation

[1413] The server generates learning units from the analyzed textbook data. These units include "fraction calculations" and "solving equations." Each unit is associated with a text lecture, video content, and quizzes.

[1414] Start a study session

[1415] Users (parents and children) log in to the system and take the initial comprehension test. The device sends the test results to the server, which analyzes the results and sets the appropriate difficulty level and starting point.

[1416] Lecture provision

[1417] The server delivers lectures for the designated learning unit to the device, which displays the lectures in text or video format, allowing parents and children to study together.

[1418] Check your understanding

[1419] As the learning progresses, the server generates quizzes and questions to check comprehension and distributes them to the device. The user (child) answers the questions, and the device sends the results to the server. The server analyzes the results and provides additional lectures or follow-ups for areas where understanding is insufficient.

[1420] Follow-up and continuous learning

[1421] The server records the user's learning history and level of understanding and reflects this in the next lesson. The server also analyzes the user's learning progress and suggests the appropriate next learning unit. The device displays the lecture for the next learning unit and notifies the user.

[1422] User Emotion Recognition

[1423] The server contains an emotion engine and analyzes emotions from the user's facial expressions and voice. The device's built-in camera and microphone are used to capture users' (parents and children's) emotional data in real time and send it to the server. Based on the emotion analysis results, the server adjusts the system's behavior as follows:

[1424] If the user feels stressed while learning, the server will adjust the difficulty level and provide additional support to promote understanding.

[1425] If the user shows excitement or interest, progress the learning content to maintain that state.

[1426] Motivational feedback

[1427] Based on the analysis results of the emotion engine, the server generates encouragement and feedback that matches the user's emotional state. For example, if it determines that the user is tired, it will suggest taking a break, and if they are concentrating, it will display a message encouraging them to continue. This makes the learning experience for parents and children smoother and more comfortable.

[1428] Specific examples

[1429] For example, when a user is learning mathematics with his / her child, the user goes through the following process.

[1430] 1. The user uploads a PDF of a mathematics textbook, and the server analyzes the data and generates a learning curriculum.

[1431] 2. Users (parents and children) log in, take the initial test, and send the results to the server.

[1432] 3. Based on the analysis results, the server sets the first learning unit, "Fraction Basics," and delivers the lecture to the device.

[1433] 4. The device displays the lecture, and parents and children can learn together.

[1434] 5. The server sends a quiz to check comprehension, and the child sends the answers to the server.

[1435] 6. The server provides additional lectures based on comprehension analysis and adjusts the difficulty level as needed.

[1436] 7. The server records the learning history and level of understanding and suggests the next learning unit.

[1437] 8. The device will display the lecture for the next learning unit and you can continue learning.

[1438] 9. The server analyzes the user's emotions through an emotion engine and adjusts the learning content and lecture method as needed.

[1439] 10. Based on the results of the emotion analysis, the server generates encouraging and motivating feedback and delivers it to the device.

[1440] In this way, it is possible to effectively support parents and children in their learning and improve learning efficiency by customizing the learning experience for each individual. This system can reduce parental anxiety, which has traditionally been a challenge, and improve children's learning outcomes.

[1441] The processing flow will be explained below.

[1442] Specific processing flow of the program

[1443] Step 1:

[1444] Uploading textbook data

[1445] Users select the textbook PDF file and drag and drop it into the system's upload form.

[1446] The terminal sends the uploaded PDF file to the server.

[1447] Step 2:

[1448] Textbook data analysis

[1449] The server analyzes the received textbook PDF file and extracts the text data.

[1450] The server uses natural language processing (NLP) techniques to categorize the textbook content by chapter, section, and topic.

[1451] Step 3:

[1452] Learning curriculum generation

[1453] Based on the analyzed data, the server generates learning units for each chapter and section.

[1454] The server associates each unit with the appropriate text lectures, video content, and quizzes.

[1455] Step 4:

[1456] User Login

[1457] Users (parents and children) log into the system.

[1458] The terminal sends the user's login information to the server for authentication.

[1459] Step 5:

[1460] First comprehension test

[1461] The server provides comprehension tests to parents and children as a preliminary step in learning.

[1462] Users (parents and children) take the test and send the results from their device to the server.

[1463] The server analyzes the received results and evaluates the user's current level of understanding.

[1464] Step 6:

[1465] Setting appropriate learning units

[1466] Based on the results of the comprehension test, the server sets the initial learning unit according to the user's level.

[1467] The server delivers the lecture for the first learning unit to the terminal.

[1468] Step 7:

[1469] Lecture provision

[1470] The terminal displays the lecture of the learning unit received from the server (e.g., text, video).

[1471] Users (parents and children) watch lectures together and progress through their learning.

[1472] Step 8:

[1473] Comprehension check

[1474] After the lecture ends, the server generates quizzes and questions to check comprehension and distributes them to the device.

[1475] The user (child) answers questions and quizzes and sends the results from the terminal to the server.

[1476] Step 9:

[1477] Analysis of check results and feedback

[1478] The server analyzes the answers to the quizzes and questions and evaluates the user's level of understanding.

[1479] The server generates and delivers additional lectures and follow-ups for areas where understanding is deemed insufficient.

[1480] Step 10:

[1481] Suggestions for the next learning unit

[1482] The server records the user's progress and level of understanding and suggests the next unit to study.

[1483] The device will display the lecture for the next learning unit and notify the user.

[1484] Step 11:

[1485] Using the Emotion Engine

[1486] The device uses its built-in camera and microphone to collect the user's facial expressions and voice in real time.

[1487] The terminal transmits the collected data to the server.

[1488] The server uses an emotion engine to analyze the received data and assess the user's emotional state.

[1489] Step 12:

[1490] Adjustment based on emotional state

[1491] Based on the analysis results of the emotion engine, the server adjusts the difficulty of the task if the user is feeling stressed and provides additional support to promote understanding.

[1492] If the user shows excitement or interest, the server will progress the learning content to maintain that emotion.

[1493] Step 13:

[1494] Motivational feedback

[1495] Based on the analysis results of the emotion engine, the server generates encouragement and feedback that is in line with the user's emotional state.

[1496] The device displays generated feedback messages, such as suggesting a break if you're tired, or encouraging you to keep going if you're concentrating.

[1497] In this way, the system of the present invention responds not only to the learning progress but also to the emotional state of the user, making it possible to provide an effective learning experience.

[1498] Example 2

[1499] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1500] While conventional learning support systems can measure a user's learning situation and level of understanding, they have the problem of being unable to customize the system to take into account the user's emotional state. This reduces learning efficiency and makes it difficult to maintain user motivation. Furthermore, when parents and children study together, the lack of appropriate support increases parental anxiety and makes it difficult to improve children's learning outcomes.

[1501] The specific processing by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes: means for receiving and analyzing textbook data from a user terminal that uploads the textbook data; means for automatically generating a learning curriculum based on the analyzed textbook data; means for providing easy-to-understand lectures based on the generated learning curriculum; means for generating and providing quizzes and questions to check the level of understanding related to the lecture; means for receiving and analyzing the results of the comprehension check from the user terminal; means for providing additional lectures and follow-ups based on the check results; means for managing the overall progress of the system and suggesting appropriate learning units; and means for analyzing the emotional state of the user during learning in real time and adjusting the system operation based on the results. This provides an optimal learning experience that takes into account not only the user's learning situation but also their emotional state, thereby improving learning efficiency and motivation.

[1502] "Textbook data" refers to digital document data containing content used for educational purposes.

[1503] A "user terminal" is an electronic device used for learning activities such as uploading textbook data, viewing lectures, and answering quizzes.

[1504] "Means of analysis" refers to the technology and equipment used to analyze textbook data, extract the necessary information, and classify it.

[1505] A "learning curriculum" is a set of learning content and activity plans that are established to help users effectively advance their learning.

[1506] "Means for automatic generation" refers to technology or devices for automatically creating a learning curriculum from analyzed textbook data.

[1507] A "lecture" is an explanation or instruction of the learning content provided based on the learning curriculum.

[1508] "Quizzes and questions to check comprehension" are questions provided to assess the user's level of comprehension of the learning content.

[1509] The "means for providing additional lectures or follow-up based on the check results" refers to a technology or device that analyzes the results of the check on the user's comprehension and provides further guidance or supplementary explanations as necessary.

[1510] "Means for managing progress and suggesting appropriate learning units" refers to technologies and devices for tracking a user's learning progress and suggesting what content to learn next.

[1511] "Means for analyzing emotional states in real time" refers to technology or devices for detecting and analyzing emotions in real time from a user's facial expressions, voice, etc.

[1512] "Means for adjusting the system's operation" refers to technology or devices for appropriately changing the learning content or lecture method based on the results of emotion analysis.

[1513] The present invention relates to an educational support system for parents and children to effectively collaborate in learning, and in particular, to a system including an emotion engine that recognizes the user's emotions and customizes the learning experience. This system is realized through the interaction between a server, a terminal, and a user.

[1514] Hardware and software used

[1515] The hardware in this system is as follows:

[1516] Device: A personal computer (PC) or tablet device used at home or in an educational institution.

[1517] Server: A central management device that analyzes and manages data.

[1518] Camera and microphone: Equipped on the device, they capture the user's facial expressions and voice in real time.

[1519] The software used is as follows:

[1520] Natural Language Processing (NLP) techniques: used to analyze textbook data.

[1521] Sentiment Analysis Engine: Used to analyze the user's emotional state.

[1522] OCR (Optical Character Recognition) technology: Used to convert PDF files into text data.

[1523] Various modules: Examples include PDFReceiverService, NLPAnalyzer, TextClassifier, CurriculumGenerator, TestEvaluator, LectureDistributor, LectureViewer, QuizGenerator, AnswerAnalyzer, LearningHistoryRecorder, NextUnitRecommender, EmotionAnalyzer, EmotionCapture, FeedbackGenerator, and FeedbackDisplay.

[1524] System Description

[1525] 1. Enter textbook data

[1526] A user logs into the system and uploads a textbook PDF file from their device. The device sends this file to the server, which uses a PDFReceiverService to receive, store, and prepare the file for processing.

[1527] 2. Textbook data analysis and curriculum generation

[1528] The server uses the OCRProcessor to scan PDF files and extract text data. The extracted text data is then analyzed by the NLPAnalyzer and classified by chapter, section, and topic. The CurriculumGenerator automatically generates a learning curriculum based on this data, associating each unit with text lectures, video content, and quizzes.

[1529] 3. First comprehension test

[1530] Users (parents and children) log in to the system and take the initial comprehension test on their device. The device sends the test results to the server, which analyzes them using the TestEvaluator and sets the appropriate difficulty level and starting point.

[1531] 4. Providing lectures

[1532] The server distributes text and video lectures to devices using LectureDistributor based on the set learning units. The devices display these lectures using LectureViewer, allowing parents and children to study together.

[1533] 5. Check your understanding

[1534] As the learning progresses, the server uses QuizGenerator to generate quizzes and questions to check comprehension and distributes them to the device. The user (child) answers the quiz on the device and sends the results to the server. The server analyzes the answers using AnswerAnalyzer and provides additional lectures or follow-ups for areas where understanding is insufficient.

[1535] 6. Recording learning history and suggesting next learning units

[1536] The server uses LearningHistoryRecorder to record the user's learning history and level of understanding. NextUnitRecommender then uses this information to suggest the next learning unit. The device then displays the lecture for the suggested next learning unit and notifies the user.

[1537] 7. User Emotion Recognition

[1538] The server uses the emotion engine to analyze the user's facial expressions and voice captured by the device's camera and microphone in real time. The data is acquired through EmotionCapture and analyzed by EmotionAnalyzer to adjust the system's behavior.

[1539] 8. Feedback and Motivation

[1540] The server uses the FeedbackGenerator to provide encouragement and feedback to the user based on the sentiment analysis results, and the device uses the FeedbackDisplay to display this feedback and improve the quality of the learning experience.

[1541] Specific examples

[1542] For example, when a user is learning mathematics with a child, the following process is carried out.

[1543] 1. The user uploads a PDF of a mathematics textbook, and the server analyzes the data and generates a learning curriculum.

[1544] 2. Users (parents and children) log in, take the initial test, and send the results to the server.

[1545] 3. Based on the analysis results, the server sets the first learning unit, "Fraction Basics," and delivers the lecture to the device.

[1546] 4. The device displays the lecture, and parents and children can learn together.

[1547] 5. The server sends a quiz to check comprehension, and the child sends the answers to the server.

[1548] 6. The server provides additional lectures based on comprehension analysis and adjusts the difficulty level as needed.

[1549] 7. The server records the learning history and level of understanding and suggests the next learning unit.

[1550] 8. The device will display the lecture for the next learning unit and you can continue learning.

[1551] 9. The server analyzes the user's emotions through an emotion engine and adjusts the learning content and lecture method as needed.

[1552] 10. Based on the results of the emotion analysis, the server generates encouraging and motivating feedback and delivers it to the device.

[1553] Prompt Sentence Examples

[1554] "Generate learning units from the following PDF textbooks and customize lecture content based on comprehension checks and sentiment analysis."

[1555] As a result, parents and children can study effectively, and the learning effect and motivation of users can be improved.

[1556] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1557] Step 1: Enter textbook data

[1558] Input: Textbook PDF file

[1559] Specific operation: The user clicks the "Upload textbook" button on the terminal, selects and uploads a textbook PDF file from the local file system, and the terminal uses the FileUploader module to send the selected PDF file to the server.

[1560] Output: PDF file saved on the server

[1561] Data processing: The server starts the PDFReceiverService and saves the received file in a temporary folder.

[1562] Step 2: Analyzing textbook data

[1563] Input: PDF file stored on the server

[1564] How it works: The server uses OCRProcessor to scan PDF files and extract text data, which is then analyzed by NLPAnalyzer and classified into chapters, sections, and topics.

[1565] Output: Analyzed textbook data

[1566] Data processing: The server scans the PDF data as an image and converts it into text data. The converted text data is then analyzed using natural language processing technology to classify each piece of content.

[1567] Step 3: Generate a learning curriculum

[1568] Input: Parsed textbook data

[1569] Specific operation: The server uses CurriculumGenerator to automatically generate learning units from the parsed data, and associates text lectures, video content, and quizzes with the generated learning units.

[1570] Output: Generated learning curriculum

[1571] Data processing: Based on each extracted topic, the server generates appropriate learning units and adds related educational resources (text, videos, quizzes).

[1572] Step 4: First assessment

[1573] Input: User (parent and child) login information

[1574] Specific operation: A user logs in to the system and takes the initial comprehension test on their device. The device displays the test questions using QuizPresenter and the user enters their answers. The answer data is sent to the server via AnswerCollector.

[1575] Output: Test result data

[1576] Data processing: The server uses TestEvaluator to analyze the submitted answers and set appropriate difficulty levels and starting points.

[1577] Step 5: Deliver the lecture

[1578] Input: Learning curriculum and initial comprehension test results

[1579] Specific operation: The server distributes the lectures of the set learning unit to the device using LectureDistributor. The device uses LectureViewer to display the received lectures in text or video format.

[1580] Output: Visual and audio lecture data

[1581] Data processing: The server selects and sends the appropriate lecture data (text, video), which is then displayed on the device.

[1582] Step 6: Check your understanding

[1583] Input: Learning status data after the lecture

[1584] Specific operation: The server uses QuizGenerator to generate quizzes and questions for comprehension checks and distributes them to the device. The user (child) answers the quiz on the device and sends the results to the server via AnswerCollector.

[1585] Output: Comprehension check result data

[1586] Data processing: The server uses AnswerAnalyzer to analyze the answer data and identify any missing parts of understanding.

[1587] Step 7: Follow-up and learning history

[1588] Input: Comprehension check result data

[1589] Specific operation: The server uses LearningHistoryRecorder to record the user's learning history and recommends the best learning unit for the next lesson. NextUnitRecommender selects the next learning unit and sends it from the server to the device. The device then displays the next lecture and notifies the user.

[1590] Output: Updated learning history data and next learning unit

[1591] Data processing: The server identifies the next learning step based on the recorded learning history and comprehension data, and updates the optimal learning plan.

[1592] Step 8: Recognizing User Emotions

[1593] Input: User facial and voice data

[1594] How it works: The device uses a camera and microphone to capture the user's facial expressions and voice in real time. This data is sent to the server via EmotionCapture, and the server uses EmotionAnalyzer to analyze the user's emotional state.

[1595] Output: Sentiment analysis data

[1596] Data processing: The server applies facial expression recognition and voice analysis technologies to identify the user's emotional state, such as stress, excitement, or concentration, in real time.

[1597] Step 9: Feedback and motivation

[1598] Input: Sentiment analysis data

[1599] Specific operation: The server uses FeedbackGenerator to automatically generate appropriate encouragement and feedback for the user based on the results of sentiment analysis, and the device notifies the user using FeedbackDisplay.

[1600] Output: An informative feedback message

[1601] Data processing: The server generates feedback messages that reflect the user's emotional state and presents them to the user to personalize the learning experience and increase motivation.

[1602] This detailed process flow enables the system to provide a personalized educational experience tailored to the user's learning needs.

[1603] (Application example 2)

[1604] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1605] Conventional educational support systems simply provide learning curricula and lack functionality to respond to users' emotions and individual progress. This makes it difficult to respond appropriately to declines in motivation or differences in understanding, especially in children's learning. Furthermore, when parents and children study together, the burden on parents increases, making it difficult to provide effective support.

[1606] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving and analyzing textbook data from a user terminal that uploads the textbook data, means for automatically generating a learning curriculum based on the analyzed textbook data, means for providing easy-to-understand lectures based on the generated learning curriculum, means for generating and providing quizzes and questions to check the level of understanding related to the lecture, means for receiving and analyzing the results of the comprehension check from the user terminal, means for providing additional lectures and follow-ups based on the check results, means for managing overall progress and suggesting appropriate learning units, and means for recognizing the user's emotions and customizing the learning experience. This enables flexible response according to the user's learning status, allowing parents and children to study together effectively.

[1607] "Textbook data" refers to digital data of books and teaching materials used in education.

[1608] A "user device" is an electronic device such as a computer, tablet, or smartphone used by a learner.

[1609] "Analysis" refers to the process of understanding and classifying the content of textbook data using techniques such as natural language processing.

[1610] A "curriculum" is a set of learning content and activities planned in accordance with educational objectives.

[1611] A "lecture" is educational content that provides explanations and instruction on a certain topic.

[1612] A "quiz" is a question or problem that allows you to check what you have learned.

[1613] A "comprehension check" is a process of assessing how well a learner has understood the learning content.

[1614] "Following up" means providing additional instruction or materials to reinforce areas of incomplete understanding.

[1615] "Progress" refers to the degree to which a learner has progressed in the learning curriculum.

[1616] A "learning unit" is a piece of learning organized around a specific educational goal.

[1617] "Emotion recognition" is a technology that analyzes emotions from a user's facial expressions, voice, etc.

[1618] A "learning experience" is any experience or sensation a learner has in the process of learning.

[1619] The present invention is an educational support system for parents and children to effectively collaborate in learning, and further includes an emotion engine that recognizes the user's emotions and customizes the learning experience.

[1620] System configuration

[1621] Textbook data entry

[1622] Users upload textbook PDF files to the system. The user's device sends the files to the server, which then analyzes the received textbook PDF files and extracts text data. Natural language processing (NLP) techniques are used to categorize the textbook content by chapter, section, and topic. For this purpose, Python and NLP libraries (e.g., spaCy) are used.

[1623] Learning curriculum generation

[1624] The server generates a learning curriculum from the analyzed textbook data. The generated units include "fraction calculation" and "solving equations." Each unit is associated with a text lecture, video content, and quizzes.

[1625] Start a study session

[1626] Users (parents and children) log in to the system and take the initial comprehension test. The user device sends the test results to the server, which analyzes the results and sets an appropriate difficulty level and starting point.

[1627] Lecture provision

[1628] The server delivers lectures for the selected learning unit to the user's device, which displays the lecture in text or video format, allowing parents and children to study together.

[1629] Check your understanding

[1630] As the learning progresses, the server generates quizzes and questions to check comprehension and distributes them to the user's device. The user (child) answers the questions, and the user's device sends the results to the server. The server analyzes the results and provides additional lectures or follow-ups for areas where understanding is insufficient.

[1631] Follow-up and continuous learning

[1632] The server records the user's learning history and level of understanding and reflects this in the next lesson. The server also analyzes the user's learning progress and suggests the appropriate next learning unit. The user's device displays the lecture for the next learning unit and notifies the user.

[1633] User Emotion Recognition

[1634] The server contains an emotion engine and analyzes emotions from the user's facial expressions and voice. The camera and microphone installed on the user's device are used to capture the user's (parent and child) emotional data in real time and send it to the server. Based on the emotion analysis results, the server adjusts the system's operation as follows:

[1635] If the user feels stressed while learning, the server will adjust the difficulty level and provide additional support to promote understanding.

[1636] If the user shows excitement or interest, progress the learning content to maintain that state.

[1637] Motivational feedback

[1638] Based on the analysis results of the emotion engine, the server generates encouragement and feedback that matches the user's emotional state. For example, if it determines that the user is tired, it will suggest taking a break, and if they are concentrating, it will display a message encouraging them to continue. This makes the learning experience for parents and children smoother and more comfortable.

[1639] Specific examples

[1640] For example, when a user and his / her child are learning mathematics in a virtual store, the user goes through the following process.

[1641] 1. The user uploads a PDF of a mathematics textbook, and the server analyzes the data and generates a learning curriculum.

[1642] 2. Users (parents and children) log in, take the initial test, and send the results to the server.

[1643] 3. Based on the analysis results, the server sets the first learning unit, "Fraction Basics," and delivers the lecture to the user's device.

[1644] 4. The user device displays the lecture, and the parent and child learn together.

[1645] 5. The server sends a quiz to check comprehension, and the child sends the answers to the server.

[1646] 6. The server provides additional lectures based on comprehension analysis and adjusts the difficulty level as needed.

[1647] 7. The server records the learning history and level of understanding and suggests the next learning unit.

[1648] 8. The user's device will display the lecture for the next learning unit and continue learning.

[1649] 9. The server analyzes the user's emotions through an emotion engine and adjusts the learning content and lecture method as needed.

[1650] 10. Based on the results of the emotion analysis, the server generates encouraging and motivating feedback and delivers it to the user's device.

[1651] Example prompt for a generative AI model:

[1652] "Analyze the content of the fourth grade textbook "Calculating Fractions" and generate a lecture in the following format.

[1653] 1. Lecture Title

[1654] 2. Lecture Overview

[1655] 3. List of key points

[1656] 4. Quiz questions to check comprehension and sample answers

[1657] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1658] Step 1:

[1659] The user uploads a PDF file of a textbook to the system. The user's device sends this file to the server. The input is the PDF data of the textbook, and the output is the PDF file sent to the server. Specifically, a file selection dialog opens and a request is made to send the file selected by the user to the server.

[1660] Step 2:

[1661] The server analyzes the received textbook PDF file and extracts the text data. The input is the PDF file, and the output is the analyzed text data. Specifically, the PDFMiner library is used to convert the contents of the PDF file into text format.

[1662] Step 3:

[1663] The server uses natural language processing (NLP) techniques to classify the parsed text data into chapters, sections, and topics. The input is text data, and the output is classified text data. Specifically, the spaCy library is used to split the text and store it as structured data.

[1664] Step 4:

[1665] The server automatically generates a learning curriculum from structured text data. The input is classified text data, and the output is a learning curriculum. Specifically, it automatically generates related lectures and quizzes for each chapter and compiles them into learning units.

[1666] Step 5:

[1667] Users (parents and children) log in to the system and take the initial comprehension test. The user device sends the test results to the server. The input is the answer data to the comprehension test, and the output is the test results sent for analysis. In concrete terms, the user answers the test and sends the results to the server.

[1668] Step 6:

[1669] The server analyzes the initial test results and sets the appropriate difficulty level and starting point. The input is the test results, and the output is the appropriate difficulty setting. Specifically, it determines the optimal learning level based on the test results and selects the next learning unit.

[1670] Step 7:

[1671] The server delivers lectures for the set learning units to the user's device. The input is the learning unit data, and the output is the lecture data for the user's device. Specifically, the server sends the lecture in text or video format for display on the user's device.

[1672] Step 8:

[1673] The user device displays the lecture, and parents and children can study together. The input is lecture data, and the output is the lecture displayed on the user device. In concrete terms, the user device provides a UI that displays the lecture content.

[1674] Step 9:

[1675] The server generates quizzes and questions to check comprehension and distributes them to the user's device. The input is the learning unit data, and the output is the quizzes and questions sent to the user's device. Specifically, the server generates quizzes using a Python script and sends them to the user's device.

[1676] Step 10:

[1677] The user (child) answers quizzes and questions, and the user device sends the results to the server. The input is the answer data to the quizzes and questions, and the answer results sent to the server are obtained as output. In concrete terms, the user answers the quiz and sends the results to the server.

[1678] Step 11:

[1679] The server analyzes the answer results and provides additional lectures or follow-ups for areas where understanding is insufficient. The answer result data is input, and additional lecture data is obtained as output. Specifically, the server analyzes the answer results, generates necessary supplementary explanations, and sends them to the user's device.

[1680] Step 12:

[1681] The server records the user's learning history and level of understanding and reflects this in the next lesson. The input is comprehension level and learning history data, and the output is the next learning plan. Specific operations include saving the learning history and level of understanding in a database and planning the next learning unit.

[1682] Step 13:

[1683] The server analyzes emotions from the user's facial expressions and voice. Facial expressions and voice data are input, and emotion analysis results are obtained as output. Specifically, emotions are analyzed using OpenCV and the Google Cloud Speech API.

[1684] Step 14:

[1685] The server adjusts the learning content and lecture method based on the emotion analysis results. The emotion analysis results are input, and the adjusted learning content is obtained as output. Specifically, the server adjusts the difficulty and progress of the learning content according to the emotional state.

[1686] Step 15:

[1687] The server generates encouraging and motivating feedback based on the emotion analysis results and delivers it to the user's device. The emotion analysis results are input, and feedback messages are obtained as output. Specifically, the server generates messages of encouragement or suggestions for taking a break based on the user's emotional state and sends them to the user's device.

[1688] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1689] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1690] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1691] [Fourth embodiment]

[1692] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1693] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1694] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1695] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1696] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1697] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1698] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1699] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1700] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1701] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1702] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1703] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1704] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1705] This invention is a support system for parents and children to work together to effectively advance learning. This system automatically generates learning curricula based on textbook data, and manages and supports learning progress through lectures and comprehension checks.

[1706] System configuration

[1707] Textbook data entry

[1708] The user uploads a PDF file of the textbook to the system, and the device sends the file to the server.

[1709] Textbook data analysis

[1710] The server analyzes the received textbook PDF file, extracts text data, and uses natural language processing (NLP) techniques to categorize the textbook content by chapter, section, and topic.

[1711] Learning curriculum generation

[1712] The server automatically generates a learning curriculum based on the analyzed data. Specifically, it determines learning units such as "fraction calculations" and "solving equations," and generates text lectures, videos, and quizzes for each unit.

[1713] Start a study session

[1714] Users (parents and children) log in to the system and take the initial comprehension test. The device sends the test results to the server, which analyzes the results and sets the appropriate difficulty level and starting point.

[1715] Lecture provision

[1716] The server sequentially distributes lectures created for each learning unit, and the devices display the lectures in text or video format, allowing parents and children to study together.

[1717] Check your understanding

[1718] The server generates quizzes and questions to check comprehension as the learning progresses and distributes them to the device. The user (child) answers the questions, and the device sends the results to the server. The server analyzes the results and provides additional lectures for areas where understanding is insufficient.

[1719] Follow-up and continuous learning

[1720] The server records the user's learning history and level of understanding and reflects this in the next lesson.The server also analyzes the user's learning progress and suggests the next appropriate learning unit for the user.

[1721] Specific examples

[1722] For example, if a user wants to study mathematics with his or her child, the learning process will proceed as follows:

[1723] 1. A user uploads a mathematics textbook PDF to the system.

[1724] 2. The server analyzes the textbook PDF and automatically generates units such as "Fraction Fundamentals," "Applications of Fractions," and "Solving Equations."

[1725] 3. Users (parents and children) take the initial comprehension test and send the results to the server.

[1726] 4. Based on the results, the server recommends starting with the first unit, "Fractions Basics."

[1727] 5. The device displays a video lecture on "Fraction Basics" that parents and children watch together.

[1728] 6. The server distributes a quiz to check comprehension after the lecture.

[1729] 7. The user (child) answers the quiz and sends the results to the server.

[1730] 8. The server analyzes the quiz results and provides additional lectures on areas where understanding is insufficient.

[1731] 9. The server will record progress and suggest the next unit, "Applications of Fractions."

[1732] In this way, parents and children can effectively advance their learning through the system. The present invention can reduce parental anxiety, which has been a problem in the past, and improve children's learning effectiveness.

[1733] The processing flow will be explained below.

[1734] Specific processing flow of the program

[1735] Step 1:

[1736] Uploading textbook data

[1737] Users select the textbook PDF file and drag and drop it into the system's upload form.

[1738] The terminal sends the uploaded PDF file to the server.

[1739] Step 2:

[1740] Textbook data analysis

[1741] The server analyzes the received textbook PDF file and extracts the text data.

[1742] The server uses natural language processing (NLP) techniques to categorize the textbook content by chapter, section, and topic.

[1743] Step 3:

[1744] Learning curriculum generation

[1745] Based on the analyzed data, the server generates learning units for each chapter and section.

[1746] The server associates each unit with the appropriate text lectures, video content, and quizzes.

[1747] Step 4:

[1748] User Login

[1749] Users (parents and children) log into the system.

[1750] The terminal sends the user's login information to the server for authentication.

[1751] Step 5:

[1752] First comprehension test

[1753] The server provides comprehension tests to parents and children as a preliminary step in learning.

[1754] Users (parents and children) take the test and send the results from their device to the server.

[1755] The server analyzes the received results and evaluates the user's current level of understanding.

[1756] Step 6:

[1757] Setting appropriate learning units

[1758] Based on the results of the comprehension test, the server sets the initial learning unit according to the user's level.

[1759] The server delivers the lecture for the first learning unit to the terminal.

[1760] Step 7:

[1761] Lecture provision

[1762] The terminal displays the lecture of the learning unit received from the server (e.g., text, video).

[1763] Users (parents and children) watch lectures together and progress through their learning.

[1764] Step 8:

[1765] Comprehension check

[1766] After the lecture ends, the server generates quizzes and questions to check comprehension and distributes them to the device.

[1767] The user (child) answers questions and quizzes and sends the results from the terminal to the server.

[1768] Step 9:

[1769] Analysis of check results and feedback

[1770] The server analyzes the answers to the quizzes and questions and evaluates the user's level of understanding.

[1771] The server generates and delivers additional lectures and follow-ups for areas where understanding is deemed insufficient.

[1772] Step 10:

[1773] Suggestions for the next learning unit

[1774] The server records the user's progress and level of understanding and suggests the next unit to study.

[1775] The device will display the lecture for the next learning unit and notify the user.

[1776] In this way, the system provides support at each stage of the process to help parents and children learn effectively.

[1777] Example 1

[1778] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1779] In recent years, there has been a demand for support systems that allow parents and children to study together, but conventional systems have the following issues.

[1780] 1. It is difficult to efficiently analyze textbook data and automatically generate appropriate learning programs.

[1781] 2. There is insufficient checking of the level of understanding of the learning content and follow-up on any deficiencies.

[1782] 3. Lack of interfaces for parents and children to learn together.

[1783] 4. There is a lack of functionality to administer an initial comprehension test and set a starting point for learning based on the results.

[1784] 5. It is not possible to continuously record learning history and comprehension and reflect this in the next learning session.

[1785] This makes it difficult to alleviate parents' anxiety and improve children's learning outcomes.

[1786] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1787] In this invention, the server includes means for receiving and analyzing textbook data from a user device that uploads the textbook data, means for automatically generating a study program based on the analyzed textbook data, and means for providing explanations based on the generated study program. This makes it possible to efficiently analyze complex textbook data and automatically generate an optimal study program.

[1788] In addition, in this invention, the server includes means for generating and providing questions to check the level of understanding related to the explanation, means for receiving and analyzing the results of the comprehension check from the user device, means for providing additional explanations and support based on the check results, and means for managing the overall learning progress of the system and suggesting appropriate learning units. This makes it possible to appropriately manage the user's learning status and effectively follow up on areas where understanding is lacking.

[1789] Furthermore, in this invention, the server includes means for analyzing textbook content using natural language processing technology, means for generating text explanations and videos using a generation AI module, means for conducting a comprehension test before the start of learning and setting a starting point based on the results, means for providing a user interface for parents and children to learn together, and means for recording the user's learning history and comprehension and reflecting this in the next learning session. This makes it possible to efficiently manage the entire learning process and support parent-child collaborative learning.

[1790] "Textbook data" refers to digital information on books and teaching materials used for education.

[1791] A "user device" is a terminal device operated by a user, and includes, for example, a personal computer, a tablet, a smartphone, etc.

[1792] "Analysis" refers to the act of breaking down complex data and information into an easy-to-understand form and classifying and organizing each element.

[1793] "Program of Study" refers to a set of educational plans or curricula designed to achieve specific learning objectives.

[1794] "Explanation" refers to explanations and answers that make the learning content easy to understand.

[1795] A "comprehension check" refers to a test or quiz that assesses how well the material has been understood.

[1796] "Questions" refer to questions or tasks that are used to assess comprehension of specific learning content.

[1797] "Support" refers to assistance and guidance provided to help users learn more efficiently.

[1798] "Learning progress" refers to data and records that show the progress a user is making at each stage.

[1799] "Natural language processing technology" refers to computer technology for understanding and processing human language.

[1800] "Generative AI Module" means an artificial intelligence program for automatically generating text or other content.

[1801] "Study history" refers to data that records what a user has learned and their progress to date.

[1802] "User interface" refers to the screens and operation methods designed to make it easy for users to operate the system.

[1803] "Comprehension test" refers to a test to assess a user's current level of knowledge before learning.

[1804] This invention relates to a support system for parents and children to work together to effectively advance their learning. The system of this invention automatically generates individual learning curricula using digital data from textbooks, and effectively manages and supports learning progress.

[1805] The user begins the learning process by uploading a textbook PDF file to the system. The textbook data is uploaded to the system by the user via a device, which can be a PC, tablet, or smartphone. The uploaded textbook data is then sent to the server via communication.

[1806] The server analyzes the received textbook PDF file using natural language processing (NLP) techniques. Specifically, it uses Tesseract OCR to extract text from PDF pages, and then uses TensorFlow and spaCy to analyze the text data. This categorizes the textbook content by chapter, section, and topic.

[1807] The server automatically generates a learning curriculum based on the analysis results. The generated curriculum is divided into learning units such as "Calculating Fractions" and "Solving Equations," and for each unit, text lectures, videos, and quizzes to check comprehension are generated. OpenAI's GPT-3 is used to generate the text lectures, and DALL-E is used to generate the videos.

[1808] Users (parents and children) log in to the system and take an initial comprehension test. The test results are sent from the device to the server, where they are analyzed. Based on the analysis results, the starting point for learning and the optimal level of difficulty are set. The comprehension test is conducted by answering questions, and the answer data is sent to the server.

[1809] Once learning begins, the server sequentially distributes lectures created for each learning unit. The lectures are displayed on the device in text or video format, allowing parents and children to study together.

[1810] As the learning progresses, the server generates quizzes and questions to check comprehension and distributes them to the device. The user (child) answers the questions and sends the results from the device to the server. The server analyzes the results and provides additional lectures or support for areas where understanding is insufficient.

[1811] The server records the user's learning history and level of understanding and reflects this in the next lesson. This allows for detailed management of learning progress and suggests appropriate learning units to tackle next. In particular, by utilizing natural language processing technology, it is possible to effectively analyze complex textbook data and generate optimal learning curricula.

[1812] As a specific example, when a user is studying mathematics with his / her child, the learning process will proceed as follows:

[1813] 1. A user uploads a mathematics textbook PDF to the system.

[1814] 2. The server analyzes the textbook PDF and automatically generates units such as "Fraction Fundamentals," "Applications of Fractions," and "Solving Equations."

[1815] 3. Users (parents and children) take the initial comprehension test and send the results to the server.

[1816] 4. Based on the results, the server suggests starting the lesson with the first unit, "Fractions Basics."

[1817] 5. The device displays a video lecture on "Fraction Basics" that parents and children watch together.

[1818] 6. The server will deliver a quiz to check comprehension after the lecture.

[1819] 7. The user (child) answers the quiz and sends the results to the server.

[1820] 8. The server analyzes the quiz results and provides additional lectures on areas where understanding is insufficient.

[1821] 9. The server will record progress and suggest the next unit, "Applications of Fractions."

[1822] Example prompt sentence:

[1823] "Please upload the PDF file of your textbook."

[1824] "Please take the comprehension test."

[1825] "Let's begin the next unit of study."

[1826] This system can reduce parental anxiety and improve children's learning outcomes.

[1827] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1828] Step 1:

[1829] Users upload PDF files of textbooks to the system.

[1830] Input: Textbook PDF file

[1831] Output: PDF file data sent to the server

[1832] Specific operation: The user opens the browser on the terminal and clicks the "Upload textbook" button on the system. A file dialog opens, selects the textbook PDF file, and presses the "Open" button. The terminal then sends this file to the server.

[1833] Step 2:

[1834] The server analyzes the received textbook PDF file.

[1835] Input: PDF file data sent to the server

[1836] Output: Text data and data categorized into chapters, sections, and topics

[1837] What it does: The server runs a Python script that first reads each page of the PDF file as an image, then uses Tesseract OCR to convert the pages to text, then uses the spaCy library to parse the extracted text and categorize it by chapter, section, and topic.

[1838] Step 3:

[1839] The server automatically generates a learning curriculum based on the analyzed data.

[1840] Input: Parsed text data and classification data

[1841] Output: Learning curriculum and text lectures, videos, quiz data

[1842] Specific operation: The server runs an algorithm that automatically divides learning units based on the analyzed text data. Then, for each unit, it calls OpenAI's GPT-3 to generate a text lecture. It also uses DALL-E to generate related videos and a set of quiz questions.

[1843] Step 4:

[1844] Users (parents and children) log into the system and take an initial comprehension test.

[1845] Input: User ID and password, comprehension test answer data

[1846] Output: Test result data

[1847] Specific operation: The user enters their ID and password on the login screen to log in to the system. After logging in, a comprehension test screen appears, and several questions are asked. Parents and children answer the quiz together, and when they are done, the results are sent from the device to the server.

[1848] Step 5:

[1849] The server analyzes the test results and sets the appropriate difficulty level and starting point.

[1850] Input: Comprehension test result data

[1851] Output: User-specific learning starting points and difficulty settings

[1852] How it works: The server analyzes the test results and runs algorithms to determine the starting point and difficulty level. Based on these results, it creates a personalized learning plan for the user.

[1853] Step 6:

[1854] The server sequentially distributes lectures created for each learning unit.

[1855] Input: Learning curriculum data, user-specific learning plans

[1856] Output: Text lecture, video lecture

[1857] Specific operation: The server determines the next learning unit and sends the URL of the corresponding text lecture or video file to the device. The device displays the lecture in the browser, and if it is a video lecture, it will automatically play it.

[1858] Step 7:

[1859] The server generates quizzes and questions to check comprehension as the learning progresses and distributes them to the device.

[1860] Input: The user's current learning progress data

[1861] Output: Quizzes and questions to check comprehension

[1862] Specific operation: The server tracks the user's learning progress data and generates quizzes to check their comprehension. The generated quizzes are sent to the device and displayed to the user.

[1863] Step 8:

[1864] The user (child) answers the questions and sends the results from the terminal to the server.

[1865] Input: User's quiz answer data

[1866] Output: Quiz result data

[1867] Specific operation: The user (child) answers the quiz displayed on the device. When the answer is complete, the device sends the result data to the server.

[1868] Step 9:

[1869] The server analyzes the quiz results and provides additional lectures and assistance for areas where understanding is lacking.

[1870] Input: Quiz result data

[1871] Output: Additional lectures and support

[1872] How it works: The server analyzes the quiz results, runs an algorithm to identify areas of insufficient understanding, and generates additional lectures based on the results and delivers them to the device.

[1873] Step 10:

[1874] The server records the user's learning history and level of understanding and reflects this in the next learning session.

[1875] Input: Learning history data and comprehension data

[1876] Output: Next lesson plan

[1877] Specific operation: The server stores the user's learning history and level of understanding in a database and executes an algorithm to reflect this in the next learning session. It also sets the learning curriculum to be tackled next.

[1878] (Application example 1)

[1879] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1880] As part of education, there is a lack of support for parents and children to study together effectively and efficiently.In addition, it is difficult to provide an appropriate curriculum and follow-up that is tailored to each child's learning progress and level of understanding, which results in poor learning outcomes.

[1881] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1882] In this invention, the server includes means for receiving and analyzing textbook data from a user terminal that uploads the textbook data, means for automatically generating a learning curriculum based on the analyzed textbook data, and means for providing easy-to-understand lectures based on the generated learning curriculum, thereby enabling parents and children to study together effectively.

[1883] "Textbook Data" means digital data of textbooks used for educational purposes that is uploaded by a User to the System.

[1884] A "user terminal" is a device used to upload textbook data, check learning progress, and check comprehension, and typically refers to a smartphone, tablet, or PC.

[1885] "Analysis" is the process of converting uploaded textbook data into text data and categorizing the content by chapter, section, and topic.

[1886] A "learning curriculum" is a learning progress plan that is automatically generated based on analyzed textbook data, and includes specific learning units and assignments.

[1887] A "lecture" is educational content provided based on a learning curriculum and displayed to users in text or video format.

[1888] "Comprehension check" refers to quizzes and questions to check the level of understanding of the learning content after the lecture.

[1889] "Quiz generation" is the process of automatically creating quizzes and questions based on learning content using a generative AI model.

[1890] The "check result" is the result data of the comprehension check sent from the user terminal, which is analyzed by the server and reflected in the next learning content.

[1891] "Additional lectures" are educational content that is provided in addition to the parts of the lecture that are not fully understood based on the results of the comprehension check.

[1892] "Follow-up" refers to ongoing support and advice provided to users based on their learning history and level of understanding.

[1893] A "generative AI model" is an artificial intelligence model that uses machine learning technology to automatically generate quizzes and questions from text data.

[1894] "Progress" is data that indicates the progress of the user's learning, and is managed by the server.

[1895] A "learning unit" is a unit of study that is classified into a specific theme or topic within a learning curriculum.

[1896] This invention is a support system for parents and children to work together to effectively advance learning. This system automatically generates a learning curriculum based on textbook data, automatically generates quizzes to check comprehension using a generative AI model, and manages and supports learning progress through lectures and comprehension checks.

[1897] System configuration

[1898] 1. Enter textbook data

[1899] The user uploads a PDF file of the textbook to the system, and the device sends the file to the server.

[1900] 2. Analysis of textbook data

[1901] The server analyzes the received textbook PDF file, extracts text data, and uses natural language processing (NLP) technology to categorize the textbook content by chapter, section, and topic.

[1902] 3. Generating learning curriculum

[1903] The server automatically generates a learning curriculum based on the analyzed data. Specifically, it determines learning units such as "fraction calculations" and "solving equations," and generates text lectures, videos, and quizzes for each unit.

[1904] 4. Start your study session

[1905] Users (parents and children) log in to the system and take the initial comprehension test. The device sends the test results to the server, which analyzes the results and sets the appropriate difficulty level and starting point.

[1906] 5. Providing lectures

[1907] The server sequentially distributes lectures created for each learning unit, and the devices display the lectures in text or video format, allowing parents and children to study together.

[1908] 6. Generating and providing quizzes to check comprehension using generative AI models

[1909] The server automatically generates quizzes based on the learning content using a generative AI model. The device provides the generated quizzes to the user, who then answers the questions.

[1910] 7. Checking understanding and providing feedback

[1911] The user (child) answers the quiz and sends the results from the device to the server. The server analyzes the quiz results and provides additional lectures on areas where the child did not fully understand.

[1912] 8. Follow-up and continuous learning

[1913] The server records the user's learning history and level of understanding and reflects this in the next lesson.The server also analyzes the user's learning progress and suggests the next appropriate learning unit for the user.

[1914] Hardware and software:

[1915] This system runs on standard PCs and smart devices (smartphones, tablets). The software and libraries used are as follows:

[1916] PyMuPDF (for text extraction)

[1917] NLTK (for text tokenization)

[1918] Transformers (for model inference, especially Facebook's BART model)

[1919] Examples:

[1920] If a user uploads a "sixth grade math textbook," the learning process will proceed as follows:

[1921] 1. A user uploads a "6th grade mathematics textbook PDF."

[1922] 2. The server analyzes the PDF and automatically generates the units "Fundamentals of Arithmetic," "Understanding Fractions," and "Fundamentals of Equations."

[1923] 3. Users (parents and children) take the initial comprehension test and send the results to the server.

[1924] 4. Based on the results, the server recommends starting with the first unit, "Fundamentals of Mathematics."

[1925] 5. The device displays a video lecture on "Mathematics Basics" that parents and children watch together.

[1926] 6. The server automatically generates a quiz using a generative AI model to check comprehension after the lecture and distributes it to the device.

[1927] 7. The user (child) answers the quiz and sends the results to the server.

[1928] 8. The server analyzes the quiz results and provides additional lectures on areas where understanding is insufficient.

[1929] 9. The server will record progress and suggest the next unit, "Understanding Fractions."

[1930] Example prompt sentence:

[1931] Generated curriculum text:

[1932] "Understanding fractions involves the following steps: First, understand the basic concept of fractions. Then practice adding and subtracting fractions. Finally, learn how to multiply and divide fractions."

[1933] Quiz generation prompt:

[1934] "Based on this text, please formulate the following questions:"

[1935] The question that is output is:

[1936] 1. What is the basic concept of fractions?

[1937] 2. Explain adding and subtracting fractions.

[1938] 3. How do you multiply and divide fractions?

[1939] This allows parents and children to study together effectively.

[1940] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1941] Step 1:

[1942] The user uploads a PDF file of a textbook to the terminal. The input is the PDF file of the textbook, and the output is the textbook PDF data sent to the server.

[1943] Step 2:

[1944] The server analyzes the received textbook PDF file and extracts the text data. Specifically, it uses the PyMuPDF library to extract text from the PDF. In this process, the input is the PDF file and the output is the text data.

[1945] Step 3:

[1946] The server uses natural language processing techniques to classify the extracted text data into chapters, sections, and topics. It uses the NLTK library to tokenize sentences and identify each section. The input of this process is the extracted text data, and the output is the classified text data.

[1947] Step 4:

[1948] The server automatically generates a learning curriculum based on the analyzed data. Specifically, it generates learning units such as "fraction calculation" and "solving equations" from the analyzed data. The input for this process is classified text data, and the output is a learning curriculum.

[1949] Step 5:

[1950] Users (parents and children) log in to the system and take the initial comprehension test. The terminal sends the test results to the server. The input is the comprehension test questions and their answers, and the output is the answer data sent to the server.

[1951] Step 6:

[1952] The server analyzes the results of the comprehension test and sets the appropriate difficulty level and starting point. The analysis uses the received answer data to determine the initial learning unit. The input of this process is the answer data, and the output is the initial learning unit and its difficulty setting.

[1953] Step 7:

[1954] The server sequentially distributes lectures created for each learning unit. The terminal displays them to the user in text or video format. The content of the lecture is based on the learning curriculum determined in the previous step. The input of this process is the learning curriculum, and the output is the lecture content to be distributed.

[1955] Step 8:

[1956] The server uses a generative AI model to automatically generate quizzes based on the learning content. The model uses the BART model from the Transformers library and prompts to generate quizzes. The inputs to this process are the learning content and prompts, and the output is an automatically generated quiz.

[1957] Step 9:

[1958] The user (child) answers the generated quiz and sends it from the terminal to the server. The input is the answer to the quiz, and the output is the answer data sent to the server.

[1959] Step 10:

[1960] The server analyzes the quiz results and provides additional lectures for areas where the student did not fully understand the quiz. The input to this process is the answer data, and the output is additional lectures as needed.

[1961] Step 11:

[1962] The server records the user's learning history and level of understanding and suggests the next learning unit. Based on the recorded data, the server analyzes the learning history and level of understanding and provides the next learning curriculum. The input for this process is the learning history and quiz results, and the output is a suggestion for the next learning unit.

[1963] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1964] This invention is an educational support system that enables parents and children to work together to effectively advance learning, and it also includes an emotion engine that recognizes the user's emotions and customizes the learning experience. This system analyzes textbook data, automatically generates learning curricula, conducts lectures and comprehension checks, and records the user's learning history and emotional state to reflect in the next lesson.

[1965] System configuration

[1966] Textbook data entry

[1967] Users upload textbook PDF files to the system. The device sends the files to the server, which then analyzes the received textbook PDF files to extract text data. Natural language processing (NLP) techniques are used to categorize the textbook content by chapter, section, and topic.

[1968] Learning curriculum generation

[1969] The server generates learning units from the analyzed textbook data. These units include "fraction calculations" and "solving equations." Each unit is associated with a text lecture, video content, and quizzes.

[1970] Start a study session

[1971] Users (parents and children) log in to the system and take the initial comprehension test. The device sends the test results to the server, which analyzes the results and sets the appropriate difficulty level and starting point.

[1972] Lecture provision

[1973] The server delivers lectures for the designated learning unit to the device, which displays the lectures in text or video format, allowing parents and children to study together.

[1974] Check your understanding

[1975] As the learning progresses, the server generates quizzes and questions to check comprehension and distributes them to the device. The user (child) answers the questions, and the device sends the results to the server. The server analyzes the results and provides additional lectures or follow-ups for areas where understanding is insufficient.

[1976] Follow-up and continuous learning

[1977] The server records the user's learning history and level of understanding and reflects this in the next lesson. The server also analyzes the user's learning progress and suggests the appropriate next learning unit. The device displays the lecture for the next learning unit and notifies the user.

[1978] User Emotion Recognition

[1979] The server contains an emotion engine and analyzes emotions from the user's facial expressions and voice. The device's built-in camera and microphone are used to capture users' (parents and children's) emotional data in real time and send it to the server. Based on the emotion analysis results, the server adjusts the system's behavior as follows:

[1980] If the user feels stressed while learning, the server will adjust the difficulty level and provide additional support to promote understanding.

[1981] If the user shows excitement or interest, progress the learning content to maintain that state.

[1982] Motivational feedback

[1983] Based on the analysis results of the emotion engine, the server generates encouragement and feedback that matches the user's emotional state. For example, if it determines that the user is tired, it will suggest taking a break, and if they are concentrating, it will display a message encouraging them to continue. This makes the learning experience for parents and children smoother and more comfortable.

[1984] Specific examples

[1985] For example, when a user is learning mathematics with his / her child, the user goes through the following process.

[1986] 1. The user uploads a PDF of a mathematics textbook, and the server analyzes the data and generates a learning curriculum.

[1987] 2. Users (parents and children) log in, take the initial test, and send the results to the server.

[1988] 3. Based on the analysis results, the server sets the first learning unit, "Fraction Basics," and delivers the lecture to the device.

[1989] 4. The device displays the lecture, and parents and children can learn together.

[1990] 5. The server sends a quiz to check comprehension, and the child sends the answers to the server.

[1991] 6. The server provides additional lectures based on comprehension analysis and adjusts the difficulty level as needed.

[1992] 7. The server records the learning history and level of understanding and suggests the next learning unit.

[1993] 8. The device will display the lecture for the next learning unit and you can continue learning.

[1994] 9. The server analyzes the user's emotions through an emotion engine and adjusts the learning content and lecture method as needed.

[1995] 10. Based on the results of the emotion analysis, the server generates encouraging and motivating feedback and delivers it to the device.

[1996] In this way, it is possible to effectively support parents and children in their learning and improve learning efficiency by customizing the learning experience for each individual. This system can reduce parental anxiety, which has traditionally been a challenge, and improve children's learning outcomes.

[1997] The processing flow will be explained below.

[1998] Specific processing flow of the program

[1999] Step 1:

[2000] Uploading textbook data

[2001] Users select the textbook PDF file and drag and drop it into the system's upload form.

[2002] The terminal sends the uploaded PDF file to the server.

[2003] Step 2:

[2004] Textbook data analysis

[2005] The server analyzes the received textbook PDF file and extracts the text data.

[2006] The server uses natural language processing (NLP) techniques to categorize the textbook content by chapter, section, and topic.

[2007] Step 3:

[2008] Learning curriculum generation

[2009] Based on the analyzed data, the server generates learning units for each chapter and section.

[2010] The server associates each unit with the appropriate text lectures, video content, and quizzes.

[2011] Step 4:

[2012] User Login

[2013] Users (parents and children) log into the system.

[2014] The terminal sends the user's login information to the server for authentication.

[2015] Step 5:

[2016] First comprehension test

[2017] The server provides comprehension tests to parents and children as a preliminary step in learning.

[2018] Users (parents and children) take the test and send the results from their device to the server.

[2019] The server analyzes the received results and evaluates the user's current level of understanding.

[2020] Step 6:

[2021] Setting appropriate learning units

[2022] Based on the results of the comprehension test, the server sets the initial learning unit according to the user's level.

[2023] The server delivers the lecture for the first learning unit to the terminal.

[2024] Step 7:

[2025] Lecture provision

[2026] The terminal displays the lecture of the learning unit received from the server (e.g., text, video).

[2027] Users (parents and children) watch lectures together and progress through their learning.

[2028] Step 8:

[2029] Comprehension check

[2030] After the lecture ends, the server generates quizzes and questions to check comprehension and distributes them to the device.

[2031] The user (child) answers questions and quizzes and sends the results from the terminal to the server.

[2032] Step 9:

[2033] Analysis of check results and feedback

[2034] The server analyzes the answers to the quizzes and questions and evaluates the user's level of understanding.

[2035] The server generates and delivers additional lectures and follow-ups for areas where understanding is deemed insufficient.

[2036] Step 10:

[2037] Suggestions for the next learning unit

[2038] The server records the user's progress and level of understanding and suggests the next unit to study.

[2039] The device will display the lecture for the next learning unit and notify the user.

[2040] Step 11:

[2041] Using the Emotion Engine

[2042] The device uses its built-in camera and microphone to collect the user's facial expressions and voice in real time.

[2043] The terminal transmits the collected data to the server.

[2044] The server uses an emotion engine to analyze the received data and assess the user's emotional state.

[2045] Step 12:

[2046] Adjustment based on emotional state

[2047] Based on the analysis results of the emotion engine, the server adjusts the difficulty of the task if the user is feeling stressed and provides additional support to promote understanding.

[2048] If the user shows excitement or interest, the server will progress the learning content to maintain that emotion.

[2049] Step 13:

[2050] Motivational feedback

[2051] Based on the analysis results of the emotion engine, the server generates encouragement and feedback that is in line with the user's emotional state.

[2052] The device displays generated feedback messages, such as suggesting a break if you're tired, or encouraging you to keep going if you're concentrating.

[2053] In this way, the system of the present invention responds not only to the learning progress but also to the emotional state of the user, making it possible to provide an effective learning experience.

[2054] Example 2

[2055] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2056] While conventional learning support systems can measure a user's learning situation and level of understanding, they have the problem of being unable to customize the system to take into account the user's emotional state. This reduces learning efficiency and makes it difficult to maintain user motivation. Furthermore, when parents and children study together, the lack of appropriate support increases parental anxiety and makes it difficult to improve children's learning outcomes.

[2057] The specific processing by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes: means for receiving and analyzing textbook data from a user terminal that uploads the textbook data; means for automatically generating a learning curriculum based on the analyzed textbook data; means for providing easy-to-understand lectures based on the generated learning curriculum; means for generating and providing quizzes and questions to check the level of understanding related to the lecture; means for receiving and analyzing the results of the comprehension check from the user terminal; means for providing additional lectures and follow-ups based on the check results; means for managing the overall progress of the system and suggesting appropriate learning units; and means for analyzing the emotional state of the user during learning in real time and adjusting the system operation based on the results. This provides an optimal learning experience that takes into account not only the user's learning situation but also their emotional state, thereby improving learning efficiency and motivation.

[2058] "Textbook data" refers to digital document data containing content used for educational purposes.

[2059] A "user terminal" is an electronic device used for learning activities such as uploading textbook data, viewing lectures, and answering quizzes.

[2060] "Means of analysis" refers to the technology and equipment used to analyze textbook data, extract the necessary information, and classify it.

[2061] A "learning curriculum" is a set of learning content and activity plans that are established to help users effectively advance their learning.

[2062] "Means for automatic generation" refers to technology or devices for automatically creating a learning curriculum from analyzed textbook data.

[2063] A "lecture" is an explanation or instruction of the learning content provided based on the learning curriculum.

[2064] "Quizzes and questions to check comprehension" are questions provided to assess the user's level of comprehension of the learning content.

[2065] The "means for providing additional lectures or follow-up based on the check results" refers to a technology or device that analyzes the results of the check on the user's comprehension and provides further guidance or supplementary explanations as necessary.

[2066] "Means for managing progress and suggesting appropriate learning units" refers to technologies and devices for tracking a user's learning progress and suggesting what content to learn next.

[2067] "Means for analyzing emotional states in real time" refers to technology or devices for detecting and analyzing emotions in real time from a user's facial expressions, voice, etc.

[2068] "Means for adjusting the system's operation" refers to technology or devices for appropriately changing the learning content or lecture method based on the results of emotion analysis.

[2069] The present invention relates to an educational support system for parents and children to effectively collaborate in learning, and in particular, to a system including an emotion engine that recognizes the user's emotions and customizes the learning experience. This system is realized through the interaction between a server, a terminal, and a user.

[2070] Hardware and software used

[2071] The hardware in this system is as follows:

[2072] Device: A personal computer (PC) or tablet device used at home or in an educational institution.

[2073] Server: A central management device that analyzes and manages data.

[2074] Camera and microphone: Equipped on the device, they capture the user's facial expressions and voice in real time.

[2075] The software used is as follows:

[2076] Natural Language Processing (NLP) techniques: used to analyze textbook data.

[2077] Sentiment Analysis Engine: Used to analyze the user's emotional state.

[2078] OCR (Optical Character Recognition) technology: Used to convert PDF files into text data.

[2079] Various modules: Examples include PDFReceiverService, NLPAnalyzer, TextClassifier, CurriculumGenerator, TestEvaluator, LectureDistributor, LectureViewer, QuizGenerator, AnswerAnalyzer, LearningHistoryRecorder, NextUnitRecommender, EmotionAnalyzer, EmotionCapture, FeedbackGenerator, and FeedbackDisplay.

[2080] System Description

[2081] 1. Enter textbook data

[2082] A user logs into the system and uploads a textbook PDF file from their device. The device sends this file to the server, which uses a PDFReceiverService to receive, store, and prepare the file for processing.

[2083] 2. Textbook data analysis and curriculum generation

[2084] The server uses the OCRProcessor to scan PDF files and extract text data. The extracted text data is then analyzed by the NLPAnalyzer and classified by chapter, section, and topic. The CurriculumGenerator automatically generates a learning curriculum based on this data, associating each unit with text lectures, video content, and quizzes.

[2085] 3. First comprehension test

[2086] Users (parents and children) log in to the system and take the initial comprehension test on their device. The device sends the test results to the server, which analyzes them using the TestEvaluator and sets the appropriate difficulty level and starting point.

[2087] 4. Providing lectures

[2088] The server distributes text and video lectures to devices using LectureDistributor based on the set learning units. The devices display these lectures using LectureViewer, allowing parents and children to study together.

[2089] 5. Check your understanding

[2090] As the learning progresses, the server uses QuizGenerator to generate quizzes and questions to check comprehension and distributes them to the device. The user (child) answers the quiz on the device and sends the results to the server. The server analyzes the answers using AnswerAnalyzer and provides additional lectures or follow-ups for areas where understanding is insufficient.

[2091] 6. Recording learning history and suggesting next learning units

[2092] The server uses LearningHistoryRecorder to record the user's learning history and level of understanding. NextUnitRecommender then uses this information to suggest the next learning unit. The device then displays the lecture for the suggested next learning unit and notifies the user.

[2093] 7. User Emotion Recognition

[2094] The server uses the emotion engine to analyze the user's facial expressions and voice captured by the device's camera and microphone in real time. The data is acquired through EmotionCapture and analyzed by EmotionAnalyzer to adjust the system's behavior.

[2095] 8. Feedback and Motivation

[2096] The server uses the FeedbackGenerator to provide encouragement and feedback to the user based on the sentiment analysis results, and the device uses the FeedbackDisplay to display this feedback and improve the quality of the learning experience.

[2097] Specific examples

[2098] For example, when a user is learning mathematics with a child, the following process is carried out.

[2099] 1. The user uploads a PDF of a mathematics textbook, and the server analyzes the data and generates a learning curriculum.

[2100] 2. Users (parents and children) log in, take the initial test, and send the results to the server.

[2101] 3. Based on the analysis results, the server sets the first learning unit, "Fraction Basics," and delivers the lecture to the device.

[2102] 4. The device displays the lecture, and parents and children can learn together.

[2103] 5. The server sends a quiz to check comprehension, and the child sends the answers to the server.

[2104] 6. The server provides additional lectures based on comprehension analysis and adjusts the difficulty level as needed.

[2105] 7. The server records the learning history and level of understanding and suggests the next learning unit.

[2106] 8. The device will display the lecture for the next learning unit and you can continue learning.

[2107] 9. The server analyzes the user's emotions through an emotion engine and adjusts the learning content and lecture method as needed.

[2108] 10. Based on the results of the emotion analysis, the server generates encouraging and motivating feedback and delivers it to the device.

[2109] Prompt Sentence Examples

[2110] "Generate learning units from the following PDF textbooks and customize lecture content based on comprehension checks and sentiment analysis."

[2111] As a result, parents and children can study effectively, and the learning effect and motivation of users can be improved.

[2112] The flow of the identification process in the second embodiment will be described with reference to FIG.

[2113] Step 1: Enter textbook data

[2114] Input: Textbook PDF file

[2115] Specific operation: The user clicks the "Upload textbook" button on the terminal, selects and uploads a textbook PDF file from the local file system, and the terminal uses the FileUploader module to send the selected PDF file to the server.

[2116] Output: PDF file saved on the server

[2117] Data processing: The server starts the PDFReceiverService and saves the received file in a temporary folder.

[2118] Step 2: Analyzing textbook data

[2119] Input: PDF file stored on the server

[2120] How it works: The server uses OCRProcessor to scan PDF files and extract text data, which is then analyzed by NLPAnalyzer and classified into chapters, sections, and topics.

[2121] Output: Analyzed textbook data

[2122] Data processing: The server scans the PDF data as an image and converts it into text data. The converted text data is then analyzed using natural language processing technology to classify each piece of content.

[2123] Step 3: Generate a learning curriculum

[2124] Input: Parsed textbook data

[2125] Specific operation: The server uses CurriculumGenerator to automatically generate learning units from the parsed data, and associates text lectures, video content, and quizzes with the generated learning units.

[2126] Output: Generated learning curriculum

[2127] Data processing: Based on each extracted topic, the server generates appropriate learning units and adds related educational resources (text, videos, quizzes).

[2128] Step 4: First assessment

[2129] Input: User (parent and child) login information

[2130] Specific operation: A user logs in to the system and takes the initial comprehension test on their device. The device displays the test questions using QuizPresenter and the user enters their answers. The answer data is sent to the server via AnswerCollector.

[2131] Output: Test result data

[2132] Data processing: The server uses TestEvaluator to analyze the submitted answers and set appropriate difficulty levels and starting points.

[2133] ...

Claims

1. means for receiving and analyzing textbook data from a user terminal that uploads the textbook data; A means to automatically generate learning curricula based on the analyzed textbook data, and A means for providing easy-to-understand lectures based on the generated learning curriculum; a means for generating and providing quizzes and questions to test comprehension related to the lecture; A means for receiving and analyzing the results of the comprehension check from the user terminal; A means to provide additional lectures or follow-ups based on the results of the checks; A means for managing the overall progress of the system and proposing appropriate learning units; A system including:

2. 10. The system of claim 1, further comprising means for providing a user interface for parent and child collaborative learning.

3. 2. The system according to claim 1, further comprising means for recording the user's learning history and level of understanding and reflecting this in the next learning session.

Images