System

Abstract

A system is provided.SOLUTION: A system comprising: means for inputting a teaching material; means for extracting features using a generative AI model for analyzing the input teaching material; means for automatically generating a quiz based on the extracted features; means for providing a user interface for checking and editing the generated quiz; means for distributing the checked and edited quiz to a target learner; means for collecting and analyzing quiz answers of the target learner; and means for generating a comprehension report based on the analyzed result.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Currently, creating quizzes and exam questions to assess learners' understanding in learning management systems and business procedure training is a time-consuming process. Furthermore, ensuring the quality and appropriateness of the questions requires specialized knowledge and experience, which hinders efficient learning. To address these challenges, a system is needed that can automatically generate high-quality quizzes and effectively assess learners' understanding. [Means for solving the problem]

[0005] To solve the above problems, the present invention provides the following means. First, a means for a user to input learning materials is provided, followed by a means for extracting features using a generative AI model that analyzes the input learning materials. Furthermore, a means for automatically generating quizzes based on the extracted features is provided, and a means for providing a user interface for reviewing and editing the generated quizzes is provided. Finally, a system is provided that includes a means for distributing the reviewed and edited quizzes to target learners, collecting and analyzing the quiz answers of the target learners, and a means for generating a comprehension report based on the analysis results. This makes it possible to efficiently and effectively measure learners' comprehension levels and significantly reduce the effort required for creating questions.

[0006] "Instructional Materials" means digital or physical materials containing learning content or information presented to a learner.

[0007] A "generative AI model" is an artificial intelligence algorithm used to analyze input data and extract useful information from it.

[0008] "Features" refer to important points, keywords, and concepts within the teaching materials, and are the basic information for creating quizzes.

[0009] A "quiz" is a collection of questions or questions used to measure a learner's comprehension.

[0010] "User interface" is a general term for screens and devices that allow users to interact with a system, and is a means of facilitating the input and output of information.

[0011] "Delivery" is the act of sending the generated quiz or information to the target learners and making it accessible.

[0012] "Student" refers to an individual or group who receives quizzes and learning materials through the system and engages in learning activities.

[0013] An "answer" is a response or solution submitted by a learner to a quiz.

[0014] "Analysis" is the process of examining collected data to derive meaningful results.

[0015] The "Comprehension Report" is a report summarizing the learner's level of understanding and progress based on their quiz answers. [Brief explanation of the drawings]

[0016] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0017] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0018] First, the terms used in the following description will be explained.

[0019] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0020] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0021] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0022] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0023] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0024] [First embodiment]

[0025] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0026] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0027] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0028] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0029] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0030] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0031] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0032] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0033] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0034] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0035] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0036] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0037] This invention relates to a system that automatically generates quizzes from learning materials and evaluates learners' understanding based on the quizzes. This system efficiently performs a series of processes: inputting learning materials, analyzing them, generating quizzes, and distributing them to learners. The following is a detailed description of this system.

[0038] Program processing explanation

[0039] 1. Uploading teaching materials

[0040] The user uses the terminal to upload learning material files to the server, for example, PDF format learning materials, and sends them to the system via a designated upload button.

[0041] The server saves the received file and displays a notification to the user that the upload is complete.

[0042] 2. Teaching material analysis

[0043] The server passes the saved teaching material files to the generative AI model.

[0044] The generative AI model analyzes the content of the teaching materials and extracts key points and keywords, such as "What is a variable?" and "What are the types of loops?"

[0045] 3. Quiz Generation

[0046] Based on the analysis results, the server issues commands to the generative AI model to automatically generate quizzes, such as multiple choice questions, true / false questions, and essay questions based on key points.

[0047] The created quiz is stored on the server and displayed to the user for prior review.

[0048] 4. Quiz confirmation

[0049] The user can use the device to check the generated quiz content and edit it as needed, for example, by modifying the question text or adding options.

[0050] Once editing is complete, the user clicks the "Confirm Quiz" button to send the finalized quiz to the server.

[0051] 5. Quiz distribution

[0052] The server checks the list of learners to distribute the confirmed quiz to the target learners, and based on this list, sends each learner an email containing an access link to the quiz.

[0053] Learners click on a link from their device to access and answer the quiz.

[0054] 6. Answer collection and analysis

[0055] Once the learner has completed the quiz, the answers are sent to the server and stored.

[0056] The server uses an automated scoring system to score each answer and tally the scores.

[0057] Based on the scoring results, the generative AI model analyzes each learner's level of understanding.

[0058] 7. Report Generation

[0059] The server generates a comprehension report from the analysis results and provides it to the user.

[0060] The user checks the report and provides feedback to the learner if necessary.

[0061] Specific examples

[0062] For example, if a learning material for learning "basic programming concepts" is uploaded, a generative AI model might analyze the material and identify key points such as:

[0063] Defining variables

[0064] Loop Types

[0065] Conditional Branching

[0066] Based on this, the following quiz is generated:

[0067] Multiple choice question: "What is the definition of a variable?"

[0068] A. Memory area

[0069] B. Temporary storage of data

[0070] C. Graphical User Interface

[0071] True or False: "The for loop is used to repeat a fixed number of times. True or False?"

[0072] Short answer question: "Explain the basic usage of a while loop."

[0073] By distributing these quizzes to learners, collecting and analyzing their answers, and generating comprehension reports, efficient learning management becomes possible.

[0074] This concludes the detailed description of one embodiment of the present invention. This system makes it easy to automatically generate quizzes and measure comprehension levels, and is expected to improve the learning efficiency of learners.

[0075] The processing flow will be explained below.

[0076] Step 1:

[0077] The user logs in to the learning management system using a device. They enter their username and password, and if authentication is successful, they are able to access the administration screen.

[0078] Step 2:

[0079] The user clicks the "Upload teaching materials" button on the administration screen, which displays a file selection dialog.

[0080] Step 3:

[0081] The user selects the teaching material file (e.g., a PDF file) to upload and clicks the "Open" button.

[0082] Step 4:

[0083] The device will send the selected teaching material file to the server. A message will appear indicating that the file has been successfully uploaded.

[0084] Step 5:

[0085] The server saves the uploaded learning material files in the specified storage area.

[0086] Step 6:

[0087] The server passes the saved teaching material files to the generative AI model.

[0088] Step 7:

[0089] The generative AI model analyzes the content of the learning material and extracts key points and keywords, such as variables, loops, and conditional branching.

[0090] Step 8:

[0091] The generative AI model returns the analysis results to the server, which include key points and keywords from the learning material.

[0092] Step 9:

[0093] The server generates quiz templates based on the analysis results, which can include multiple choice questions, true / false questions, and essay questions.

[0094] Step 10:

[0095] A generative AI model automatically generates specific quizzes based on templates, including multiple-choice questions such as "What is the definition of a variable?"

[0096] Step 11:

[0097] The server temporarily stores the generated quiz and provides an interface for displaying it to the user.

[0098] Step 12:

[0099] The user can check the generated quiz from the management screen, specifically, the question text, options, correct answers, etc.

[0100] Step 13:

[0101] Users can edit the content of each quiz, for example, by correcting incorrect answers or adding additional options.

[0102] Step 14:

[0103] When the user has finished editing, they click the "Confirm Quiz" button, and the confirmed quiz is sent to the server.

[0104] Step 15:

[0105] The server acquires the learner list and generates an email for delivering the confirmed quiz to each learner.

[0106] Step 16:

[0107] The server sends the learner an email containing an access link to the quiz.

[0108] Step 17:

[0109] Learners receive the email on their device and click on the link to access the quiz screen.

[0110] Step 18:

[0111] The learner answers the quiz and submits the answers, which are automatically sent to the server.

[0112] Step 19:

[0113] The server receives and stores the learner's answers.

[0114] Step 20:

[0115] The server runs an automated scoring system to score each answer, with the scoring criteria set in advance.

[0116] Step 21:

[0117] The server compiles the scoring results and analyzes the level of understanding, such as calculating each student's score and percentage of correct answers.

[0118] Step 22:

[0119] The server generates a comprehension report based on the analysis results, which includes each learner's grades and learning progress.

[0120] Step 23:

[0121] The server provides the generated comprehension report to the user, who can view the report on the management screen.

[0122] Step 24:

[0123] The user can review the comprehension report and provide feedback to the learner as needed, for example by providing explanations for specific questions or additional practice questions.

[0124] Example 1

[0125] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0126] Conventional learning systems have struggled to efficiently generate quizzes from learning materials and accurately assess learners' comprehension. Furthermore, the processes of quiz generation, confirmation, editing, distribution, answer collection, and analysis all require manual work, which is time-consuming and labor-intensive. Furthermore, the quality of the generated quizzes and the reliability of the assessment of learners' comprehension have also been issues.

[0127] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0128] In this invention, the server includes: means for a user to input learning materials; means for extracting key points using a generative AI model that analyzes the input learning materials; means for automatically generating quizzes based on the extracted key points; means for providing a user interface for reviewing and editing the generated quizzes; means for delivering the reviewed and edited quizzes to target learners; means for collecting and scoring quiz answers from target learners; and means for analyzing the scoring results and generating a comprehension report. This makes it possible to efficiently generate quizzes from learning materials and accurately evaluate learners' comprehension.

[0129] "User" refers to the entity that operates the system, uploads learning materials, and checks and edits quizzes.

[0130] "Learning materials" refers to files or data containing information or content that a learner uses to study.

[0131] A "generative AI model" is an artificial intelligence model that analyzes input learning materials, extracts key points, and generates quizzes.

[0132] "Key points" refer to the particularly important concepts and information in the learning material that learners should understand.

[0133] A "quiz" is a problem or question posed to assess a learner's level of understanding.

[0134] A "user interface" refers to the screens and interaction means that allow a user to interact with and operate a system.

[0135] "Delivery" refers to the process of sending the generated quiz or information to the intended recipient.

[0136] A "learner" is an individual who answers quizzes and learns the learning material.

[0137] "Scoring" refers to the process of evaluating and scoring quizzes completed by learners.

[0138] A "comprehension report" is a report that shows the learner's level of understanding based on the scoring results.

[0139] This invention relates to a system that automatically generates quizzes from learning materials and evaluates the learner's level of understanding based on the quizzes. This system efficiently performs a series of processes: inputting learning materials, analyzing them, generating quizzes, and distributing them to learners.

[0140] The system is implemented using the following hardware and software.

[0141] Hardware: Server, user device, learner device

[0142] Software: Generative AI models (e.g., GPT-4), automated scoring systems, email distribution systems

[0143] Program processing explanation

[0144] Uploading teaching materials

[0145] The user uploads the learning material file to the server using his / her own terminal. For example, the user sends the learning material file in PDF format to the system via a designated upload button.

[0146] The server saves the received file and displays a notification to the user that the upload is complete.

[0147] Teaching material analysis

[0148] The server passes the saved teaching material file to the generative AI model and begins analysis.

[0149] The generative AI model analyzes the content of the teaching materials and extracts key points and keywords, such as "What is a variable?" and "What are the types of loops?"

[0150] Quiz Generation

[0151] The server issues commands to the generative AI model based on the analysis results to automatically generate quizzes, such as multiple choice questions, true / false questions, and essay questions based on key points.

[0152] The created quiz is stored on the server and displayed to the user for prior review.

[0153] Quiz Check

[0154] The user can use the device to check the generated quiz content and edit it as needed, for example by modifying the question text or adding options.

[0155] Once editing is complete, the user clicks the "Confirm Quiz" button to send the finalized quiz to the server.

[0156] Quiz broadcast

[0157] The server checks the list of learners to distribute the confirmed quiz to the target learners, and based on this list, sends each learner an email containing an access link to the quiz.

[0158] Learners click on a link from their device to access the quiz and answer the questions.

[0159] Answer collection and analysis

[0160] Once the learner has completed the quiz, the answers are sent to the server and stored.

[0161] The server uses an automated scoring system to score each answer and tally the scores.

[0162] Based on the scoring results, the generative AI model analyzes each learner's level of understanding.

[0163] Report Generation

[0164] The server generates a comprehension report from the analysis results and provides it to the user.

[0165] The user checks the report and provides feedback to the learner if necessary.

[0166] Specific examples

[0167] For example, if a learning material for learning "basic programming concepts" is uploaded, a generative AI model might analyze the material and identify key points such as:

[0168] Defining variables

[0169] Loop Types

[0170] Conditional Branching

[0171] Based on this, the following quiz is generated:

[0172] Multiple choice question: "What is the definition of a variable?"

[0173] A. Memory area

[0174] B. Temporary storage of data

[0175] C. Graphical User Interface

[0176] True or False: "The for loop is used to repeat a fixed number of times. True or False?"

[0177] Short answer question: "Explain the basic usage of a while loop."

[0178] Example prompts to input to the generative AI model

[0179] "Please analyze the following PDF file and extract the main points and keywords."

[0180] "Based on the analysis results below, please generate a quiz in the form of multiple choice, true / false, or written questions."

[0181] "Collect answers corresponding to the next quiz and analyze learners' understanding."

[0182] This completes the detailed description of one embodiment of the present invention. This system makes it easy to automatically generate quizzes and measure comprehension levels, and is expected to improve the learning efficiency of learners.

[0183] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0184] Step 1: Upload your materials

[0185] Users use their own devices to select learning material files, such as PDF files, and click the designated upload button to send them to the server.

[0186] Input: Learning material file selected by the user

[0187] Output: Teaching material files saved on the server

[0188] Specific operation: The user accesses the "Upload learning material" screen and clicks the "Select file" button to select the learning material file. When the user clicks the "Upload" button, the file is sent to the server, and the server displays the message "Learning material upload completed" on the user's device.

[0189] Step 2: Analysis of teaching materials

[0190] The server passes the saved teaching material file to the generative AI model and begins analysis.

[0191] Input: Teaching material file saved on the server

[0192] Output: Key points and keywords extracted by the generative AI model

[0193] Specific operation: The server creates a prompt to input the learning material file into the generative AI model (e.g., "Please extract the key points about the programming concept."). The generative AI model analyzes the learning material file and extracts important information such as "variable definitions," "loop types," and "conditional branching."

[0194] Step 3: Quiz Generation

[0195] Based on the analysis results, the server issues commands to the generative AI model to automatically generate quizzes.

[0196] Input: Key points and keywords extracted by the generative AI model

[0197] Output: Generated quiz (multiple choice, true / false, short answer)

[0198] Specific operation: The server instructs the generative AI model to "create a quiz based on the following points." The generative AI model creates quizzes such as "What is the definition of a variable?" and "A for loop is used to repeat a fixed number of times. Is it true or false?" and saves them on the server.

[0199] Step 4: Quiz Review

[0200] The user uses the terminal to check the generated quiz content and edit it as necessary.

[0201] Input: Generated quiz stored on the server

[0202] Output: Quiz confirmed and edited by the user

[0203] Specific operation: The user accesses the "Quiz Confirmation" screen, reviews the generated quiz, and makes edits such as "revise the question text" and "add options." The user clicks the "Confirm Quiz" button, and the confirmed quiz is sent to the server.

[0204] Step 5: Stream the quiz

[0205] The server checks the learner list to deliver the determined quiz to the target learners.

[0206] Input: Quiz confirmed and edited by user, learner list

[0207] Output: An email containing the quiz access link delivered to the learner.

[0208] Specific operation: The server starts the "quiz distribution" process and checks the learner list. The server automatically sends an email containing an access link to the quiz to each learner. The learner clicks the link in the email and accesses the quiz.

[0209] Step 6: Collect and analyze answers

[0210] The learner answers the quiz and sends the answers to the server.

[0211] The server uses an automatic scoring system to score the answers and tally the scores.

[0212] Input: Quiz answered by learner, scoring system

[0213] Output: Scoring results, comprehension analysis results

[0214] Specific operation: The learner answers the quiz and clicks the "Submit" button. The server passes the received answers to the automatic scoring system, which generates a scoring result. The generative AI model analyzes the scoring result and analyzes the learner's level of understanding.

[0215] Step 7: Generate reports

[0216] The server generates a report from the comprehension analysis results and provides it to the user.

[0217] Input: Comprehension analysis results

[0218] Output: User-viewable comprehension report

[0219] Specific operation: The server executes the "report generation" process and creates a report based on the analysis results. The user checks the "comprehension report" and selects a means to share it with the learner (e.g., send it by email).

[0220] By using the above processing steps, the present invention makes it possible to efficiently generate quizzes from learning materials and accurately evaluate the learner's level of understanding.

[0221] (Application example 1)

[0222] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0223] Conventional systems that create quizzes based on teaching materials and assess learners' comprehension have difficulty meeting the diverse needs of learners and on-site training that requires specific expertise. It has also been difficult to generate and evaluate quizzes for efficient training of new employees and skill development. In particular, there has been a demand for quiz creation and comprehension assessment that effectively utilizes procedures and manuals in training engineers and workers in factories.

[0224] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0225] In this invention, the server includes means for inputting learning materials, means for extracting features using a generative AI model that analyzes the input learning materials, means for automatically generating quizzes based on the extracted features, means for providing a user interface for checking and editing the generated quizzes, means for distributing the checked and edited quizzes to target learners, means for collecting and analyzing the quiz answers of the target learners, means for generating a comprehension report based on the analysis results, means for uploading learning procedures and manuals and generating quizzes to evaluate the comprehension of engineers and workers, and means for use as an application installed on factory robots for training engineers and workers. This makes it possible to efficiently and accurately generate quizzes based on the content of the learning materials and evaluate the comprehension.

[0226] "Teaching materials" are information and materials used for the purpose of learner understanding.

[0227] A "generative AI model" is a model that uses artificial intelligence technology to perform text analysis and information extraction.

[0228] "Features" are important points or keywords extracted from the teaching materials.

[0229] A "quiz" is a question or problem used to assess a learner's understanding.

[0230] A "user interface" is an interface through which a user interacts with a system or application.

[0231] An "engineer" is someone who has specific skills or expertise and works in a factory or on-site.

[0232] "Workers" are workers who perform manual labor or operate machinery in factories or on-site.

[0233] A "procedure" is a document that describes the steps and methods for performing a specific task or operation.

[0234] A "manual" is a document that contains instructions or guidance for a particular operation or procedure.

[0235] "Training" refers to educational activities aimed at improving the skills and knowledge of engineers and workers.

[0236] An "understanding report" is a report summarizing the results of an evaluation of the level of understanding and proficiency of a learner or worker.

[0237] A "factory robot" is a machine or device used to automate work within a factory.

[0238] The present invention relates to a system that automatically generates quizzes from learning materials and evaluates the learner's level of understanding based on the quizzes. This system is installed on factory robots and is effectively used for training engineers and workers.

[0239] The server first receives the learning materials uploaded by the user. These learning materials, such as learning procedures or manuals, are uploaded in PDF format. The PDF file is converted to text using a Python library called pdfplumber. The server then passes this converted text to the generative AI model.

[0240] The generative AI model uses generative AI technology to analyze the teaching materials. This analysis extracts key points and keywords. For example,

[0241] This information includes "What is a variable?", "What type of loop?", etc. In this step, the OpenAI API is used. An example of a prompt sentence output by the generative AI model is as follows:

[0242] Based on the text below, please extract the key points:

[0243] A variable is a symbol that stores specific data. For example, in Python, you can define it as x = 10. There are two types of loops: for loops and while loops. A for loop is used to repeat a fixed number of times, while a while loop repeats as long as a condition is met.

[0244] Based on the analysis results, the server again uses the OpenAI API to automatically generate quizzes. The quizzes generated range from multiple choice questions, true / false questions, and essay questions. For example, the following quiz may be generated:

[0245] Multiple choice question: "What is the definition of a variable?"

[0246] A. Memory area

[0247] B. Temporary storage of data

[0248] C. Graphical User Interface

[0249] True or False: "The for loop is used to repeat a fixed number of times. True or False?"

[0250] Short answer question: "Explain the basic usage of a while loop."

[0251] After the quiz is created, the user can view and edit the content through a dedicated user interface. The quiz is then distributed to the target learners via email or other means, with an access link to the quiz.

[0252] After a learner answers the quiz, their answers are sent to the server and graded by an automatic scoring system. Based on the results, a generative AI model analyzes each learner's level of understanding and generates a comprehension report. This report is provided to the user, who can then provide feedback based on it.

[0253] In one embodiment of the present invention, when used for training engineers and workers in a factory, for example, manuals on how to operate new machinery or safety procedures can be uploaded and analyzed, and quizzes can be generated, allowing for efficient assessment of the level of understanding of specific work tasks and enhancing the effectiveness of training.

[0254] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0255] Step 1:

[0256] The server receives the teaching material files uploaded by the user. Specifically, when the user uploads the teaching material in PDF format, the server saves it and displays a notification to the user that the upload is complete. The input is a PDF file, and the output is PDF data saved on the server.

[0257] Step 2:

[0258] The server uses the pdfplumber library to convert the uploaded PDF file to text data. Specifically, it uses pdfplumber to extract text from each page of the PDF and combine them into a single text data. The input is the saved PDF file, and the output is the extracted text data.

[0259] Step 3:

[0260] The server passes the extracted text data to the generative AI model for analysis. Specifically, it uses the OpenAI API to send the text data to the generative AI model as a prompt sentence, and extracts key points and keywords. The input is the text data, and the output is the analysis results (keywords and points) generated by the AI ​​model.

[0261] Step 4:

[0262] The server then uses the generative AI model again based on the analysis results to automatically generate quizzes. Specifically, the analysis results are used as prompts to generate multiple choice, true / false, and written questions via the OpenAI API. The input is the analysis results, and the output is the generated quiz questions.

[0263] Step 5:

[0264] The server provides the generated quiz questions to the user through a user interface. Specifically, the server displays the generated quiz content on the user interface for the user to review and edit. The input is the generated quiz questions, and the output is the quiz content displayed to the user.

[0265] Step 6:

[0266] The user can view and edit the quiz content using the provided user interface. Specifically, the user can modify the question text or add options. The input is the displayed quiz content, and the output is the final quiz content that the user has viewed and edited.

[0267] Step 7:

[0268] The server distributes the quiz that the user has confirmed to the target learners. Specifically, based on the learner list, it sends an email containing an access link to the quiz to each learner. The input is the confirmed quiz content and the learner list, and the output is the quiz link sent to the learner.

[0269] Step 8:

[0270] The learner uses their own device to access the quiz from the link they received and answers the questions. Specifically, the learner answers the quiz and sends the results to the server. The input is the learner's answer, and the output is the answer data sent to the server.

[0271] Step 9:

[0272] The server automatically grades the collected answer data and tally up the scores of each learner. Specifically, the server's automatic grading system grades the answers and calculates the scores. The input is the answer data, and the output is the graded results.

[0273] Step 10:

[0274] The server uses a generative AI model based on the scoring results to create a comprehension report and provides it to the user. Specifically, it evaluates each learner's level of comprehension based on the analyzed answers and scores, and compiles the results in a report format. The input is the scoring results, and the output is a comprehension report.

[0275] These are the specific processing steps of this system. By processing and calculating data at each step, it is possible to efficiently generate quizzes and evaluate the learner's level of understanding.

[0276] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0277] The present invention relates to a system that automatically generates quizzes from learning materials, evaluates the learner's level of understanding, and simultaneously recognizes the user's emotional state and reflects it in the system. The following is a detailed description of this system.

[0278] Program processing explanation

[0279] 1. Uploading teaching materials

[0280] The user logs in to the learning management system using a device. They enter their username and password, and if authentication is successful, they are able to access the administration screen.

[0281] When a user clicks the "Upload teaching materials" button on the administration screen, a file selection dialog box is displayed.

[0282] The user selects the teaching material file (e.g., a PDF file) to upload and clicks the "Open" button.

[0283] The device will send the selected teaching material file to the server. A message will appear indicating that the file has been successfully uploaded.

[0284] 2. Teaching material analysis

[0285] The server saves the uploaded learning material files in the specified storage area.

[0286] The server passes the saved teaching material files to the generative AI model.

[0287] The generative AI model analyzes the content of the learning material and extracts key points and keywords, such as variables, loops, and conditional branches.

[0288] 3. User Emotion Recognition

[0289] The device uses its built-in camera and microphone to capture the user's emotional data.

[0290] The server utilizes an emotion engine to analyze the acquired emotion data and determine the user's emotional state (eg, stress, interest, concentration, etc.).

[0291] 4. Quiz Generation

[0292] The server issues commands to the generative AI model based on the characteristics of the learning material and the user's emotional state that have been analyzed in advance, and automatically generates quizzes. Specifically, multiple-choice questions, true / false questions, and essay-style questions are generated based on key points.

[0293] The content and difficulty of the generated quiz are adjusted based on the user's emotional state. For example, if the user is feeling stressed, questions with lower difficulty will be given priority.

[0294] 5. Quiz confirmation

[0295] The server temporarily stores the generated quiz and provides an interface for displaying it to the user.

[0296] The user can check the generated quiz from the management screen, specifically, the question text, options, correct answers, etc.

[0297] Users can edit the content of each quiz, for example, by correcting incorrect answers or adding additional options.

[0298] When the user has finished editing, he clicks the "Confirm Quiz" button and the confirmed quiz is sent to the server.

[0299] 6. Quiz Streaming

[0300] The server acquires the learner list and generates an email for delivering the confirmed quiz to each learner.

[0301] The server sends the learner an email containing an access link to the quiz.

[0302] 7. Answer collection and analysis

[0303] Learners receive the email on their device and click on the link to access the quiz screen.

[0304] The learner answers the quiz and submits the answers, which are automatically sent to the server.

[0305] The server receives and stores the learner's answers.

[0306] The server uses an automated scoring system to score each answer and tally the scores.

[0307] Based on the scoring results, the generative AI model analyzes each learner's level of understanding.

[0308] 8. Report Generation

[0309] The server generates a comprehension report from the analysis results and provides it to the user.

[0310] The user checks the report and provides feedback to the learner if necessary.

[0311] Specific examples

[0312] For example, if a learning material for learning "basic programming concepts" is uploaded, the generative AI model analyzes the material and identifies key points such as "variable definitions," "types of loops," and "conditional branching." Furthermore, if the user's emotion engine determines that their "stress level is high," the difficulty level of the generated quiz will be set low:

[0313] Multiple choice question: "What is the definition of a variable?"

[0314] A. Memory area

[0315] B. Temporary storage of data

[0316] C. Graphical User Interface

[0317] True or False: "The for loop is used to repeat a fixed number of times. True or False?"

[0318] Short answer question: "Explain the basic usage of a while loop."

[0319] By distributing these quizzes to learners, collecting and analyzing their answers, and generating comprehension reports, we can efficiently and effectively manage learning and provide an optimal learning experience according to the user's emotional state.

[0320] This concludes the detailed description of one embodiment of the present invention. This system facilitates the automatic generation of quizzes and the measurement of comprehension, and is expected to provide an optimal learning environment for both users and learners.

[0321] The processing flow will be explained below.

[0322] Step 1:

[0323] The user logs in to the learning management system using a device. They enter their username and password, and if authentication is successful, they are able to access the administration screen.

[0324] Step 2:

[0325] The user clicks the "Upload teaching materials" button on the administration screen, which displays a file selection dialog.

[0326] Step 3:

[0327] The user selects the teaching material file (e.g., a PDF file) to upload and clicks the "Open" button.

[0328] Step 4:

[0329] The device will send the selected teaching material file to the server. A message will appear indicating that the file has been successfully uploaded.

[0330] Step 5:

[0331] The server saves the uploaded learning material files in the specified storage area.

[0332] Step 6:

[0333] The server passes the saved teaching material files to the generative AI model.

[0334] Step 7:

[0335] The generative AI model analyzes the content of the learning material and extracts key points and keywords, such as variables, loops, and conditional branching.

[0336] Step 8:

[0337] The device uses its built-in camera and microphone to capture the user's emotional data.

[0338] Step 9:

[0339] The server utilizes an emotion engine to analyze the acquired emotion data and determine the user's emotional state (eg, stress, interest, concentration, etc.).

[0340] Step 10:

[0341] The server issues commands to the generative AI model based on the characteristics of the learning material and the user's emotional state that have been analyzed in advance, and automatically generates quizzes. Specifically, multiple-choice questions, true / false questions, and essay-style questions are generated based on key points.

[0342] Step 11:

[0343] The content and difficulty of the generated quiz are adjusted based on the user's emotional state. For example, if the user is feeling stressed, questions with lower difficulty will be given priority.

[0344] Step 12:

[0345] The server temporarily stores the generated quiz and provides an interface for displaying it to the user.

[0346] Step 13:

[0347] The user can check the generated quiz from the management screen, specifically, the question text, options, correct answers, etc.

[0348] Step 14:

[0349] Users can edit the content of each quiz, for example, by correcting incorrect answers or adding additional options.

[0350] Step 15:

[0351] When the user has finished editing, they click the "Confirm Quiz" button, and the confirmed quiz is sent to the server.

[0352] Step 16:

[0353] The server acquires the learner list and generates an email for delivering the confirmed quiz to each learner.

[0354] Step 17:

[0355] The server sends the learner an email containing an access link to the quiz.

[0356] Step 18:

[0357] Learners receive the email on their device and click on the link to access the quiz screen.

[0358] Step 19:

[0359] The learner answers the quiz and submits the answers, which are automatically sent to the server.

[0360] Step 20:

[0361] The server receives and stores the learner's answers.

[0362] Step 21:

[0363] The server runs an automated scoring system to score each answer, with the scoring criteria set in advance.

[0364] Step 22:

[0365] The server compiles the scoring results and analyzes the level of understanding, such as calculating each student's score and percentage of correct answers.

[0366] Step 23:

[0367] The server generates a comprehension report based on the analysis results, which includes each learner's grades and learning progress.

[0368] Step 24:

[0369] The server provides the generated comprehension report to the user and makes it viewable on the management screen.

[0370] Step 25:

[0371] The user can review the comprehension report and provide feedback to the learner as needed, for example by providing explanations for specific questions or additional practice questions.

[0372] Example 2

[0373] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0374] While remote learning and online education have become increasingly important in recent years, accurately assessing learners' comprehension and providing an optimal learning experience that takes into account their emotional state remains a challenging task. Traditional systems require manual analysis of learning materials and quiz generation, placing a heavy burden on human resources and making them inefficient. Furthermore, they lack the ability to reflect learners' emotional state in real time and maintain their motivation.

[0375] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for inputting learning materials, means for extracting features using a generation AI model that analyzes the input learning materials, means for automatically generating quizzes based on the extracted features, means for providing a user interface for confirming and editing the generated quizzes, means for delivering the confirmed and edited quizzes to target learners, means for collecting and analyzing quiz answers from target learners, means for evaluating the target learners' understanding levels based on the collected data and generating comprehension reports based on the analysis results, and means for acquiring user emotional data from the terminal, analyzing the emotional data, and reflecting the emotional data in quiz generation. This automates everything from learning material analysis to quiz generation, analysis of learners' answers, and even providing a learning experience that takes emotional states into account, enabling efficient and effective learning management.

[0376] "Instructional materials" refer to materials containing data and information used by learners for learning purposes.

[0377] A "generative AI model" is a program or algorithm that uses artificial intelligence to automatically perform specific tasks.

[0378] A "quiz" is a set of questions given to assess a learner's understanding.

[0379] A "user interface" is a part that provides a screen and operating means for a user to interact with a system.

[0380] A "server" is a computer that processes and manages data on a network.

[0381] "Terminal" means the device through which a user accesses the system.

[0382] "Emotion data" is information that represents the user's emotional state, and is data acquired through a camera or microphone.

[0383] "Analysis" refers to processing data or information to extract specific patterns or features.

[0384] "Comprehension report" refers to the evaluation results that show how well a learner has understood the learning content.

[0385] A "prompt" is a sentence that describes instructions or questions for a generative AI model.

[0386] MODE FOR CARRYING OUT THE INVENTION

[0387] The present invention relates to a system that automatically generates quizzes from learning materials, evaluates the learner's level of understanding, and simultaneously recognizes the user's emotional state and reflects it in the system. The following is a detailed description of this system.

[0388] First, the user logs in to the learning management system using a terminal. To log in, they enter their username and password, and if authentication is successful, they can access the administration screen. Next, the user clicks the "Upload Learning Materials" button on the administration screen and selects the learning material file (e.g., a PDF file) to upload from the file selection dialog. The terminal sends the selected learning material file to the server and displays a message indicating that the file has been successfully uploaded.

[0389] The server saves the uploaded learning material files in a designated storage area (e.g., AWS S3). The server then passes the saved learning material files to a generative AI model (e.g., OpenAI GPT-4), which analyzes the learning material content and extracts key points and keywords, such as "variables," "loops," and "conditional branching."

[0390] Meanwhile, to understand the user's emotional state, the device uses a built-in camera and microphone to capture the user's emotional data (e.g., facial expressions and voice tone). This emotional data is sent to the server in real time, and the server uses an emotion engine (e.g., Amazon Rekognition, Microsoft Azure Face API) to analyze the emotional state (e.g., stress, interest, concentration level, etc.). The analysis results are reflected in the generation of the next quiz.

[0391] The server issues commands to the generative AI model based on the analyzed characteristics of the learning material and the user's emotional state to automatically generate quizzes. Specifically, multiple choice questions, true / false questions, and written questions are generated based on important points. The difficulty and content of the quiz are adjusted according to the user's emotional state. For example, if the user is feeling stressed, easier questions will be given priority.

[0392] The generated quiz is temporarily saved and can be viewed on a management screen for display to the user. From this management screen, the user can check the questions and options of the generated quiz and edit them as necessary. Once editing is complete, the user clicks the "Confirm Quiz" button and the confirmed quiz is sent to the server.

[0393] Next, the server retrieves the list of learners and generates emails to deliver the finalized quiz to each learner. This is done using an email sending API (e.g., SendGrid, Amazon SES). Learners receive the email on their devices and click a link to access the quiz screen.

[0394] When a learner answers a quiz and submits their answers, the answer data is automatically sent to the server. The server receives this answer data and uses an automatic scoring system (e.g., a Google Sheets script or a Microsoft Excel macro) to score each answer and tally the scores. Finally, the server uses a generative AI model based on the tally to analyze each learner's level of comprehension and generate a comprehension report. This report is provided to the user, who can then provide the necessary feedback to the learner.

[0395] For example, if learning materials for learning "basic programming concepts" are uploaded, the generative AI model will analyze the materials and identify key points such as "variable definitions," "types of loops," and "conditional branching." Furthermore, if the emotion engine determines that the "stress level is high," the difficulty level of the generated quiz will be set low.

[0396] Below are some example prompts to input to a generative AI model:

[0397] "Analyze the content of the following learning materials and automatically generate quizzes for learners. The types of quizzes should be multiple choice, true / false, and written. Also, if the user's stress level is high, adjust the difficulty level to a lower level."

[0398] This concludes the detailed description of one embodiment of the present invention. This system makes it easy to automatically generate quizzes and measure comprehension levels, and is expected to provide an optimal learning environment for both users and learners.

[0399] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0400] Step 1:

[0401] A user logs in to a learning management system using a terminal. The user enters a username and password on the login screen, and the terminal sends this to the server. The server performs authentication, and if successful, provides the user with an administration screen. The input is the username and password, and the output is the authentication result. If the authentication result is successful, the administration screen is displayed.

[0402] Step 2:

[0403] The user clicks the "Upload teaching materials" button on the administration screen. The terminal displays a file selection dialog, and the user selects the teaching materials file (e.g., PDF file) to upload. When the "Open" button is clicked, the terminal sends the selected file to the server. The input is the teaching materials file, and the output is a message indicating that the upload has been completed successfully. A message is displayed indicating that the file has been uploaded successfully.

[0404] Step 3:

[0405] The server saves the uploaded learning material file in a storage area (e.g. AWS S3). The input is the uploaded learning material file, and the output is the result of saving it to storage. If saving is successful, the location where the file is saved is recorded.

[0406] Step 4:

[0407] The server passes the saved teaching material file to the generative AI model. It generates a prompt for the generative AI model and analyzes the content of the teaching material. The input is the saved teaching material file and the prompt, and the output is the main points and keywords of the teaching material. The generative AI model extracts key points such as "variables," "loops," and "conditional branching" from the content.

[0408] Step 5:

[0409] The device acquires the user's emotional data using the built-in camera and microphone. The data is sent to the server in real time. The input is the user's video and audio, and the output is emotional data. The acquired data is sent to the server.

[0410] Step 6:

[0411] The server uses an emotion engine to analyze the emotion data. The input is the transmitted emotion data, and the output is the judgment result of the emotion state (e.g., stress, interest, concentration). The emotion engine judges the emotion state and saves the result.

[0412] Step 7:

[0413] The server issues commands to the generative AI model based on the analyzed characteristics of the learning material and the user's emotional state to automatically generate a quiz. The prompt text reflects important points and the user's emotional state. The input is the characteristics of the learning material and the user's emotional state, and the output is the generated quiz (multiple choice questions, true / false questions, and written questions). The difficulty and content of the quiz are adjusted.

[0414] Step 8:

[0415] The server temporarily stores the generated quiz and provides an interface for user confirmation. A list of quizzes is displayed on the management screen. The input is the generated quiz, and the output is the interface for the management screen. The user can check the question text and options.

[0416] Step 9:

[0417] The user checks and edits the generated quiz on the management screen. Once the edits are complete, the user clicks the "Confirm Quiz" button. The input is the user's edits, and the output is the confirmed quiz data. Clicking the Confirm button sends the final data to the server.

[0418] Step 10:

[0419] The server retrieves the learner list and generates emails to distribute the confirmed quiz to each learner. It uses an email sending API. The input is the learner list and confirmed quiz data, and the output is email. An email is sent to each learner.

[0420] Step 11:

[0421] The learner receives the email using their device and clicks on the quiz link. They access the quiz screen, enter their answer, and submit it. The input is the learner's answer, and the output is the result of sending it to the server. The learner's answer is sent to the server.

[0422] Step 12:

[0423] The server receives the learner's answers and uses an automatic scoring system to tally up the scores. The input is the learner's answers and the output is the scoring results. The answers are scored by the automatic scoring system.

[0424] Step 13:

[0425] The server aggregates the results and the generative AI model analyzes each learner's level of understanding. The input is the scoring result, and the output is the comprehension analysis result. The generative AI model evaluates the level of understanding.

[0426] Step 14:

[0427] The server generates a comprehension report from the comprehension analysis results and provides it to the user. The input is the comprehension analysis results and the output is the comprehension report. The user can check the report and provide feedback.

[0428] (Application example 2)

[0429] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0430] Conventional online learning systems struggle to provide a personalized learning experience for each learner, resulting in insufficient learning effectiveness. Furthermore, they lack a means to provide learning content that takes into account the learner's emotional state, often resulting in learners feeling stressed or losing concentration. To solve these problems, a system is needed that recognizes the learner's emotional state in real time and adjusts the difficulty and content of the learning content accordingly.

[0431] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0432] In this invention, the server includes means for inputting learning materials, means for extracting features using a generative AI model that analyzes the input learning materials, means for automatically generating quizzes based on the extracted features, means for recognizing the emotional state of a user while watching, means for adjusting the difficulty of the quiz based on the recognized emotional state, means for providing a user interface for reviewing and editing the generated quiz, means for delivering the reviewed and edited quiz to a target learner, means for collecting and analyzing the quiz answers of the target learner, and means for generating a comprehension report based on the analysis results, thereby enabling the provision of personalized learning content that takes into account the learner's emotional state.

[0433] "Means for inputting teaching materials" refers to a function for uploading educational content (videos, PDFs, text, etc.) to a server from the device used by the learner.

[0434] A "generative AI model" is an artificial intelligence algorithm that analyzes the content of input teaching materials, extracts key points and keywords, and is used to automatically generate quizzes.

[0435] "Means for extracting features" refers to a function that uses a generative AI model to identify key points and important keywords from input learning materials.

[0436] The "means for automatically generating quizzes" is a function that automatically creates quizzes in various formats (multiple choice questions, true / false questions, written questions, etc.) based on the extracted features.

[0437] "Emotion recognition means" is a function that uses technology (such as facial recognition and voice analysis) to recognize the user's emotional state in real time while watching.

[0438] The "means for adjusting the difficulty of the quiz" is a function that dynamically changes the difficulty and content of the generated quiz based on the recognized emotional state of the user.

[0439] The "means for providing a user interface" is a function that provides a screen and operation means for the user to check and edit the generated quiz.

[0440] "Means for delivery to target learners" refers to the functionality for electronically transmitting the reviewed and edited quiz to the target learners.

[0441] "Means for collecting and analyzing quiz answers" is a function for collecting the results of quizzes answered by target learners and analyzing them to understand the learners' level of understanding and learning situation.

[0442] The "means for generating a comprehension report" is a function that creates a report summarizing each learner's level of comprehension and learning progress information based on the quiz answer results and analysis thereof.

[0443] MODE FOR CARRYING OUT THE INVENTION

[0444] The present invention relates to a system for automatically generating quizzes from video content in an online learning platform, and providing a personalized learning experience by recognizing a learner's emotional state in real time. A specific embodiment of the system and its operation are described below.

[0445] System Program

[0446] The system of the present invention consists of the following major components:

[0447] 1. How to input materials:

[0448] This function allows users to upload educational content (videos, PDFs, text, etc.) to the server from their devices. Devices include PCs and smartphones.

[0449] 2. How to extract features using generative AI models:

[0450] The server inputs the uploaded learning materials into a generative AI model to extract key points and keywords. For example, OpenAI GPT-3 is used as the generative AI model.

[0451] 3. Emotion recognition means to recognize the user's emotional state while watching:

[0452] This function uses the device's built-in camera and microphone to capture the user's facial expressions and voice, and analyzes their emotional state in real time. The emotion analysis engine used is, for example, the Microsoft Azure Emotion Analysis API.

[0453] 4. How to adjust the difficulty of the quiz:

[0454] This function allows the server to dynamically change the difficulty and content of the quiz based on the user's emotional state. For example, if the user is feeling stressed, easier questions will be prioritized.

[0455] 5. A means to provide a user interface for reviewing and editing the quiz:

[0456] This function allows the server to display the generated quiz to the user and provides an interface for checking and editing.

[0457] 6. Means of delivery to target learners:

[0458] This function allows the server to distribute the reviewed and edited quiz to the target learners via email or other means.

[0459] 7. How quiz answers will be collected and analyzed:

[0460] This function allows the server to collect and analyze learners' answers to distributed quizzes to assess their level of understanding.

[0461] 8. How to generate a comprehension report:

[0462] This function allows the server to generate a report showing each learner's level of understanding based on the analysis results and provide this to the user.

[0463] Program processing

[0464] The server uses a generative AI model (e.g., OpenAI GPT-3) to analyze the uploaded learning material. Once the learning material is uploaded, the server inputs it into the generative AI model to extract key points and keywords. For example, if a video on "SEO Strategy" is uploaded, the generative AI model will identify key points such as "keyword research" and "content marketing."

[0465] The device uses a built-in camera and microphone to recognize the user's emotional state in real time while watching, and performs analysis using an emotion analysis engine (for example, Microsoft Azure Emotion Analysis API), which determines the user's stress level and interest level.

[0466] The server adjusts the difficulty and content of the generated quiz based on the user's perceived emotional state: for example, it presents easier questions when the user is stressed and more detailed questions when the user is focused.

[0467] The user interface presents the generated quiz to the user and provides a screen for the user to review and edit, allowing the user to modify the question text and answer options.

[0468] The server then distributes the reviewed and edited quizzes to the target learners, who then answer the quizzes. The answers are sent to the server, where they are automatically compiled and analyzed. Finally, a report showing the learners' comprehension is generated and provided to the user.

[0469] Examples and prompts

[0470] For example, if a user is watching a video on "SEO Strategies," the following quiz will be auto-generated:

[0471] Multiple Choice Question: "What are the essential elements of SEO?"

[0472] A. Keyword Research

[0473] B. Visual Design

[0474] C. Database Management

[0475] True or False: "Meta descriptions affect SEO. True or False?"

[0476] Short answer question: "Explain the importance of on-page SEO."

[0477] Example prompt sentence:

[0478] Analyze the content of the following educational material and generate quiz questions. Please take into account the user's emotional state and set the difficulty level low if the user's stress level is high.

[0479] Material Content: SEO Strategies

[0480] This makes it possible to provide learners with personalized learning content and maximize learning effectiveness.

[0481] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0482] Step 1:

[0483] Uploading teaching materials

[0484] Subject: User

[0485] Description: A user uploads educational content (e.g., video, PDF, text file) from a device to a server. Specifically, the user logs in to a learning management system using a device, selects the educational material file, and clicks the "Upload" button. The input is the file path of the target educational material, and the output is the saving of the file to the server.

[0486] Step 2:

[0487] Teaching material analysis

[0488] Subject: Server

[0489] Description: The server inputs the uploaded learning material file into a generative AI model to analyze the content of the learning material. For example, the server uses a generative AI model (OpenAI GPT-3) to extract key points and keywords within the learning material. The input is the uploaded learning material file, and the output is a list of key points and keywords. Data processing includes text extraction and key point detection.

[0490] Step 3:

[0491] emotion recognition

[0492] Subject: Device

[0493] Description: The device uses the built-in camera and microphone to capture the user's facial expressions and voice. This data is sent in real time to an emotion analysis engine (Microsoft Azure Emotion Analysis API) to analyze the user's emotional state. The input is facial expression data and voice data, and the output is a judgment of the user's emotional state (e.g., stress level, interest level, etc.). Operations include facial recognition using the camera and voice analysis using the microphone.

[0494] Step 4:

[0495] Quiz Generation

[0496] Subject: Server

[0497] Description: The server issues commands to a generative AI model based on the extracted key points and emotional state data to automatically generate quizzes. The generated quizzes include a variety of formats, such as multiple choice, true / false, and written questions. The input is a list of key points and emotional state data, and the output is a set of generated quiz questions. Data calculation involves automatically generating quizzes using a generative AI model.

[0498] Step 5:

[0499] Quiz difficulty adjustment

[0500] Subject: Server

[0501] Description: The server dynamically adjusts the difficulty of the generated quiz based on emotional state data. For example, if the user is feeling stressed, easier questions will be prioritized. The input is the emotional state data and the generated quiz, and the output is the quiz questions with the adjusted difficulty. Actions include changing the question text and the format of the options.

[0502] Step 6:

[0503] Check and edit quiz

[0504] Subject: Server

[0505] Description: The server temporarily stores generated quizzes and provides an interface for users to review and edit them. Users can review quizzes from an admin screen and make corrections as needed. The input is the generated quiz set, and the output is the final quiz set edited by the user. Behavior includes providing a user interface and supporting quiz editing.

[0506] Step 7:

[0507] Quiz broadcast

[0508] Subject: Server

[0509] Description: The server generates and sends emails to distribute edited quizzes to target learners. The input is the final quiz set and a list of learners, and the output is an email containing an access link to the quiz. Actions include generating and sending emails.

[0510] Step 8:

[0511] Answer collection and analysis

[0512] Subject: Server

[0513] Description: The target learner answers a quiz on their device and sends the answers to the server. The server collects the answers and performs automatic scoring and comprehension analysis. The input is the learner's quiz answers, and the output is the scoring results and comprehension analysis results. Data calculations include comprehension analysis using an automatic scoring algorithm and a generative AI model.

[0514] Step 9:

[0515] Report Generation

[0516] Subject: Server

[0517] Description: The server generates a comprehension report for each learner based on the analysis results and provides it to the user. The input is the analysis result data, and the output is the comprehension report. The operations include generating the report and providing it to the user.

[0518] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0519] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0520] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0521] [Second embodiment]

[0522] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0523] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0524] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0525] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0526] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0527] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0528] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0529] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0530] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0531] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0532] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0533] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0534] This invention relates to a system that automatically generates quizzes from learning materials and evaluates learners' understanding based on the quizzes. This system efficiently performs a series of processes: inputting learning materials, analyzing them, generating quizzes, and distributing them to learners. The following is a detailed description of this system.

[0535] Program processing explanation

[0536] 1. Uploading teaching materials

[0537] The user uses the terminal to upload learning material files to the server, for example, PDF format learning materials, and sends them to the system via a designated upload button.

[0538] The server saves the received file and displays a notification to the user that the upload is complete.

[0539] 2. Teaching material analysis

[0540] The server passes the saved teaching material files to the generative AI model.

[0541] The generative AI model analyzes the content of the teaching materials and extracts key points and keywords, such as "What is a variable?" and "What are the types of loops?"

[0542] 3. Quiz Generation

[0543] Based on the analysis results, the server issues commands to the generative AI model to automatically generate quizzes, such as multiple choice questions, true / false questions, and essay questions based on key points.

[0544] The created quiz is stored on the server and displayed to the user for prior review.

[0545] 4. Quiz confirmation

[0546] The user can use the device to check the generated quiz content and edit it as needed, for example, by modifying the question text or adding options.

[0547] Once editing is complete, the user clicks the "Confirm Quiz" button to send the finalized quiz to the server.

[0548] 5. Quiz distribution

[0549] The server checks the list of learners to distribute the confirmed quiz to the target learners, and based on this list, sends each learner an email containing an access link to the quiz.

[0550] Learners click on a link from their device to access and answer the quiz.

[0551] 6. Answer collection and analysis

[0552] Once the learner has completed the quiz, the answers are sent to the server and stored.

[0553] The server uses an automated scoring system to score each answer and tally the scores.

[0554] Based on the scoring results, the generative AI model analyzes each learner's level of understanding.

[0555] 7. Report Generation

[0556] The server generates a comprehension report from the analysis results and provides it to the user.

[0557] The user checks the report and provides feedback to the learner if necessary.

[0558] Specific examples

[0559] For example, if a learning material for learning "basic programming concepts" is uploaded, a generative AI model might analyze the material and identify key points such as:

[0560] Defining variables

[0561] Loop Types

[0562] Conditional Branching

[0563] Based on this, the following quiz is generated:

[0564] Multiple choice question: "What is the definition of a variable?"

[0565] A. Memory area

[0566] B. Temporary storage of data

[0567] C. Graphical User Interface

[0568] True or False: "The for loop is used to repeat a fixed number of times. True or False?"

[0569] Short answer question: "Explain the basic usage of a while loop."

[0570] By distributing these quizzes to learners, collecting and analyzing their answers, and generating comprehension reports, efficient learning management becomes possible.

[0571] This concludes the detailed description of one embodiment of the present invention. This system makes it easy to automatically generate quizzes and measure comprehension levels, and is expected to improve the learning efficiency of learners.

[0572] The processing flow will be explained below.

[0573] Step 1:

[0574] The user logs in to the learning management system using a device. They enter their username and password, and if authentication is successful, they are able to access the administration screen.

[0575] Step 2:

[0576] The user clicks the "Upload teaching materials" button on the administration screen, which displays a file selection dialog.

[0577] Step 3:

[0578] The user selects the teaching material file (e.g., a PDF file) to upload and clicks the "Open" button.

[0579] Step 4:

[0580] The device will send the selected teaching material file to the server. A message will appear indicating that the file has been successfully uploaded.

[0581] Step 5:

[0582] The server saves the uploaded learning material files in the specified storage area.

[0583] Step 6:

[0584] The server passes the saved teaching material files to the generative AI model.

[0585] Step 7:

[0586] The generative AI model analyzes the content of the learning material and extracts key points and keywords, such as variables, loops, and conditional branching.

[0587] Step 8:

[0588] The generative AI model returns the analysis results to the server, which include key points and keywords from the learning material.

[0589] Step 9:

[0590] The server generates quiz templates based on the analysis results, which can include multiple choice questions, true / false questions, and essay questions.

[0591] Step 10:

[0592] A generative AI model automatically generates specific quizzes based on templates, including multiple-choice questions such as "What is the definition of a variable?"

[0593] Step 11:

[0594] The server temporarily stores the generated quiz and provides an interface for displaying it to the user.

[0595] Step 12:

[0596] The user can check the generated quiz from the management screen, specifically, the question text, options, correct answers, etc.

[0597] Step 13:

[0598] Users can edit the content of each quiz, for example, by correcting incorrect answers or adding additional options.

[0599] Step 14:

[0600] When the user has finished editing, they click the "Confirm Quiz" button, and the confirmed quiz is sent to the server.

[0601] Step 15:

[0602] The server acquires the learner list and generates an email for delivering the confirmed quiz to each learner.

[0603] Step 16:

[0604] The server sends the learner an email containing an access link to the quiz.

[0605] Step 17:

[0606] Learners receive the email on their device and click on the link to access the quiz screen.

[0607] Step 18:

[0608] The learner answers the quiz and submits the answers, which are automatically sent to the server.

[0609] Step 19:

[0610] The server receives and stores the learner's answers.

[0611] Step 20:

[0612] The server runs an automated scoring system to score each answer, with the scoring criteria set in advance.

[0613] Step 21:

[0614] The server compiles the scoring results and analyzes the level of understanding, such as calculating each student's score and percentage of correct answers.

[0615] Step 22:

[0616] The server generates a comprehension report based on the analysis results, which includes each learner's grades and learning progress.

[0617] Step 23:

[0618] The server provides the generated comprehension report to the user, who can view the report on the management screen.

[0619] Step 24:

[0620] The user can review the comprehension report and provide feedback to the learner as needed, for example by providing explanations for specific questions or additional practice questions.

[0621] Example 1

[0622] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0623] Conventional learning systems have struggled to efficiently generate quizzes from learning materials and accurately assess learners' comprehension. Furthermore, the processes of quiz generation, confirmation, editing, distribution, answer collection, and analysis all require manual work, which is time-consuming and labor-intensive. Furthermore, the quality of the generated quizzes and the reliability of the assessment of learners' comprehension have also been issues.

[0624] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0625] In this invention, the server includes: means for a user to input learning materials; means for extracting key points using a generative AI model that analyzes the input learning materials; means for automatically generating quizzes based on the extracted key points; means for providing a user interface for reviewing and editing the generated quizzes; means for delivering the reviewed and edited quizzes to target learners; means for collecting and scoring quiz answers from target learners; and means for analyzing the scoring results and generating a comprehension report. This makes it possible to efficiently generate quizzes from learning materials and accurately evaluate learners' comprehension.

[0626] "User" refers to the entity that operates the system, uploads learning materials, and checks and edits quizzes.

[0627] "Learning materials" refers to files or data containing information or content that a learner uses to study.

[0628] A "generative AI model" is an artificial intelligence model that analyzes input learning materials, extracts key points, and generates quizzes.

[0629] "Key points" refer to the particularly important concepts and information in the learning material that learners should understand.

[0630] A "quiz" is a problem or question posed to assess a learner's level of understanding.

[0631] A "user interface" refers to the screens and interaction means that allow a user to interact with and operate a system.

[0632] "Delivery" refers to the process of sending the generated quiz or information to the intended recipient.

[0633] A "learner" is an individual who answers quizzes and learns the learning material.

[0634] "Scoring" refers to the process of evaluating and scoring quizzes completed by learners.

[0635] A "comprehension report" is a report that shows the learner's level of understanding based on the scoring results.

[0636] This invention relates to a system that automatically generates quizzes from learning materials and evaluates the learner's level of understanding based on the quizzes. This system efficiently performs a series of processes: inputting learning materials, analyzing them, generating quizzes, and distributing them to learners.

[0637] The system is implemented using the following hardware and software.

[0638] Hardware: Server, user device, learner device

[0639] Software: Generative AI models (e.g., GPT-4), automated scoring systems, email distribution systems

[0640] Program processing explanation

[0641] Uploading teaching materials

[0642] The user uploads the learning material file to the server using his / her own terminal. For example, the user sends the learning material file in PDF format to the system via a designated upload button.

[0643] The server saves the received file and displays a notification to the user that the upload is complete.

[0644] Teaching material analysis

[0645] The server passes the saved teaching material file to the generative AI model and begins analysis.

[0646] The generative AI model analyzes the content of the teaching materials and extracts key points and keywords, such as "What is a variable?" and "What are the types of loops?"

[0647] Quiz Generation

[0648] The server issues commands to the generative AI model based on the analysis results to automatically generate quizzes, such as multiple choice questions, true / false questions, and essay questions based on key points.

[0649] The created quiz is stored on the server and displayed to the user for prior review.

[0650] Quiz Check

[0651] The user can use the device to check the generated quiz content and edit it as needed, for example by modifying the question text or adding options.

[0652] Once editing is complete, the user clicks the "Confirm Quiz" button to send the finalized quiz to the server.

[0653] Quiz broadcast

[0654] The server checks the list of learners to distribute the confirmed quiz to the target learners, and based on this list, sends each learner an email containing an access link to the quiz.

[0655] Learners click on a link from their device to access the quiz and answer the questions.

[0656] Answer collection and analysis

[0657] Once the learner has completed the quiz, the answers are sent to the server and stored.

[0658] The server uses an automated scoring system to score each answer and tally the scores.

[0659] Based on the scoring results, the generative AI model analyzes each learner's level of understanding.

[0660] Report Generation

[0661] The server generates a comprehension report from the analysis results and provides it to the user.

[0662] The user checks the report and provides feedback to the learner if necessary.

[0663] Specific examples

[0664] For example, if a learning material for learning "basic programming concepts" is uploaded, a generative AI model might analyze the material and identify key points such as:

[0665] Defining variables

[0666] Loop Types

[0667] Conditional Branching

[0668] Based on this, the following quiz is generated:

[0669] Multiple choice question: "What is the definition of a variable?"

[0670] A. Memory area

[0671] B. Temporary storage of data

[0672] C. Graphical User Interface

[0673] True or False: "The for loop is used to repeat a fixed number of times. True or False?"

[0674] Short answer question: "Explain the basic usage of a while loop."

[0675] Example prompts to input to the generative AI model

[0676] "Please analyze the following PDF file and extract the main points and keywords."

[0677] "Based on the analysis results below, please generate a quiz in the form of multiple choice, true / false, or written questions."

[0678] "Collect answers corresponding to the next quiz and analyze learners' understanding."

[0679] This completes the detailed description of one embodiment of the present invention. This system makes it easy to automatically generate quizzes and measure comprehension levels, and is expected to improve the learning efficiency of learners.

[0680] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0681] Step 1: Upload your materials

[0682] Users use their own devices to select learning material files, such as PDF files, and click the designated upload button to send them to the server.

[0683] Input: Learning material file selected by the user

[0684] Output: Teaching material files saved on the server

[0685] Specific operation: The user accesses the "Upload learning material" screen and clicks the "Select file" button to select the learning material file. When the user clicks the "Upload" button, the file is sent to the server, and the server displays the message "Learning material upload completed" on the user's device.

[0686] Step 2: Analysis of teaching materials

[0687] The server passes the saved teaching material file to the generative AI model and begins analysis.

[0688] Input: Teaching material file saved on the server

[0689] Output: Key points and keywords extracted by the generative AI model

[0690] Specific operation: The server creates a prompt to input the learning material file into the generative AI model (e.g., "Please extract the key points about the programming concept."). The generative AI model analyzes the learning material file and extracts important information such as "variable definitions," "loop types," and "conditional branching."

[0691] Step 3: Quiz Generation

[0692] Based on the analysis results, the server issues commands to the generative AI model to automatically generate quizzes.

[0693] Input: Key points and keywords extracted by the generative AI model

[0694] Output: Generated quiz (multiple choice, true / false, short answer)

[0695] Specific operation: The server instructs the generative AI model to "create a quiz based on the following points." The generative AI model creates quizzes such as "What is the definition of a variable?" and "A for loop is used to repeat a fixed number of times. Is it true or false?" and saves them on the server.

[0696] Step 4: Quiz Review

[0697] The user uses the terminal to check the generated quiz content and edit it as necessary.

[0698] Input: Generated quiz stored on the server

[0699] Output: Quiz confirmed and edited by the user

[0700] Specific operation: The user accesses the "Quiz Confirmation" screen, reviews the generated quiz, and makes edits such as "revise the question text" and "add options." The user clicks the "Confirm Quiz" button, and the confirmed quiz is sent to the server.

[0701] Step 5: Stream the quiz

[0702] The server checks the learner list to deliver the determined quiz to the target learners.

[0703] Input: Quiz confirmed and edited by user, learner list

[0704] Output: An email containing the quiz access link delivered to the learner.

[0705] Specific operation: The server starts the "quiz distribution" process and checks the learner list. The server automatically sends an email containing an access link to the quiz to each learner. The learner clicks the link in the email and accesses the quiz.

[0706] Step 6: Collect and analyze answers

[0707] The learner answers the quiz and sends the answers to the server.

[0708] The server uses an automatic scoring system to score the answers and tally the scores.

[0709] Input: Quiz answered by learner, scoring system

[0710] Output: Scoring results, comprehension analysis results

[0711] Specific operation: The learner answers the quiz and clicks the "Submit" button. The server passes the received answers to the automatic scoring system, which generates a scoring result. The generative AI model analyzes the scoring result and analyzes the learner's level of understanding.

[0712] Step 7: Generate reports

[0713] The server generates a report from the comprehension analysis results and provides it to the user.

[0714] Input: Comprehension analysis results

[0715] Output: User-viewable comprehension report

[0716] Specific operation: The server executes the "report generation" process and creates a report based on the analysis results. The user checks the "comprehension report" and selects a means to share it with the learner (e.g., send it by email).

[0717] By using the above processing steps, the present invention makes it possible to efficiently generate quizzes from learning materials and accurately evaluate the learner's level of understanding.

[0718] (Application example 1)

[0719] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0720] Conventional systems that create quizzes based on teaching materials and assess learners' comprehension have difficulty meeting the diverse needs of learners and on-site training that requires specific expertise. It has also been difficult to generate and evaluate quizzes for efficient training of new employees and skill development. In particular, there has been a demand for quiz creation and comprehension assessment that effectively utilizes procedures and manuals in training engineers and workers in factories.

[0721] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0722] In this invention, the server includes means for inputting learning materials, means for extracting features using a generative AI model that analyzes the input learning materials, means for automatically generating quizzes based on the extracted features, means for providing a user interface for checking and editing the generated quizzes, means for distributing the checked and edited quizzes to target learners, means for collecting and analyzing the quiz answers of the target learners, means for generating a comprehension report based on the analysis results, means for uploading learning procedures and manuals and generating quizzes to evaluate the comprehension of engineers and workers, and means for use as an application installed on factory robots for training engineers and workers. This makes it possible to efficiently and accurately generate quizzes based on the content of the learning materials and evaluate the comprehension.

[0723] "Teaching materials" are information and materials used for the purpose of learner understanding.

[0724] A "generative AI model" is a model that uses artificial intelligence technology to perform text analysis and information extraction.

[0725] "Features" are important points or keywords extracted from the teaching materials.

[0726] A "quiz" is a question or problem used to assess a learner's understanding.

[0727] A "user interface" is an interface through which a user interacts with a system or application.

[0728] An "engineer" is someone who has specific skills or expertise and works in a factory or on-site.

[0729] "Workers" are workers who perform manual labor or operate machinery in factories or on-site.

[0730] A "procedure" is a document that describes the steps and methods for performing a specific task or operation.

[0731] A "manual" is a document that contains instructions or guidance for a particular operation or procedure.

[0732] "Training" refers to educational activities aimed at improving the skills and knowledge of engineers and workers.

[0733] An "understanding report" is a report summarizing the results of an evaluation of the level of understanding and proficiency of a learner or worker.

[0734] A "factory robot" is a machine or device used to automate work within a factory.

[0735] The present invention relates to a system that automatically generates quizzes from learning materials and evaluates the learner's level of understanding based on the quizzes. This system is installed on factory robots and is effectively used for training engineers and workers.

[0736] The server first receives the learning materials uploaded by the user. These learning materials, such as learning procedures or manuals, are uploaded in PDF format. The PDF file is converted to text using a Python library called pdfplumber. The server then passes this converted text to the generative AI model.

[0737] The generative AI model uses generative AI technology to analyze the teaching materials. This analysis extracts key points and keywords. For example,

[0738] This information includes "What is a variable?", "What type of loop?", etc. In this step, the OpenAI API is used. An example of a prompt sentence output by the generative AI model is as follows:

[0739] Based on the text below, please extract the key points:

[0740] A variable is a symbol that stores specific data. For example, in Python, you can define it as x = 10. There are two types of loops: for loops and while loops. A for loop is used to repeat a fixed number of times, while a while loop repeats as long as a condition is met.

[0741] Based on the analysis results, the server again uses the OpenAI API to automatically generate quizzes. The quizzes generated range from multiple choice questions, true / false questions, and essay questions. For example, the following quiz may be generated:

[0742] Multiple choice question: "What is the definition of a variable?"

[0743] A. Memory area

[0744] B. Temporary storage of data

[0745] C. Graphical User Interface

[0746] True or False: "The for loop is used to repeat a fixed number of times. True or False?"

[0747] Short answer question: "Explain the basic usage of a while loop."

[0748] After the quiz is created, the user can view and edit the content through a dedicated user interface. The quiz is then distributed to the target learners via email or other means, with an access link to the quiz.

[0749] After a learner answers the quiz, their answers are sent to the server and graded by an automatic scoring system. Based on the results, a generative AI model analyzes each learner's level of understanding and generates a comprehension report. This report is provided to the user, who can then provide feedback based on it.

[0750] In one embodiment of the present invention, when used for training engineers and workers in a factory, for example, manuals on how to operate new machinery or safety procedures can be uploaded and analyzed, and quizzes can be generated, allowing for efficient assessment of the level of understanding of specific work tasks and enhancing the effectiveness of training.

[0751] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0752] Step 1:

[0753] The server receives the teaching material files uploaded by the user. Specifically, when the user uploads the teaching material in PDF format, the server saves it and displays a notification to the user that the upload is complete. The input is a PDF file, and the output is PDF data saved on the server.

[0754] Step 2:

[0755] The server uses the pdfplumber library to convert the uploaded PDF file to text data. Specifically, it uses pdfplumber to extract text from each page of the PDF and combine them into a single text data. The input is the saved PDF file, and the output is the extracted text data.

[0756] Step 3:

[0757] The server passes the extracted text data to the generative AI model for analysis. Specifically, it uses the OpenAI API to send the text data to the generative AI model as a prompt sentence, and extracts key points and keywords. The input is the text data, and the output is the analysis results (keywords and points) generated by the AI ​​model.

[0758] Step 4:

[0759] The server then uses the generative AI model again based on the analysis results to automatically generate quizzes. Specifically, the analysis results are used as prompts to generate multiple choice, true / false, and written questions via the OpenAI API. The input is the analysis results, and the output is the generated quiz questions.

[0760] Step 5:

[0761] The server provides the generated quiz questions to the user through a user interface. Specifically, the server displays the generated quiz content on the user interface for the user to review and edit. The input is the generated quiz questions, and the output is the quiz content displayed to the user.

[0762] Step 6:

[0763] The user can view and edit the quiz content using the provided user interface. Specifically, the user can modify the question text or add options. The input is the displayed quiz content, and the output is the final quiz content that the user has viewed and edited.

[0764] Step 7:

[0765] The server distributes the quiz that the user has confirmed to the target learners. Specifically, based on the learner list, it sends an email containing an access link to the quiz to each learner. The input is the confirmed quiz content and the learner list, and the output is the quiz link sent to the learner.

[0766] Step 8:

[0767] The learner uses their own device to access the quiz from the link they received and answers the questions. Specifically, the learner answers the quiz and sends the results to the server. The input is the learner's answer, and the output is the answer data sent to the server.

[0768] Step 9:

[0769] The server automatically grades the collected answer data and tally up the scores of each learner. Specifically, the server's automatic grading system grades the answers and calculates the scores. The input is the answer data, and the output is the graded results.

[0770] Step 10:

[0771] The server uses a generative AI model based on the scoring results to create a comprehension report and provides it to the user. Specifically, it evaluates each learner's level of comprehension based on the analyzed answers and scores, and compiles the results in a report format. The input is the scoring results, and the output is a comprehension report.

[0772] These are the specific processing steps of this system. By processing and calculating data at each step, it is possible to efficiently generate quizzes and evaluate the learner's level of understanding.

[0773] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0774] The present invention relates to a system that automatically generates quizzes from learning materials, evaluates the learner's level of understanding, and simultaneously recognizes the user's emotional state and reflects it in the system. The following is a detailed description of this system.

[0775] Program processing explanation

[0776] 1. Uploading teaching materials

[0777] The user logs in to the learning management system using a device. They enter their username and password, and if authentication is successful, they are able to access the administration screen.

[0778] When a user clicks the "Upload teaching materials" button on the administration screen, a file selection dialog box is displayed.

[0779] The user selects the teaching material file (e.g., a PDF file) to upload and clicks the "Open" button.

[0780] The device will send the selected teaching material file to the server. A message will appear indicating that the file has been successfully uploaded.

[0781] 2. Teaching material analysis

[0782] The server saves the uploaded learning material files in the specified storage area.

[0783] The server passes the saved teaching material files to the generative AI model.

[0784] The generative AI model analyzes the content of the learning material and extracts key points and keywords, such as variables, loops, and conditional branches.

[0785] 3. User Emotion Recognition

[0786] The device uses its built-in camera and microphone to capture the user's emotional data.

[0787] The server utilizes an emotion engine to analyze the acquired emotion data and determine the user's emotional state (eg, stress, interest, concentration, etc.).

[0788] 4. Quiz Generation

[0789] The server issues commands to the generative AI model based on the characteristics of the learning material and the user's emotional state that have been analyzed in advance, and automatically generates quizzes. Specifically, multiple-choice questions, true / false questions, and essay-style questions are generated based on key points.

[0790] The content and difficulty of the generated quiz are adjusted based on the user's emotional state. For example, if the user is feeling stressed, questions with lower difficulty will be given priority.

[0791] 5. Quiz confirmation

[0792] The server temporarily stores the generated quiz and provides an interface for displaying it to the user.

[0793] The user can check the generated quiz from the management screen, specifically, the question text, options, correct answers, etc.

[0794] Users can edit the content of each quiz, for example, by correcting incorrect answers or adding additional options.

[0795] When the user has finished editing, he clicks the "Confirm Quiz" button and the confirmed quiz is sent to the server.

[0796] 6. Quiz Streaming

[0797] The server acquires the learner list and generates an email for delivering the confirmed quiz to each learner.

[0798] The server sends the learner an email containing an access link to the quiz.

[0799] 7. Answer collection and analysis

[0800] Learners receive the email on their device and click on the link to access the quiz screen.

[0801] The learner answers the quiz and submits the answers, which are automatically sent to the server.

[0802] The server receives and stores the learner's answers.

[0803] The server uses an automated scoring system to score each answer and tally the scores.

[0804] Based on the scoring results, the generative AI model analyzes each learner's level of understanding.

[0805] 8. Report Generation

[0806] The server generates a comprehension report from the analysis results and provides it to the user.

[0807] The user checks the report and provides feedback to the learner if necessary.

[0808] Specific examples

[0809] For example, if a learning material for learning "basic programming concepts" is uploaded, the generative AI model analyzes the material and identifies key points such as "variable definitions," "types of loops," and "conditional branching." Furthermore, if the user's emotion engine determines that their "stress level is high," the difficulty level of the generated quiz will be set low:

[0810] Multiple choice question: "What is the definition of a variable?"

[0811] A. Memory area

[0812] B. Temporary storage of data

[0813] C. Graphical User Interface

[0814] True or False: "The for loop is used to repeat a fixed number of times. True or False?"

[0815] Short answer question: "Explain the basic usage of a while loop."

[0816] By distributing these quizzes to learners, collecting and analyzing their answers, and generating comprehension reports, we can efficiently and effectively manage learning and provide an optimal learning experience according to the user's emotional state.

[0817] This concludes the detailed description of one embodiment of the present invention. This system facilitates the automatic generation of quizzes and the measurement of comprehension, and is expected to provide an optimal learning environment for both users and learners.

[0818] The processing flow will be explained below.

[0819] Step 1:

[0820] The user logs in to the learning management system using a device. They enter their username and password, and if authentication is successful, they are able to access the administration screen.

[0821] Step 2:

[0822] The user clicks the "Upload teaching materials" button on the administration screen, which displays a file selection dialog.

[0823] Step 3:

[0824] The user selects the teaching material file (e.g., a PDF file) to upload and clicks the "Open" button.

[0825] Step 4:

[0826] The device will send the selected teaching material file to the server. A message will appear indicating that the file has been successfully uploaded.

[0827] Step 5:

[0828] The server saves the uploaded learning material files in the specified storage area.

[0829] Step 6:

[0830] The server passes the saved teaching material files to the generative AI model.

[0831] Step 7:

[0832] The generative AI model analyzes the content of the learning material and extracts key points and keywords, such as variables, loops, and conditional branching.

[0833] Step 8:

[0834] The device uses its built-in camera and microphone to capture the user's emotional data.

[0835] Step 9:

[0836] The server utilizes an emotion engine to analyze the acquired emotion data and determine the user's emotional state (eg, stress, interest, concentration, etc.).

[0837] Step 10:

[0838] The server issues commands to the generative AI model based on the characteristics of the learning material and the user's emotional state that have been analyzed in advance, and automatically generates quizzes. Specifically, multiple-choice questions, true / false questions, and essay-style questions are generated based on key points.

[0839] Step 11:

[0840] The content and difficulty of the generated quiz are adjusted based on the user's emotional state. For example, if the user is feeling stressed, questions with lower difficulty will be given priority.

[0841] Step 12:

[0842] The server temporarily stores the generated quiz and provides an interface for displaying it to the user.

[0843] Step 13:

[0844] The user can check the generated quiz from the management screen, specifically, the question text, options, correct answers, etc.

[0845] Step 14:

[0846] Users can edit the content of each quiz, for example, by correcting incorrect answers or adding additional options.

[0847] Step 15:

[0848] When the user has finished editing, they click the "Confirm Quiz" button, and the confirmed quiz is sent to the server.

[0849] Step 16:

[0850] The server acquires the learner list and generates an email for delivering the confirmed quiz to each learner.

[0851] Step 17:

[0852] The server sends the learner an email containing an access link to the quiz.

[0853] Step 18:

[0854] Learners receive the email on their device and click on the link to access the quiz screen.

[0855] Step 19:

[0856] The learner answers the quiz and submits the answers, which are automatically sent to the server.

[0857] Step 20:

[0858] The server receives and stores the learner's answers.

[0859] Step 21:

[0860] The server runs an automated scoring system to score each answer, with the scoring criteria set in advance.

[0861] Step 22:

[0862] The server compiles the scoring results and analyzes the level of understanding, such as calculating each student's score and percentage of correct answers.

[0863] Step 23:

[0864] The server generates a comprehension report based on the analysis results, which includes each learner's grades and learning progress.

[0865] Step 24:

[0866] The server provides the generated comprehension report to the user and makes it viewable on the management screen.

[0867] Step 25:

[0868] The user can review the comprehension report and provide feedback to the learner as needed, for example by providing explanations for specific questions or additional practice questions.

[0869] Example 2

[0870] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0871] While remote learning and online education have become increasingly important in recent years, accurately assessing learners' comprehension and providing an optimal learning experience that takes into account their emotional state remains a challenging task. Traditional systems require manual analysis of learning materials and quiz generation, placing a heavy burden on human resources and making them inefficient. Furthermore, they lack the ability to reflect learners' emotional state in real time and maintain their motivation.

[0872] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for inputting learning materials, means for extracting features using a generation AI model that analyzes the input learning materials, means for automatically generating quizzes based on the extracted features, means for providing a user interface for confirming and editing the generated quizzes, means for delivering the confirmed and edited quizzes to target learners, means for collecting and analyzing quiz answers from target learners, means for evaluating the target learners' understanding levels based on the collected data and generating comprehension reports based on the analysis results, and means for acquiring user emotional data from the terminal, analyzing the emotional data, and reflecting the emotional data in quiz generation. This automates everything from learning material analysis to quiz generation, analysis of learners' answers, and even providing a learning experience that takes emotional states into account, enabling efficient and effective learning management.

[0873] "Instructional materials" refer to materials containing data and information used by learners for learning purposes.

[0874] A "generative AI model" is a program or algorithm that uses artificial intelligence to automatically perform specific tasks.

[0875] A "quiz" is a set of questions given to assess a learner's understanding.

[0876] A "user interface" is a part that provides a screen and operating means for a user to interact with a system.

[0877] A "server" is a computer that processes and manages data on a network.

[0878] "Terminal" means the device through which a user accesses the system.

[0879] "Emotion data" is information that represents the user's emotional state, and is data acquired through a camera or microphone.

[0880] "Analysis" refers to processing data or information to extract specific patterns or features.

[0881] "Comprehension report" refers to the evaluation results that show how well a learner has understood the learning content.

[0882] A "prompt" is a sentence that describes instructions or questions for a generative AI model.

[0883] MODE FOR CARRYING OUT THE INVENTION

[0884] The present invention relates to a system that automatically generates quizzes from learning materials, evaluates the learner's level of understanding, and simultaneously recognizes the user's emotional state and reflects it in the system. The following is a detailed description of this system.

[0885] First, the user logs in to the learning management system using a terminal. To log in, they enter their username and password, and if authentication is successful, they can access the administration screen. Next, the user clicks the "Upload Learning Materials" button on the administration screen and selects the learning material file (e.g., a PDF file) to upload from the file selection dialog. The terminal sends the selected learning material file to the server and displays a message indicating that the file has been successfully uploaded.

[0886] The server saves the uploaded learning material files in a designated storage area (e.g., AWS S3). The server then passes the saved learning material files to a generative AI model (e.g., OpenAI GPT-4), which analyzes the learning material content and extracts key points and keywords, such as "variables," "loops," and "conditional branching."

[0887] Meanwhile, to understand the user's emotional state, the device uses a built-in camera and microphone to capture the user's emotional data (e.g., facial expressions and voice tone). This emotional data is sent to the server in real time, and the server uses an emotion engine (e.g., Amazon Rekognition, Microsoft Azure Face API) to analyze the emotional state (e.g., stress, interest, concentration level, etc.). The analysis results are reflected in the generation of the next quiz.

[0888] The server issues commands to the generative AI model based on the analyzed characteristics of the learning material and the user's emotional state to automatically generate quizzes. Specifically, multiple choice questions, true / false questions, and written questions are generated based on important points. The difficulty and content of the quiz are adjusted according to the user's emotional state. For example, if the user is feeling stressed, easier questions will be given priority.

[0889] The generated quiz is temporarily saved and can be viewed on a management screen for display to the user. From this management screen, the user can check the questions and options of the generated quiz and edit them as necessary. Once editing is complete, the user clicks the "Confirm Quiz" button and the confirmed quiz is sent to the server.

[0890] Next, the server retrieves the list of learners and generates emails to deliver the finalized quiz to each learner. This is done using an email sending API (e.g., SendGrid, Amazon SES). Learners receive the email on their devices and click a link to access the quiz screen.

[0891] When a learner answers a quiz and submits their answers, the answer data is automatically sent to the server. The server receives this answer data and uses an automatic scoring system (e.g., a Google Sheets script or a Microsoft Excel macro) to score each answer and tally the scores. Finally, the server uses a generative AI model based on the tally to analyze each learner's level of comprehension and generate a comprehension report. This report is provided to the user, who can then provide the necessary feedback to the learner.

[0892] For example, if learning materials for learning "basic programming concepts" are uploaded, the generative AI model will analyze the materials and identify key points such as "variable definitions," "types of loops," and "conditional branching." Furthermore, if the emotion engine determines that the "stress level is high," the difficulty level of the generated quiz will be set low.

[0893] Below are some example prompts to input to a generative AI model:

[0894] "Analyze the content of the following learning materials and automatically generate quizzes for learners. The types of quizzes should be multiple choice, true / false, and written. Also, if the user's stress level is high, adjust the difficulty level to a lower level."

[0895] This concludes the detailed description of one embodiment of the present invention. This system makes it easy to automatically generate quizzes and measure comprehension levels, and is expected to provide an optimal learning environment for both users and learners.

[0896] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0897] Step 1:

[0898] A user logs in to a learning management system using a terminal. The user enters a username and password on the login screen, and the terminal sends this to the server. The server performs authentication, and if successful, provides the user with an administration screen. The input is the username and password, and the output is the authentication result. If the authentication result is successful, the administration screen is displayed.

[0899] Step 2:

[0900] The user clicks the "Upload teaching materials" button on the administration screen. The terminal displays a file selection dialog, and the user selects the teaching materials file (e.g., PDF file) to upload. When the "Open" button is clicked, the terminal sends the selected file to the server. The input is the teaching materials file, and the output is a message indicating that the upload has been completed successfully. A message is displayed indicating that the file has been uploaded successfully.

[0901] Step 3:

[0902] The server saves the uploaded learning material file in a storage area (e.g. AWS S3). The input is the uploaded learning material file, and the output is the result of saving it to storage. If saving is successful, the location where the file is saved is recorded.

[0903] Step 4:

[0904] The server passes the saved teaching material file to the generative AI model. It generates a prompt for the generative AI model and analyzes the content of the teaching material. The input is the saved teaching material file and the prompt, and the output is the main points and keywords of the teaching material. The generative AI model extracts key points such as "variables," "loops," and "conditional branching" from the content.

[0905] Step 5:

[0906] The device acquires the user's emotional data using the built-in camera and microphone. The data is sent to the server in real time. The input is the user's video and audio, and the output is emotional data. The acquired data is sent to the server.

[0907] Step 6:

[0908] The server uses an emotion engine to analyze the emotion data. The input is the transmitted emotion data, and the output is the judgment result of the emotion state (e.g., stress, interest, concentration). The emotion engine judges the emotion state and saves the result.

[0909] Step 7:

[0910] The server issues commands to the generative AI model based on the analyzed characteristics of the learning material and the user's emotional state to automatically generate a quiz. The prompt text reflects important points and the user's emotional state. The input is the characteristics of the learning material and the user's emotional state, and the output is the generated quiz (multiple choice questions, true / false questions, and written questions). The difficulty and content of the quiz are adjusted.

[0911] Step 8:

[0912] The server temporarily stores the generated quiz and provides an interface for user confirmation. A list of quizzes is displayed on the management screen. The input is the generated quiz, and the output is the interface for the management screen. The user can check the question text and options.

[0913] Step 9:

[0914] The user checks and edits the generated quiz on the management screen. Once the edits are complete, the user clicks the "Confirm Quiz" button. The input is the user's edits, and the output is the confirmed quiz data. Clicking the Confirm button sends the final data to the server.

[0915] Step 10:

[0916] The server retrieves the learner list and generates emails to distribute the confirmed quiz to each learner. It uses an email sending API. The input is the learner list and confirmed quiz data, and the output is email. An email is sent to each learner.

[0917] Step 11:

[0918] The learner receives the email using their device and clicks on the quiz link. They access the quiz screen, enter their answer, and submit it. The input is the learner's answer, and the output is the result of sending it to the server. The learner's answer is sent to the server.

[0919] Step 12:

[0920] The server receives the learner's answers and uses an automatic scoring system to tally up the scores. The input is the learner's answers and the output is the scoring results. The answers are scored by the automatic scoring system.

[0921] Step 13:

[0922] The server aggregates the results and the generative AI model analyzes each learner's level of understanding. The input is the scoring result, and the output is the comprehension analysis result. The generative AI model evaluates the level of understanding.

[0923] Step 14:

[0924] The server generates a comprehension report from the comprehension analysis results and provides it to the user. The input is the comprehension analysis results and the output is the comprehension report. The user can check the report and provide feedback.

[0925] (Application example 2)

[0926] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0927] Conventional online learning systems struggle to provide a personalized learning experience for each learner, resulting in insufficient learning effectiveness. Furthermore, they lack a means to provide learning content that takes into account the learner's emotional state, often resulting in learners feeling stressed or losing concentration. To solve these problems, a system is needed that recognizes the learner's emotional state in real time and adjusts the difficulty and content of the learning content accordingly.

[0928] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0929] In this invention, the server includes means for inputting learning materials, means for extracting features using a generative AI model that analyzes the input learning materials, means for automatically generating quizzes based on the extracted features, means for recognizing the emotional state of a user while watching, means for adjusting the difficulty of the quiz based on the recognized emotional state, means for providing a user interface for reviewing and editing the generated quiz, means for delivering the reviewed and edited quiz to a target learner, means for collecting and analyzing the quiz answers of the target learner, and means for generating a comprehension report based on the analysis results, thereby enabling the provision of personalized learning content that takes into account the learner's emotional state.

[0930] "Means for inputting teaching materials" refers to a function for uploading educational content (videos, PDFs, text, etc.) to a server from the device used by the learner.

[0931] A "generative AI model" is an artificial intelligence algorithm that analyzes the content of input teaching materials, extracts key points and keywords, and is used to automatically generate quizzes.

[0932] "Means for extracting features" refers to a function that uses a generative AI model to identify key points and important keywords from input learning materials.

[0933] The "means for automatically generating quizzes" is a function that automatically creates quizzes in various formats (multiple choice questions, true / false questions, written questions, etc.) based on the extracted features.

[0934] "Emotion recognition means" is a function that uses technology (such as facial recognition and voice analysis) to recognize the user's emotional state in real time while watching.

[0935] The "means for adjusting the difficulty of the quiz" is a function that dynamically changes the difficulty and content of the generated quiz based on the recognized emotional state of the user.

[0936] The "means for providing a user interface" is a function that provides a screen and operation means for the user to check and edit the generated quiz.

[0937] "Means for delivery to target learners" refers to the functionality for electronically transmitting the reviewed and edited quiz to the target learners.

[0938] "Means for collecting and analyzing quiz answers" is a function for collecting the results of quizzes answered by target learners and analyzing them to understand the learners' level of understanding and learning situation.

[0939] The "means for generating a comprehension report" is a function that creates a report summarizing each learner's level of comprehension and learning progress information based on the quiz answer results and analysis thereof.

[0940] MODE FOR CARRYING OUT THE INVENTION

[0941] The present invention relates to a system for automatically generating quizzes from video content in an online learning platform, and providing a personalized learning experience by recognizing a learner's emotional state in real time. A specific embodiment of the system and its operation are described below.

[0942] System Program

[0943] The system of the present invention consists of the following major components:

[0944] 1. How to input materials:

[0945] This function allows users to upload educational content (videos, PDFs, text, etc.) to the server from their devices. Devices include PCs and smartphones.

[0946] 2. How to extract features using generative AI models:

[0947] The server inputs the uploaded learning materials into a generative AI model to extract key points and keywords. For example, OpenAI GPT-3 is used as the generative AI model.

[0948] 3. Emotion recognition means to recognize the user's emotional state while watching:

[0949] This function uses the device's built-in camera and microphone to capture the user's facial expressions and voice, and analyzes their emotional state in real time. The emotion analysis engine used is, for example, the Microsoft Azure Emotion Analysis API.

[0950] 4. How to adjust the difficulty of the quiz:

[0951] This function allows the server to dynamically change the difficulty and content of the quiz based on the user's emotional state. For example, if the user is feeling stressed, easier questions will be prioritized.

[0952] 5. A means to provide a user interface for reviewing and editing the quiz:

[0953] This function allows the server to display the generated quiz to the user and provides an interface for checking and editing.

[0954] 6. Means of delivery to target learners:

[0955] This function allows the server to distribute the reviewed and edited quiz to the target learners via email or other means.

[0956] 7. How quiz answers will be collected and analyzed:

[0957] This function allows the server to collect and analyze learners' answers to distributed quizzes to assess their level of understanding.

[0958] 8. How to generate a comprehension report:

[0959] This function allows the server to generate a report showing each learner's level of understanding based on the analysis results and provide this to the user.

[0960] Program processing

[0961] The server uses a generative AI model (e.g., OpenAI GPT-3) to analyze the uploaded learning material. Once the learning material is uploaded, the server inputs it into the generative AI model to extract key points and keywords. For example, if a video on "SEO Strategy" is uploaded, the generative AI model will identify key points such as "keyword research" and "content marketing."

[0962] The device uses a built-in camera and microphone to recognize the user's emotional state in real time while watching, and performs analysis using an emotion analysis engine (for example, Microsoft Azure Emotion Analysis API), which determines the user's stress level and interest level.

[0963] The server adjusts the difficulty and content of the generated quiz based on the user's perceived emotional state: for example, it presents easier questions when the user is stressed and more detailed questions when the user is focused.

[0964] The user interface presents the generated quiz to the user and provides a screen for the user to review and edit, allowing the user to modify the question text and answer options.

[0965] The server then distributes the reviewed and edited quizzes to the target learners, who then answer the quizzes. The answers are sent to the server, where they are automatically compiled and analyzed. Finally, a report showing the learners' comprehension is generated and provided to the user.

[0966] Examples and prompts

[0967] For example, if a user is watching a video on "SEO Strategies," the following quiz will be auto-generated:

[0968] Multiple Choice Question: "What are the essential elements of SEO?"

[0969] A. Keyword Research

[0970] B. Visual Design

[0971] C. Database Management

[0972] True or False: "Meta descriptions affect SEO. True or False?"

[0973] Short answer question: "Explain the importance of on-page SEO."

[0974] Example prompt sentence:

[0975] Analyze the content of the following educational material and generate quiz questions. Please take into account the user's emotional state and set the difficulty level low if the user's stress level is high.

[0976] Material Content: SEO Strategies

[0977] This makes it possible to provide learners with personalized learning content and maximize learning effectiveness.

[0978] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0979] Step 1:

[0980] Uploading teaching materials

[0981] Subject: User

[0982] Description: A user uploads educational content (e.g., video, PDF, text file) from a device to a server. Specifically, the user logs in to a learning management system using a device, selects the educational material file, and clicks the "Upload" button. The input is the file path of the target educational material, and the output is the saving of the file to the server.

[0983] Step 2:

[0984] Teaching material analysis

[0985] Subject: Server

[0986] Description: The server inputs the uploaded learning material file into a generative AI model to analyze the content of the learning material. For example, the server uses a generative AI model (OpenAI GPT-3) to extract key points and keywords within the learning material. The input is the uploaded learning material file, and the output is a list of key points and keywords. Data processing includes text extraction and key point detection.

[0987] Step 3:

[0988] emotion recognition

[0989] Subject: Device

[0990] Description: The device uses the built-in camera and microphone to capture the user's facial expressions and voice. This data is sent in real time to an emotion analysis engine (Microsoft Azure Emotion Analysis API) to analyze the user's emotional state. The input is facial expression data and voice data, and the output is a judgment of the user's emotional state (e.g., stress level, interest level, etc.). Operations include facial recognition using the camera and voice analysis using the microphone.

[0991] Step 4:

[0992] Quiz Generation

[0993] Subject: Server

[0994] Description: The server issues commands to a generative AI model based on the extracted key points and emotional state data to automatically generate quizzes. The generated quizzes include a variety of formats, such as multiple choice, true / false, and written questions. The input is a list of key points and emotional state data, and the output is a set of generated quiz questions. Data calculation involves automatically generating quizzes using a generative AI model.

[0995] Step 5:

[0996] Quiz difficulty adjustment

[0997] Subject: Server

[0998] Description: The server dynamically adjusts the difficulty of the generated quiz based on emotional state data. For example, if the user is feeling stressed, easier questions will be prioritized. The input is the emotional state data and the generated quiz, and the output is the quiz questions with the adjusted difficulty. Actions include changing the question text and the format of the options.

[0999] Step 6:

[1000] Check and edit quiz

[1001] Subject: Server

[1002] Description: The server temporarily stores generated quizzes and provides an interface for users to review and edit them. Users can review quizzes from an admin screen and make corrections as needed. The input is the generated quiz set, and the output is the final quiz set edited by the user. Behavior includes providing a user interface and supporting quiz editing.

[1003] Step 7:

[1004] Quiz broadcast

[1005] Subject: Server

[1006] Description: The server generates and sends emails to distribute edited quizzes to target learners. The input is the final quiz set and a list of learners, and the output is an email containing an access link to the quiz. Actions include generating and sending emails.

[1007] Step 8:

[1008] Answer collection and analysis

[1009] Subject: Server

[1010] Description: The target learner answers a quiz on their device and sends the answers to the server. The server collects the answers and performs automatic scoring and comprehension analysis. The input is the learner's quiz answers, and the output is the scoring results and comprehension analysis results. Data calculations include comprehension analysis using an automatic scoring algorithm and a generative AI model.

[1011] Step 9:

[1012] Report Generation

[1013] Subject: Server

[1014] Description: The server generates a comprehension report for each learner based on the analysis results and provides it to the user. The input is the analysis result data, and the output is the comprehension report. The operations include generating the report and providing it to the user.

[1015] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1016] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1017] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1018] [Third embodiment]

[1019] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1020] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1021] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1022] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1023] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1024] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1025] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1026] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1027] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1028] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1029] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1030] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1031] This invention relates to a system that automatically generates quizzes from learning materials and evaluates learners' understanding based on the quizzes. This system efficiently performs a series of processes: inputting learning materials, analyzing them, generating quizzes, and distributing them to learners. The following is a detailed description of this system.

[1032] Program processing explanation

[1033] 1. Uploading teaching materials

[1034] The user uses the terminal to upload learning material files to the server, for example, PDF format learning materials, and sends them to the system via a designated upload button.

[1035] The server saves the received file and displays a notification to the user that the upload is complete.

[1036] 2. Teaching material analysis

[1037] The server passes the saved teaching material files to the generative AI model.

[1038] The generative AI model analyzes the content of the teaching materials and extracts key points and keywords, such as "What is a variable?" and "What are the types of loops?"

[1039] 3. Quiz Generation

[1040] Based on the analysis results, the server issues commands to the generative AI model to automatically generate quizzes, such as multiple choice questions, true / false questions, and essay questions based on key points.

[1041] The created quiz is stored on the server and displayed to the user for prior review.

[1042] 4. Quiz confirmation

[1043] The user can use the device to check the generated quiz content and edit it as needed, for example, by modifying the question text or adding options.

[1044] Once editing is complete, the user clicks the "Confirm Quiz" button to send the finalized quiz to the server.

[1045] 5. Quiz distribution

[1046] The server checks the list of learners to distribute the confirmed quiz to the target learners, and based on this list, sends each learner an email containing an access link to the quiz.

[1047] Learners click on a link from their device to access and answer the quiz.

[1048] 6. Answer collection and analysis

[1049] Once the learner has completed the quiz, the answers are sent to the server and stored.

[1050] The server uses an automated scoring system to score each answer and tally the scores.

[1051] Based on the scoring results, the generative AI model analyzes each learner's level of understanding.

[1052] 7. Report Generation

[1053] The server generates a comprehension report from the analysis results and provides it to the user.

[1054] The user checks the report and provides feedback to the learner if necessary.

[1055] Specific examples

[1056] For example, if a learning material for learning "basic programming concepts" is uploaded, a generative AI model might analyze the material and identify key points such as:

[1057] Defining variables

[1058] Loop Types

[1059] Conditional Branching

[1060] Based on this, the following quiz is generated:

[1061] Multiple choice question: "What is the definition of a variable?"

[1062] A. Memory area

[1063] B. Temporary storage of data

[1064] C. Graphical User Interface

[1065] True or False: "The for loop is used to repeat a fixed number of times. True or False?"

[1066] Short answer question: "Explain the basic usage of a while loop."

[1067] By distributing these quizzes to learners, collecting and analyzing their answers, and generating comprehension reports, efficient learning management becomes possible.

[1068] This concludes the detailed description of one embodiment of the present invention. This system makes it easy to automatically generate quizzes and measure comprehension levels, and is expected to improve the learning efficiency of learners.

[1069] The processing flow will be explained below.

[1070] Step 1:

[1071] The user logs in to the learning management system using a device. They enter their username and password, and if authentication is successful, they are able to access the administration screen.

[1072] Step 2:

[1073] The user clicks the "Upload teaching materials" button on the administration screen, which displays a file selection dialog.

[1074] Step 3:

[1075] The user selects the teaching material file (e.g., a PDF file) to upload and clicks the "Open" button.

[1076] Step 4:

[1077] The device will send the selected teaching material file to the server. A message will appear indicating that the file has been successfully uploaded.

[1078] Step 5:

[1079] The server saves the uploaded learning material files in the specified storage area.

[1080] Step 6:

[1081] The server passes the saved teaching material files to the generative AI model.

[1082] Step 7:

[1083] The generative AI model analyzes the content of the learning material and extracts key points and keywords, such as variables, loops, and conditional branching.

[1084] Step 8:

[1085] The generative AI model returns the analysis results to the server, which include key points and keywords from the learning material.

[1086] Step 9:

[1087] The server generates quiz templates based on the analysis results, which can include multiple choice questions, true / false questions, and essay questions.

[1088] Step 10:

[1089] A generative AI model automatically generates specific quizzes based on templates, including multiple-choice questions such as "What is the definition of a variable?"

[1090] Step 11:

[1091] The server temporarily stores the generated quiz and provides an interface for displaying it to the user.

[1092] Step 12:

[1093] The user can check the generated quiz from the management screen, specifically, the question text, options, correct answers, etc.

[1094] Step 13:

[1095] Users can edit the content of each quiz, for example, by correcting incorrect answers or adding additional options.

[1096] Step 14:

[1097] When the user has finished editing, they click the "Confirm Quiz" button, and the confirmed quiz is sent to the server.

[1098] Step 15:

[1099] The server acquires the learner list and generates an email for delivering the confirmed quiz to each learner.

[1100] Step 16:

[1101] The server sends the learner an email containing an access link to the quiz.

[1102] Step 17:

[1103] Learners receive the email on their device and click on the link to access the quiz screen.

[1104] Step 18:

[1105] The learner answers the quiz and submits the answers, which are automatically sent to the server.

[1106] Step 19:

[1107] The server receives and stores the learner's answers.

[1108] Step 20:

[1109] The server runs an automated scoring system to score each answer, with the scoring criteria set in advance.

[1110] Step 21:

[1111] The server compiles the scoring results and analyzes the level of understanding, such as calculating each student's score and percentage of correct answers.

[1112] Step 22:

[1113] The server generates a comprehension report based on the analysis results, which includes each learner's grades and learning progress.

[1114] Step 23:

[1115] The server provides the generated comprehension report to the user, who can view the report on the management screen.

[1116] Step 24:

[1117] The user can review the comprehension report and provide feedback to the learner as needed, for example by providing explanations for specific questions or additional practice questions.

[1118] Example 1

[1119] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1120] Conventional learning systems have struggled to efficiently generate quizzes from learning materials and accurately assess learners' comprehension. Furthermore, the processes of quiz generation, confirmation, editing, distribution, answer collection, and analysis all require manual work, which is time-consuming and labor-intensive. Furthermore, the quality of the generated quizzes and the reliability of the assessment of learners' comprehension have also been issues.

[1121] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1122] In this invention, the server includes: means for a user to input learning materials; means for extracting key points using a generative AI model that analyzes the input learning materials; means for automatically generating quizzes based on the extracted key points; means for providing a user interface for reviewing and editing the generated quizzes; means for delivering the reviewed and edited quizzes to target learners; means for collecting and scoring quiz answers from target learners; and means for analyzing the scoring results and generating a comprehension report. This makes it possible to efficiently generate quizzes from learning materials and accurately evaluate learners' comprehension.

[1123] "User" refers to the entity that operates the system, uploads learning materials, and checks and edits quizzes.

[1124] "Learning materials" refers to files or data containing information or content that a learner uses to study.

[1125] A "generative AI model" is an artificial intelligence model that analyzes input learning materials, extracts key points, and generates quizzes.

[1126] "Key points" refer to the particularly important concepts and information in the learning material that learners should understand.

[1127] A "quiz" is a problem or question posed to assess a learner's level of understanding.

[1128] A "user interface" refers to the screens and interaction means that allow a user to interact with and operate a system.

[1129] "Delivery" refers to the process of sending the generated quiz or information to the intended recipient.

[1130] A "learner" is an individual who answers quizzes and learns the learning material.

[1131] "Scoring" refers to the process of evaluating and scoring quizzes completed by learners.

[1132] A "comprehension report" is a report that shows the learner's level of understanding based on the scoring results.

[1133] This invention relates to a system that automatically generates quizzes from learning materials and evaluates the learner's level of understanding based on the quizzes. This system efficiently performs a series of processes: inputting learning materials, analyzing them, generating quizzes, and distributing them to learners.

[1134] The system is implemented using the following hardware and software.

[1135] Hardware: Server, user device, learner device

[1136] Software: Generative AI models (e.g., GPT-4), automated scoring systems, email distribution systems

[1137] Program processing explanation

[1138] Uploading teaching materials

[1139] The user uploads the learning material file to the server using his / her own terminal. For example, the user sends the learning material file in PDF format to the system via a designated upload button.

[1140] The server saves the received file and displays a notification to the user that the upload is complete.

[1141] Teaching material analysis

[1142] The server passes the saved teaching material file to the generative AI model and begins analysis.

[1143] The generative AI model analyzes the content of the teaching materials and extracts key points and keywords, such as "What is a variable?" and "What are the types of loops?"

[1144] Quiz Generation

[1145] The server issues commands to the generative AI model based on the analysis results to automatically generate quizzes, such as multiple choice questions, true / false questions, and essay questions based on key points.

[1146] The created quiz is stored on the server and displayed to the user for prior review.

[1147] Quiz Check

[1148] The user can use the device to check the generated quiz content and edit it as needed, for example by modifying the question text or adding options.

[1149] Once editing is complete, the user clicks the "Confirm Quiz" button to send the finalized quiz to the server.

[1150] Quiz broadcast

[1151] The server checks the list of learners to distribute the confirmed quiz to the target learners, and based on this list, sends each learner an email containing an access link to the quiz.

[1152] Learners click on a link from their device to access the quiz and answer the questions.

[1153] Answer collection and analysis

[1154] Once the learner has completed the quiz, the answers are sent to the server and stored.

[1155] The server uses an automated scoring system to score each answer and tally the scores.

[1156] Based on the scoring results, the generative AI model analyzes each learner's level of understanding.

[1157] Report Generation

[1158] The server generates a comprehension report from the analysis results and provides it to the user.

[1159] The user checks the report and provides feedback to the learner if necessary.

[1160] Specific examples

[1161] For example, if a learning material for learning "basic programming concepts" is uploaded, a generative AI model might analyze the material and identify key points such as:

[1162] Defining variables

[1163] Loop Types

[1164] Conditional Branching

[1165] Based on this, the following quiz is generated:

[1166] Multiple choice question: "What is the definition of a variable?"

[1167] A. Memory area

[1168] B. Temporary storage of data

[1169] C. Graphical User Interface

[1170] True or False: "The for loop is used to repeat a fixed number of times. True or False?"

[1171] Short answer question: "Explain the basic usage of a while loop."

[1172] Example prompts to input to the generative AI model

[1173] "Please analyze the following PDF file and extract the main points and keywords."

[1174] "Based on the analysis results below, please generate a quiz in the form of multiple choice, true / false, or written questions."

[1175] "Collect answers corresponding to the next quiz and analyze learners' understanding."

[1176] This completes the detailed description of one embodiment of the present invention. This system makes it easy to automatically generate quizzes and measure comprehension levels, and is expected to improve the learning efficiency of learners.

[1177] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1178] Step 1: Upload your materials

[1179] Users use their own devices to select learning material files, such as PDF files, and click the designated upload button to send them to the server.

[1180] Input: Learning material file selected by the user

[1181] Output: Teaching material files saved on the server

[1182] Specific operation: The user accesses the "Upload learning material" screen and clicks the "Select file" button to select the learning material file. When the user clicks the "Upload" button, the file is sent to the server, and the server displays the message "Learning material upload completed" on the user's device.

[1183] Step 2: Analysis of teaching materials

[1184] The server passes the saved teaching material file to the generative AI model and begins analysis.

[1185] Input: Teaching material file saved on the server

[1186] Output: Key points and keywords extracted by the generative AI model

[1187] Specific operation: The server creates a prompt to input the learning material file into the generative AI model (e.g., "Please extract the key points about the programming concept."). The generative AI model analyzes the learning material file and extracts important information such as "variable definitions," "loop types," and "conditional branching."

[1188] Step 3: Quiz Generation

[1189] Based on the analysis results, the server issues commands to the generative AI model to automatically generate quizzes.

[1190] Input: Key points and keywords extracted by the generative AI model

[1191] Output: Generated quiz (multiple choice, true / false, short answer)

[1192] Specific operation: The server instructs the generative AI model to "create a quiz based on the following points." The generative AI model creates quizzes such as "What is the definition of a variable?" and "A for loop is used to repeat a fixed number of times. Is it true or false?" and saves them on the server.

[1193] Step 4: Quiz Review

[1194] The user uses the terminal to check the generated quiz content and edit it as necessary.

[1195] Input: Generated quiz stored on the server

[1196] Output: Quiz confirmed and edited by the user

[1197] Specific operation: The user accesses the "Quiz Confirmation" screen, reviews the generated quiz, and makes edits such as "revise the question text" and "add options." The user clicks the "Confirm Quiz" button, and the confirmed quiz is sent to the server.

[1198] Step 5: Stream the quiz

[1199] The server checks the learner list to deliver the determined quiz to the target learners.

[1200] Input: Quiz confirmed and edited by user, learner list

[1201] Output: An email containing the quiz access link delivered to the learner.

[1202] Specific operation: The server starts the "quiz distribution" process and checks the learner list. The server automatically sends an email containing an access link to the quiz to each learner. The learner clicks the link in the email and accesses the quiz.

[1203] Step 6: Collect and analyze answers

[1204] The learner answers the quiz and sends the answers to the server.

[1205] The server uses an automatic scoring system to score the answers and tally the scores.

[1206] Input: Quiz answered by learner, scoring system

[1207] Output: Scoring results, comprehension analysis results

[1208] Specific operation: The learner answers the quiz and clicks the "Submit" button. The server passes the received answers to the automatic scoring system, which generates a scoring result. The generative AI model analyzes the scoring result and analyzes the learner's level of understanding.

[1209] Step 7: Generate reports

[1210] The server generates a report from the comprehension analysis results and provides it to the user.

[1211] Input: Comprehension analysis results

[1212] Output: User-viewable comprehension report

[1213] Specific operation: The server executes the "report generation" process and creates a report based on the analysis results. The user checks the "comprehension report" and selects a means to share it with the learner (e.g., send it by email).

[1214] By using the above processing steps, the present invention makes it possible to efficiently generate quizzes from learning materials and accurately evaluate the learner's level of understanding.

[1215] (Application example 1)

[1216] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1217] Conventional systems that create quizzes based on teaching materials and assess learners' comprehension have difficulty meeting the diverse needs of learners and on-site training that requires specific expertise. It has also been difficult to generate and evaluate quizzes for efficient training of new employees and skill development. In particular, there has been a demand for quiz creation and comprehension assessment that effectively utilizes procedures and manuals in training engineers and workers in factories.

[1218] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1219] In this invention, the server includes means for inputting learning materials, means for extracting features using a generative AI model that analyzes the input learning materials, means for automatically generating quizzes based on the extracted features, means for providing a user interface for checking and editing the generated quizzes, means for distributing the checked and edited quizzes to target learners, means for collecting and analyzing the quiz answers of the target learners, means for generating a comprehension report based on the analysis results, means for uploading learning procedures and manuals and generating quizzes to evaluate the comprehension of engineers and workers, and means for use as an application installed on factory robots for training engineers and workers. This makes it possible to efficiently and accurately generate quizzes based on the content of the learning materials and evaluate the comprehension.

[1220] "Teaching materials" are information and materials used for the purpose of learner understanding.

[1221] A "generative AI model" is a model that uses artificial intelligence technology to perform text analysis and information extraction.

[1222] "Features" are important points or keywords extracted from the teaching materials.

[1223] A "quiz" is a question or problem used to assess a learner's understanding.

[1224] A "user interface" is an interface through which a user interacts with a system or application.

[1225] An "engineer" is someone who has specific skills or expertise and works in a factory or on-site.

[1226] "Workers" are workers who perform manual labor or operate machinery in factories or on-site.

[1227] A "procedure" is a document that describes the steps and methods for performing a specific task or operation.

[1228] A "manual" is a document that contains instructions or guidance for a particular operation or procedure.

[1229] "Training" refers to educational activities aimed at improving the skills and knowledge of engineers and workers.

[1230] An "understanding report" is a report summarizing the results of an evaluation of the level of understanding and proficiency of a learner or worker.

[1231] A "factory robot" is a machine or device used to automate work within a factory.

[1232] The present invention relates to a system that automatically generates quizzes from learning materials and evaluates the learner's level of understanding based on the quizzes. This system is installed on factory robots and is effectively used for training engineers and workers.

[1233] The server first receives the learning materials uploaded by the user. These learning materials, such as learning procedures or manuals, are uploaded in PDF format. The PDF file is converted to text using a Python library called pdfplumber. The server then passes this converted text to the generative AI model.

[1234] The generative AI model uses generative AI technology to analyze the teaching materials. This analysis extracts key points and keywords. For example,

[1235] This information includes "What is a variable?", "What type of loop?", etc. In this step, the OpenAI API is used. An example of a prompt sentence output by the generative AI model is as follows:

[1236] Based on the text below, please extract the key points:

[1237] A variable is a symbol that stores specific data. For example, in Python, you can define it as x = 10. There are two types of loops: for loops and while loops. A for loop is used to repeat a fixed number of times, while a while loop repeats as long as a condition is met.

[1238] Based on the analysis results, the server again uses the OpenAI API to automatically generate quizzes. The quizzes generated range from multiple choice questions, true / false questions, and essay questions. For example, the following quiz may be generated:

[1239] Multiple choice question: "What is the definition of a variable?"

[1240] A. Memory area

[1241] B. Temporary storage of data

[1242] C. Graphical User Interface

[1243] True or False: "The for loop is used to repeat a fixed number of times. True or False?"

[1244] Short answer question: "Explain the basic usage of a while loop."

[1245] After the quiz is created, the user can view and edit the content through a dedicated user interface. The quiz is then distributed to the target learners via email or other means, with an access link to the quiz.

[1246] After a learner answers the quiz, their answers are sent to the server and graded by an automatic scoring system. Based on the results, a generative AI model analyzes each learner's level of understanding and generates a comprehension report. This report is provided to the user, who can then provide feedback based on it.

[1247] In one embodiment of the present invention, when used for training engineers and workers in a factory, for example, manuals on how to operate new machinery or safety procedures can be uploaded and analyzed, and quizzes can be generated, allowing for efficient assessment of the level of understanding of specific work tasks and enhancing the effectiveness of training.

[1248] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1249] Step 1:

[1250] The server receives the teaching material files uploaded by the user. Specifically, when the user uploads the teaching material in PDF format, the server saves it and displays a notification to the user that the upload is complete. The input is a PDF file, and the output is PDF data saved on the server.

[1251] Step 2:

[1252] The server uses the pdfplumber library to convert the uploaded PDF file to text data. Specifically, it uses pdfplumber to extract text from each page of the PDF and combine them into a single text data. The input is the saved PDF file, and the output is the extracted text data.

[1253] Step 3:

[1254] The server passes the extracted text data to the generative AI model for analysis. Specifically, it uses the OpenAI API to send the text data to the generative AI model as a prompt sentence, and extracts key points and keywords. The input is the text data, and the output is the analysis results (keywords and points) generated by the AI ​​model.

[1255] Step 4:

[1256] The server then uses the generative AI model again based on the analysis results to automatically generate quizzes. Specifically, the analysis results are used as prompts to generate multiple choice, true / false, and written questions via the OpenAI API. The input is the analysis results, and the output is the generated quiz questions.

[1257] Step 5:

[1258] The server provides the generated quiz questions to the user through a user interface. Specifically, the server displays the generated quiz content on the user interface for the user to review and edit. The input is the generated quiz questions, and the output is the quiz content displayed to the user.

[1259] Step 6:

[1260] The user can view and edit the quiz content using the provided user interface. Specifically, the user can modify the question text or add options. The input is the displayed quiz content, and the output is the final quiz content that the user has viewed and edited.

[1261] Step 7:

[1262] The server distributes the quiz that the user has confirmed to the target learners. Specifically, based on the learner list, it sends an email containing an access link to the quiz to each learner. The input is the confirmed quiz content and the learner list, and the output is the quiz link sent to the learner.

[1263] Step 8:

[1264] The learner uses their own device to access the quiz from the link they received and answers the questions. Specifically, the learner answers the quiz and sends the results to the server. The input is the learner's answer, and the output is the answer data sent to the server.

[1265] Step 9:

[1266] The server automatically grades the collected answer data and tally up the scores of each learner. Specifically, the server's automatic grading system grades the answers and calculates the scores. The input is the answer data, and the output is the graded results.

[1267] Step 10:

[1268] The server uses a generative AI model based on the scoring results to create a comprehension report and provides it to the user. Specifically, it evaluates each learner's level of comprehension based on the analyzed answers and scores, and compiles the results in a report format. The input is the scoring results, and the output is a comprehension report.

[1269] These are the specific processing steps of this system. By processing and calculating data at each step, it is possible to efficiently generate quizzes and evaluate the learner's level of understanding.

[1270] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1271] The present invention relates to a system that automatically generates quizzes from learning materials, evaluates the learner's level of understanding, and simultaneously recognizes the user's emotional state and reflects it in the system. The following is a detailed description of this system.

[1272] Program processing explanation

[1273] 1. Uploading teaching materials

[1274] The user logs in to the learning management system using a device. They enter their username and password, and if authentication is successful, they are able to access the administration screen.

[1275] When a user clicks the "Upload teaching materials" button on the administration screen, a file selection dialog box is displayed.

[1276] The user selects the teaching material file (e.g., a PDF file) to upload and clicks the "Open" button.

[1277] The device will send the selected teaching material file to the server. A message will appear indicating that the file has been successfully uploaded.

[1278] 2. Teaching material analysis

[1279] The server saves the uploaded learning material files in the specified storage area.

[1280] The server passes the saved teaching material files to the generative AI model.

[1281] The generative AI model analyzes the content of the learning material and extracts key points and keywords, such as variables, loops, and conditional branches.

[1282] 3. User Emotion Recognition

[1283] The device uses its built-in camera and microphone to capture the user's emotional data.

[1284] The server utilizes an emotion engine to analyze the acquired emotion data and determine the user's emotional state (eg, stress, interest, concentration, etc.).

[1285] 4. Quiz Generation

[1286] The server issues commands to the generative AI model based on the characteristics of the learning material and the user's emotional state that have been analyzed in advance, and automatically generates quizzes. Specifically, multiple-choice questions, true / false questions, and essay-style questions are generated based on key points.

[1287] The content and difficulty of the generated quiz are adjusted based on the user's emotional state. For example, if the user is feeling stressed, questions with lower difficulty will be given priority.

[1288] 5. Quiz confirmation

[1289] The server temporarily stores the generated quiz and provides an interface for displaying it to the user.

[1290] The user can check the generated quiz from the management screen, specifically, the question text, options, correct answers, etc.

[1291] Users can edit the content of each quiz, for example, by correcting incorrect answers or adding additional options.

[1292] When the user has finished editing, he clicks the "Confirm Quiz" button and the confirmed quiz is sent to the server.

[1293] 6. Quiz Streaming

[1294] The server acquires the learner list and generates an email for delivering the confirmed quiz to each learner.

[1295] The server sends the learner an email containing an access link to the quiz.

[1296] 7. Answer collection and analysis

[1297] Learners receive the email on their device and click on the link to access the quiz screen.

[1298] The learner answers the quiz and submits the answers, which are automatically sent to the server.

[1299] The server receives and stores the learner's answers.

[1300] The server uses an automated scoring system to score each answer and tally the scores.

[1301] Based on the scoring results, the generative AI model analyzes each learner's level of understanding.

[1302] 8. Report Generation

[1303] The server generates a comprehension report from the analysis results and provides it to the user.

[1304] The user checks the report and provides feedback to the learner if necessary.

[1305] Specific examples

[1306] For example, if a learning material for learning "basic programming concepts" is uploaded, the generative AI model analyzes the material and identifies key points such as "variable definitions," "types of loops," and "conditional branching." Furthermore, if the user's emotion engine determines that their "stress level is high," the difficulty level of the generated quiz will be set low:

[1307] Multiple choice question: "What is the definition of a variable?"

[1308] A. Memory area

[1309] B. Temporary storage of data

[1310] C. Graphical User Interface

[1311] True or False: "The for loop is used to repeat a fixed number of times. True or False?"

[1312] Short answer question: "Explain the basic usage of a while loop."

[1313] By distributing these quizzes to learners, collecting and analyzing their answers, and generating comprehension reports, we can efficiently and effectively manage learning and provide an optimal learning experience according to the user's emotional state.

[1314] This concludes the detailed description of one embodiment of the present invention. This system facilitates the automatic generation of quizzes and the measurement of comprehension, and is expected to provide an optimal learning environment for both users and learners.

[1315] The processing flow will be explained below.

[1316] Step 1:

[1317] The user logs in to the learning management system using a device. They enter their username and password, and if authentication is successful, they are able to access the administration screen.

[1318] Step 2:

[1319] The user clicks the "Upload teaching materials" button on the administration screen, which displays a file selection dialog.

[1320] Step 3:

[1321] The user selects the teaching material file (e.g., a PDF file) to upload and clicks the "Open" button.

[1322] Step 4:

[1323] The device will send the selected teaching material file to the server. A message will appear indicating that the file has been successfully uploaded.

[1324] Step 5:

[1325] The server saves the uploaded learning material files in the specified storage area.

[1326] Step 6:

[1327] The server passes the saved teaching material files to the generative AI model.

[1328] Step 7:

[1329] The generative AI model analyzes the content of the learning material and extracts key points and keywords, such as variables, loops, and conditional branching.

[1330] Step 8:

[1331] The device uses its built-in camera and microphone to capture the user's emotional data.

[1332] Step 9:

[1333] The server utilizes an emotion engine to analyze the acquired emotion data and determine the user's emotional state (eg, stress, interest, concentration, etc.).

[1334] Step 10:

[1335] The server issues commands to the generative AI model based on the characteristics of the learning material and the user's emotional state that have been analyzed in advance, and automatically generates quizzes. Specifically, multiple-choice questions, true / false questions, and essay-style questions are generated based on key points.

[1336] Step 11:

[1337] The content and difficulty of the generated quiz are adjusted based on the user's emotional state. For example, if the user is feeling stressed, questions with lower difficulty will be given priority.

[1338] Step 12:

[1339] The server temporarily stores the generated quiz and provides an interface for displaying it to the user.

[1340] Step 13:

[1341] The user can check the generated quiz from the management screen, specifically, the question text, options, correct answers, etc.

[1342] Step 14:

[1343] Users can edit the content of each quiz, for example, by correcting incorrect answers or adding additional options.

[1344] Step 15:

[1345] When the user has finished editing, they click the "Confirm Quiz" button, and the confirmed quiz is sent to the server.

[1346] Step 16:

[1347] The server acquires the learner list and generates an email for delivering the confirmed quiz to each learner.

[1348] Step 17:

[1349] The server sends the learner an email containing an access link to the quiz.

[1350] Step 18:

[1351] Learners receive the email on their device and click on the link to access the quiz screen.

[1352] Step 19:

[1353] The learner answers the quiz and submits the answers, which are automatically sent to the server.

[1354] Step 20:

[1355] The server receives and stores the learner's answers.

[1356] Step 21:

[1357] The server runs an automated scoring system to score each answer, with the scoring criteria set in advance.

[1358] Step 22:

[1359] The server compiles the scoring results and analyzes the level of understanding, such as calculating each student's score and percentage of correct answers.

[1360] Step 23:

[1361] The server generates a comprehension report based on the analysis results, which includes each learner's grades and learning progress.

[1362] Step 24:

[1363] The server provides the generated comprehension report to the user and makes it viewable on the management screen.

[1364] Step 25:

[1365] The user can review the comprehension report and provide feedback to the learner as needed, for example by providing explanations for specific questions or additional practice questions.

[1366] Example 2

[1367] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1368] While remote learning and online education have become increasingly important in recent years, accurately assessing learners' comprehension and providing an optimal learning experience that takes into account their emotional state remains a challenging task. Traditional systems require manual analysis of learning materials and quiz generation, placing a heavy burden on human resources and making them inefficient. Furthermore, they lack the ability to reflect learners' emotional state in real time and maintain their motivation.

[1369] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for inputting learning materials, means for extracting features using a generation AI model that analyzes the input learning materials, means for automatically generating quizzes based on the extracted features, means for providing a user interface for confirming and editing the generated quizzes, means for delivering the confirmed and edited quizzes to target learners, means for collecting and analyzing quiz answers from target learners, means for evaluating the target learners' understanding levels based on the collected data and generating comprehension reports based on the analysis results, and means for acquiring user emotional data from the terminal, analyzing the emotional data, and reflecting the emotional data in quiz generation. This automates everything from learning material analysis to quiz generation, analysis of learners' answers, and even providing a learning experience that takes emotional states into account, enabling efficient and effective learning management.

[1370] "Instructional materials" refer to materials containing data and information used by learners for learning purposes.

[1371] A "generative AI model" is a program or algorithm that uses artificial intelligence to automatically perform specific tasks.

[1372] A "quiz" is a set of questions given to assess a learner's understanding.

[1373] A "user interface" is a part that provides a screen and operating means for a user to interact with a system.

[1374] A "server" is a computer that processes and manages data on a network.

[1375] "Terminal" means the device through which a user accesses the system.

[1376] "Emotion data" is information that represents the user's emotional state, and is data acquired through a camera or microphone.

[1377] "Analysis" refers to processing data or information to extract specific patterns or features.

[1378] "Comprehension report" refers to the evaluation results that show how well a learner has understood the learning content.

[1379] A "prompt" is a sentence that describes instructions or questions for a generative AI model.

[1380] MODE FOR CARRYING OUT THE INVENTION

[1381] The present invention relates to a system that automatically generates quizzes from learning materials, evaluates the learner's level of understanding, and simultaneously recognizes the user's emotional state and reflects it in the system. The following is a detailed description of this system.

[1382] First, the user logs in to the learning management system using a terminal. To log in, they enter their username and password, and if authentication is successful, they can access the administration screen. Next, the user clicks the "Upload Learning Materials" button on the administration screen and selects the learning material file (e.g., a PDF file) to upload from the file selection dialog. The terminal sends the selected learning material file to the server and displays a message indicating that the file has been successfully uploaded.

[1383] The server saves the uploaded learning material files in a designated storage area (e.g., AWS S3). The server then passes the saved learning material files to a generative AI model (e.g., OpenAI GPT-4), which analyzes the learning material content and extracts key points and keywords, such as "variables," "loops," and "conditional branching."

[1384] Meanwhile, to understand the user's emotional state, the device uses a built-in camera and microphone to capture the user's emotional data (e.g., facial expressions and voice tone). This emotional data is sent to the server in real time, and the server uses an emotion engine (e.g., Amazon Rekognition, Microsoft Azure Face API) to analyze the emotional state (e.g., stress, interest, concentration level, etc.). The analysis results are reflected in the generation of the next quiz.

[1385] The server issues commands to the generative AI model based on the analyzed characteristics of the learning material and the user's emotional state to automatically generate quizzes. Specifically, multiple choice questions, true / false questions, and written questions are generated based on important points. The difficulty and content of the quiz are adjusted according to the user's emotional state. For example, if the user is feeling stressed, easier questions will be given priority.

[1386] The generated quiz is temporarily saved and can be viewed on a management screen for display to the user. From this management screen, the user can check the questions and options of the generated quiz and edit them as necessary. Once editing is complete, the user clicks the "Confirm Quiz" button and the confirmed quiz is sent to the server.

[1387] Next, the server retrieves the list of learners and generates emails to deliver the finalized quiz to each learner. This is done using an email sending API (e.g., SendGrid, Amazon SES). Learners receive the email on their devices and click a link to access the quiz screen.

[1388] When a learner answers a quiz and submits their answers, the answer data is automatically sent to the server. The server receives this answer data and uses an automatic scoring system (e.g., a Google Sheets script or a Microsoft Excel macro) to score each answer and tally the scores. Finally, the server uses a generative AI model based on the tally to analyze each learner's level of comprehension and generate a comprehension report. This report is provided to the user, who can then provide the necessary feedback to the learner.

[1389] For example, if learning materials for learning "basic programming concepts" are uploaded, the generative AI model will analyze the materials and identify key points such as "variable definitions," "types of loops," and "conditional branching." Furthermore, if the emotion engine determines that the "stress level is high," the difficulty level of the generated quiz will be set low.

[1390] Below are some example prompts to input to a generative AI model:

[1391] "Analyze the content of the following learning materials and automatically generate quizzes for learners. The types of quizzes should be multiple choice, true / false, and written. Also, if the user's stress level is high, adjust the difficulty level to a lower level."

[1392] This concludes the detailed description of one embodiment of the present invention. This system makes it easy to automatically generate quizzes and measure comprehension levels, and is expected to provide an optimal learning environment for both users and learners.

[1393] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1394] Step 1:

[1395] A user logs in to a learning management system using a terminal. The user enters a username and password on the login screen, and the terminal sends this to the server. The server performs authentication, and if successful, provides the user with an administration screen. The input is the username and password, and the output is the authentication result. If the authentication result is successful, the administration screen is displayed.

[1396] Step 2:

[1397] The user clicks the "Upload teaching materials" button on the administration screen. The terminal displays a file selection dialog, and the user selects the teaching materials file (e.g., PDF file) to upload. When the "Open" button is clicked, the terminal sends the selected file to the server. The input is the teaching materials file, and the output is a message indicating that the upload has been completed successfully. A message is displayed indicating that the file has been uploaded successfully.

[1398] Step 3:

[1399] The server saves the uploaded learning material file in a storage area (e.g. AWS S3). The input is the uploaded learning material file, and the output is the result of saving it to storage. If saving is successful, the location where the file is saved is recorded.

[1400] Step 4:

[1401] The server passes the saved teaching material file to the generative AI model. It generates a prompt for the generative AI model and analyzes the content of the teaching material. The input is the saved teaching material file and the prompt, and the output is the main points and keywords of the teaching material. The generative AI model extracts key points such as "variables," "loops," and "conditional branching" from the content.

[1402] Step 5:

[1403] The device acquires the user's emotional data using the built-in camera and microphone. The data is sent to the server in real time. The input is the user's video and audio, and the output is emotional data. The acquired data is sent to the server.

[1404] Step 6:

[1405] The server uses an emotion engine to analyze the emotion data. The input is the transmitted emotion data, and the output is the judgment result of the emotion state (e.g., stress, interest, concentration). The emotion engine judges the emotion state and saves the result.

[1406] Step 7:

[1407] The server issues commands to the generative AI model based on the analyzed characteristics of the learning material and the user's emotional state to automatically generate a quiz. The prompt text reflects important points and the user's emotional state. The input is the characteristics of the learning material and the user's emotional state, and the output is the generated quiz (multiple choice questions, true / false questions, and written questions). The difficulty and content of the quiz are adjusted.

[1408] Step 8:

[1409] The server temporarily stores the generated quiz and provides an interface for user confirmation. A list of quizzes is displayed on the management screen. The input is the generated quiz, and the output is the interface for the management screen. The user can check the question text and options.

[1410] Step 9:

[1411] The user checks and edits the generated quiz on the management screen. Once the edits are complete, the user clicks the "Confirm Quiz" button. The input is the user's edits, and the output is the confirmed quiz data. Clicking the Confirm button sends the final data to the server.

[1412] Step 10:

[1413] The server retrieves the learner list and generates emails to distribute the confirmed quiz to each learner. It uses an email sending API. The input is the learner list and confirmed quiz data, and the output is email. An email is sent to each learner.

[1414] Step 11:

[1415] The learner receives the email using their device and clicks on the quiz link. They access the quiz screen, enter their answer, and submit it. The input is the learner's answer, and the output is the result of sending it to the server. The learner's answer is sent to the server.

[1416] Step 12:

[1417] The server receives the learner's answers and uses an automatic scoring system to tally up the scores. The input is the learner's answers and the output is the scoring results. The answers are scored by the automatic scoring system.

[1418] Step 13:

[1419] The server aggregates the results and the generative AI model analyzes each learner's level of understanding. The input is the scoring result, and the output is the comprehension analysis result. The generative AI model evaluates the level of understanding.

[1420] Step 14:

[1421] The server generates a comprehension report from the comprehension analysis results and provides it to the user. The input is the comprehension analysis results and the output is the comprehension report. The user can check the report and provide feedback.

[1422] (Application example 2)

[1423] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1424] Conventional online learning systems struggle to provide a personalized learning experience for each learner, resulting in insufficient learning effectiveness. Furthermore, they lack a means to provide learning content that takes into account the learner's emotional state, often resulting in learners feeling stressed or losing concentration. To solve these problems, a system is needed that recognizes the learner's emotional state in real time and adjusts the difficulty and content of the learning content accordingly.

[1425] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1426] In this invention, the server includes means for inputting learning materials, means for extracting features using a generative AI model that analyzes the input learning materials, means for automatically generating quizzes based on the extracted features, means for recognizing the emotional state of a user while watching, means for adjusting the difficulty of the quiz based on the recognized emotional state, means for providing a user interface for reviewing and editing the generated quiz, means for delivering the reviewed and edited quiz to a target learner, means for collecting and analyzing the quiz answers of the target learner, and means for generating a comprehension report based on the analysis results, thereby enabling the provision of personalized learning content that takes into account the learner's emotional state.

[1427] "Means for inputting teaching materials" refers to a function for uploading educational content (videos, PDFs, text, etc.) to a server from the device used by the learner.

[1428] A "generative AI model" is an artificial intelligence algorithm that analyzes the content of input teaching materials, extracts key points and keywords, and is used to automatically generate quizzes.

[1429] "Means for extracting features" refers to a function that uses a generative AI model to identify key points and important keywords from input learning materials.

[1430] The "means for automatically generating quizzes" is a function that automatically creates quizzes in various formats (multiple choice questions, true / false questions, written questions, etc.) based on the extracted features.

[1431] "Emotion recognition means" is a function that uses technology (such as facial recognition and voice analysis) to recognize the user's emotional state in real time while watching.

[1432] The "means for adjusting the difficulty of the quiz" is a function that dynamically changes the difficulty and content of the generated quiz based on the recognized emotional state of the user.

[1433] The "means for providing a user interface" is a function that provides a screen and operation means for the user to check and edit the generated quiz.

[1434] "Means for delivery to target learners" refers to the functionality for electronically transmitting the reviewed and edited quiz to the target learners.

[1435] "Means for collecting and analyzing quiz answers" is a function for collecting the results of quizzes answered by target learners and analyzing them to understand the learners' level of understanding and learning situation.

[1436] The "means for generating a comprehension report" is a function that creates a report summarizing each learner's level of comprehension and learning progress information based on the quiz answer results and analysis thereof.

[1437] MODE FOR CARRYING OUT THE INVENTION

[1438] The present invention relates to a system for automatically generating quizzes from video content in an online learning platform, and providing a personalized learning experience by recognizing a learner's emotional state in real time. A specific embodiment of the system and its operation are described below.

[1439] System Program

[1440] The system of the present invention consists of the following major components:

[1441] 1. How to input materials:

[1442] This function allows users to upload educational content (videos, PDFs, text, etc.) to the server from their devices. Devices include PCs and smartphones.

[1443] 2. How to extract features using generative AI models:

[1444] The server inputs the uploaded learning materials into a generative AI model to extract key points and keywords. For example, OpenAI GPT-3 is used as the generative AI model.

[1445] 3. Emotion recognition means to recognize the user's emotional state while watching:

[1446] This function uses the device's built-in camera and microphone to capture the user's facial expressions and voice, and analyzes their emotional state in real time. The emotion analysis engine used is, for example, the Microsoft Azure Emotion Analysis API.

[1447] 4. How to adjust the difficulty of the quiz:

[1448] This function allows the server to dynamically change the difficulty and content of the quiz based on the user's emotional state. For example, if the user is feeling stressed, easier questions will be prioritized.

[1449] 5. A means to provide a user interface for reviewing and editing the quiz:

[1450] This function allows the server to display the generated quiz to the user and provides an interface for checking and editing.

[1451] 6. Means of delivery to target learners:

[1452] This function allows the server to distribute the reviewed and edited quiz to the target learners via email or other means.

[1453] 7. How quiz answers will be collected and analyzed:

[1454] This function allows the server to collect and analyze learners' answers to distributed quizzes to assess their level of understanding.

[1455] 8. How to generate a comprehension report:

[1456] This function allows the server to generate a report showing each learner's level of understanding based on the analysis results and provide this to the user.

[1457] Program processing

[1458] The server uses a generative AI model (e.g., OpenAI GPT-3) to analyze the uploaded learning material. Once the learning material is uploaded, the server inputs it into the generative AI model to extract key points and keywords. For example, if a video on "SEO Strategy" is uploaded, the generative AI model will identify key points such as "keyword research" and "content marketing."

[1459] The device uses a built-in camera and microphone to recognize the user's emotional state in real time while watching, and performs analysis using an emotion analysis engine (for example, Microsoft Azure Emotion Analysis API), which determines the user's stress level and interest level.

[1460] The server adjusts the difficulty and content of the generated quiz based on the user's perceived emotional state: for example, it presents easier questions when the user is stressed and more detailed questions when the user is focused.

[1461] The user interface presents the generated quiz to the user and provides a screen for the user to review and edit, allowing the user to modify the question text and answer options.

[1462] The server then distributes the reviewed and edited quizzes to the target learners, who then answer the quizzes. The answers are sent to the server, where they are automatically compiled and analyzed. Finally, a report showing the learners' comprehension is generated and provided to the user.

[1463] Examples and prompts

[1464] For example, if a user is watching a video on "SEO Strategies," the following quiz will be auto-generated:

[1465] Multiple Choice Question: "What are the essential elements of SEO?"

[1466] A. Keyword Research

[1467] B. Visual Design

[1468] C. Database Management

[1469] True or False: "Meta descriptions affect SEO. True or False?"

[1470] Short answer question: "Explain the importance of on-page SEO."

[1471] Example prompt sentence:

[1472] Analyze the content of the following educational material and generate quiz questions. Please take into account the user's emotional state and set the difficulty level low if the user's stress level is high.

[1473] Material Content: SEO Strategies

[1474] This makes it possible to provide learners with personalized learning content and maximize learning effectiveness.

[1475] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1476] Step 1:

[1477] Uploading teaching materials

[1478] Subject: User

[1479] Description: A user uploads educational content (e.g., video, PDF, text file) from a device to a server. Specifically, the user logs in to a learning management system using a device, selects the educational material file, and clicks the "Upload" button. The input is the file path of the target educational material, and the output is the saving of the file to the server.

[1480] Step 2:

[1481] Teaching material analysis

[1482] Subject: Server

[1483] Description: The server inputs the uploaded learning material file into a generative AI model to analyze the content of the learning material. For example, the server uses a generative AI model (OpenAI GPT-3) to extract key points and keywords within the learning material. The input is the uploaded learning material file, and the output is a list of key points and keywords. Data processing includes text extraction and key point detection.

[1484] Step 3:

[1485] emotion recognition

[1486] Subject: Device

[1487] Description: The device uses the built-in camera and microphone to capture the user's facial expressions and voice. This data is sent in real time to an emotion analysis engine (Microsoft Azure Emotion Analysis API) to analyze the user's emotional state. The input is facial expression data and voice data, and the output is a judgment of the user's emotional state (e.g., stress level, interest level, etc.). Operations include facial recognition using the camera and voice analysis using the microphone.

[1488] Step 4:

[1489] Quiz Generation

[1490] Subject: Server

[1491] Description: The server issues commands to a generative AI model based on the extracted key points and emotional state data to automatically generate quizzes. The generated quizzes include a variety of formats, such as multiple choice, true / false, and written questions. The input is a list of key points and emotional state data, and the output is a set of generated quiz questions. Data calculation involves automatically generating quizzes using a generative AI model.

[1492] Step 5:

[1493] Quiz difficulty adjustment

[1494] Subject: Server

[1495] Description: The server dynamically adjusts the difficulty of the generated quiz based on emotional state data. For example, if the user is feeling stressed, easier questions will be prioritized. The input is the emotional state data and the generated quiz, and the output is the quiz questions with the adjusted difficulty. Actions include changing the question text and the format of the options.

[1496] Step 6:

[1497] Check and edit quiz

[1498] Subject: Server

[1499] Description: The server temporarily stores generated quizzes and provides an interface for users to review and edit them. Users can review quizzes from an admin screen and make corrections as needed. The input is the generated quiz set, and the output is the final quiz set edited by the user. Behavior includes providing a user interface and supporting quiz editing.

[1500] Step 7:

[1501] Quiz broadcast

[1502] Subject: Server

[1503] Description: The server generates and sends emails to distribute edited quizzes to target learners. The input is the final quiz set and a list of learners, and the output is an email containing an access link to the quiz. Actions include generating and sending emails.

[1504] Step 8:

[1505] Answer collection and analysis

[1506] Subject: Server

[1507] Description: The target learner answers a quiz on their device and sends the answers to the server. The server collects the answers and performs automatic scoring and comprehension analysis. The input is the learner's quiz answers, and the output is the scoring results and comprehension analysis results. Data calculations include comprehension analysis using an automatic scoring algorithm and a generative AI model.

[1508] Step 9:

[1509] Report Generation

[1510] Subject: Server

[1511] Description: The server generates a comprehension report for each learner based on the analysis results and provides it to the user. The input is the analysis result data, and the output is the comprehension report. The operations include generating the report and providing it to the user.

[1512] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1513] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1514] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1515] [Fourth embodiment]

[1516] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1517] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1518] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1519] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1520] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1521] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1522] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1523] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1524] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1525] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1526] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1527] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1528] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1529] This invention relates to a system that automatically generates quizzes from learning materials and evaluates learners' understanding based on the quizzes. This system efficiently performs a series of processes: inputting learning materials, analyzing them, generating quizzes, and distributing them to learners. The following is a detailed description of this system.

[1530] Program processing explanation

[1531] 1. Uploading teaching materials

[1532] The user uses the terminal to upload learning material files to the server, for example, PDF format learning materials, and sends them to the system via a designated upload button.

[1533] The server saves the received file and displays a notification to the user that the upload is complete.

[1534] 2. Teaching material analysis

[1535] The server passes the saved teaching material files to the generative AI model.

[1536] The generative AI model analyzes the content of the teaching materials and extracts key points and keywords, such as "What is a variable?" and "What are the types of loops?"

[1537] 3. Quiz Generation

[1538] Based on the analysis results, the server issues commands to the generative AI model to automatically generate quizzes, such as multiple choice questions, true / false questions, and essay questions based on key points.

[1539] The created quiz is stored on the server and displayed to the user for prior review.

[1540] 4. Quiz confirmation

[1541] The user can use the device to check the generated quiz content and edit it as needed, for example, by modifying the question text or adding options.

[1542] Once editing is complete, the user clicks the "Confirm Quiz" button to send the finalized quiz to the server.

[1543] 5. Quiz distribution

[1544] The server checks the list of learners to distribute the confirmed quiz to the target learners, and based on this list, sends each learner an email containing an access link to the quiz.

[1545] Learners click on a link from their device to access and answer the quiz.

[1546] 6. Answer collection and analysis

[1547] Once the learner has completed the quiz, the answers are sent to the server and stored.

[1548] The server uses an automated scoring system to score each answer and tally the scores.

[1549] Based on the scoring results, the generative AI model analyzes each learner's level of understanding.

[1550] 7. Report Generation

[1551] The server generates a comprehension report from the analysis results and provides it to the user.

[1552] The user checks the report and provides feedback to the learner if necessary.

[1553] Specific examples

[1554] For example, if a learning material for learning "basic programming concepts" is uploaded, a generative AI model might analyze the material and identify key points such as:

[1555] Defining variables

[1556] Loop Types

[1557] Conditional Branching

[1558] Based on this, the following quiz is generated:

[1559] Multiple choice question: "What is the definition of a variable?"

[1560] A. Memory area

[1561] B. Temporary storage of data

[1562] C. Graphical User Interface

[1563] True or False: "The for loop is used to repeat a fixed number of times. True or False?"

[1564] Short answer question: "Explain the basic usage of a while loop."

[1565] By distributing these quizzes to learners, collecting and analyzing their answers, and generating comprehension reports, efficient learning management becomes possible.

[1566] This concludes the detailed description of one embodiment of the present invention. This system makes it easy to automatically generate quizzes and measure comprehension levels, and is expected to improve the learning efficiency of learners.

[1567] The processing flow will be explained below.

[1568] Step 1:

[1569] The user logs in to the learning management system using a device. They enter their username and password, and if authentication is successful, they are able to access the administration screen.

[1570] Step 2:

[1571] The user clicks the "Upload teaching materials" button on the administration screen, which displays a file selection dialog.

[1572] Step 3:

[1573] The user selects the teaching material file (e.g., a PDF file) to upload and clicks the "Open" button.

[1574] Step 4:

[1575] The device will send the selected teaching material file to the server. A message will appear indicating that the file has been successfully uploaded.

[1576] Step 5:

[1577] The server saves the uploaded learning material files in the specified storage area.

[1578] Step 6:

[1579] The server passes the saved teaching material files to the generative AI model.

[1580] Step 7:

[1581] The generative AI model analyzes the content of the learning material and extracts key points and keywords, such as variables, loops, and conditional branching.

[1582] Step 8:

[1583] The generative AI model returns the analysis results to the server, which include key points and keywords from the learning material.

[1584] Step 9:

[1585] The server generates quiz templates based on the analysis results, which can include multiple choice questions, true / false questions, and essay questions.

[1586] Step 10:

[1587] A generative AI model automatically generates specific quizzes based on templates, including multiple-choice questions such as "What is the definition of a variable?"

[1588] Step 11:

[1589] The server temporarily stores the generated quiz and provides an interface for displaying it to the user.

[1590] Step 12:

[1591] The user can check the generated quiz from the management screen, specifically, the question text, options, correct answers, etc.

[1592] Step 13:

[1593] Users can edit the content of each quiz, for example, by correcting incorrect answers or adding additional options.

[1594] Step 14:

[1595] When the user has finished editing, they click the "Confirm Quiz" button, and the confirmed quiz is sent to the server.

[1596] Step 15:

[1597] The server acquires the learner list and generates an email for delivering the confirmed quiz to each learner.

[1598] Step 16:

[1599] The server sends the learner an email containing an access link to the quiz.

[1600] Step 17:

[1601] Learners receive the email on their device and click on the link to access the quiz screen.

[1602] Step 18:

[1603] The learner answers the quiz and submits the answers, which are automatically sent to the server.

[1604] Step 19:

[1605] The server receives and stores the learner's answers.

[1606] Step 20:

[1607] The server runs an automated scoring system to score each answer, with the scoring criteria set in advance.

[1608] Step 21:

[1609] The server compiles the scoring results and analyzes the level of understanding, such as calculating each student's score and percentage of correct answers.

[1610] Step 22:

[1611] The server generates a comprehension report based on the analysis results, which includes each learner's grades and learning progress.

[1612] Step 23:

[1613] The server provides the generated comprehension report to the user, who can view the report on the management screen.

[1614] Step 24:

[1615] The user can review the comprehension report and provide feedback to the learner as needed, for example by providing explanations for specific questions or additional practice questions.

[1616] Example 1

[1617] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1618] Conventional learning systems have struggled to efficiently generate quizzes from learning materials and accurately assess learners' comprehension. Furthermore, the processes of quiz generation, confirmation, editing, distribution, answer collection, and analysis all require manual work, which is time-consuming and labor-intensive. Furthermore, the quality of the generated quizzes and the reliability of the assessment of learners' comprehension have also been issues.

[1619] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1620] In this invention, the server includes: means for a user to input learning materials; means for extracting key points using a generative AI model that analyzes the input learning materials; means for automatically generating quizzes based on the extracted key points; means for providing a user interface for reviewing and editing the generated quizzes; means for delivering the reviewed and edited quizzes to target learners; means for collecting and scoring quiz answers from target learners; and means for analyzing the scoring results and generating a comprehension report. This makes it possible to efficiently generate quizzes from learning materials and accurately evaluate learners' comprehension.

[1621] "User" refers to the entity that operates the system, uploads learning materials, and checks and edits quizzes.

[1622] "Learning materials" refers to files or data containing information or content that a learner uses to study.

[1623] A "generative AI model" is an artificial intelligence model that analyzes input learning materials, extracts key points, and generates quizzes.

[1624] "Key points" refer to the particularly important concepts and information in the learning material that learners should understand.

[1625] A "quiz" is a problem or question posed to assess a learner's level of understanding.

[1626] A "user interface" refers to the screens and interaction means that allow a user to interact with and operate a system.

[1627] "Delivery" refers to the process of sending the generated quiz or information to the intended recipient.

[1628] A "learner" is an individual who answers quizzes and learns the learning material.

[1629] "Scoring" refers to the process of evaluating and scoring quizzes completed by learners.

[1630] A "comprehension report" is a report that shows the learner's level of understanding based on the scoring results.

[1631] This invention relates to a system that automatically generates quizzes from learning materials and evaluates the learner's level of understanding based on the quizzes. This system efficiently performs a series of processes: inputting learning materials, analyzing them, generating quizzes, and distributing them to learners.

[1632] The system is implemented using the following hardware and software.

[1633] Hardware: Server, user device, learner device

[1634] Software: Generative AI models (e.g., GPT-4), automated scoring systems, email distribution systems

[1635] Program processing explanation

[1636] Uploading teaching materials

[1637] The user uploads the learning material file to the server using his / her own terminal. For example, the user sends the learning material file in PDF format to the system via a designated upload button.

[1638] The server saves the received file and displays a notification to the user that the upload is complete.

[1639] Teaching material analysis

[1640] The server passes the saved teaching material file to the generative AI model and begins analysis.

[1641] The generative AI model analyzes the content of the teaching materials and extracts key points and keywords, such as "What is a variable?" and "What are the types of loops?"

[1642] Quiz Generation

[1643] The server issues commands to the generative AI model based on the analysis results to automatically generate quizzes, such as multiple choice questions, true / false questions, and essay questions based on key points.

[1644] The created quiz is stored on the server and displayed to the user for prior review.

[1645] Quiz Check

[1646] The user can use the device to check the generated quiz content and edit it as needed, for example by modifying the question text or adding options.

[1647] Once editing is complete, the user clicks the "Confirm Quiz" button to send the finalized quiz to the server.

[1648] Quiz broadcast

[1649] The server checks the list of learners to distribute the confirmed quiz to the target learners, and based on this list, sends each learner an email containing an access link to the quiz.

[1650] Learners click on a link from their device to access the quiz and answer the questions.

[1651] Answer collection and analysis

[1652] Once the learner has completed the quiz, the answers are sent to the server and stored.

[1653] The server uses an automated scoring system to score each answer and tally the scores.

[1654] Based on the scoring results, the generative AI model analyzes each learner's level of understanding.

[1655] Report Generation

[1656] The server generates a comprehension report from the analysis results and provides it to the user.

[1657] The user checks the report and provides feedback to the learner if necessary.

[1658] Specific examples

[1659] For example, if a learning material for learning "basic programming concepts" is uploaded, a generative AI model might analyze the material and identify key points such as:

[1660] Defining variables

[1661] Loop Types

[1662] Conditional Branching

[1663] Based on this, the following quiz is generated:

[1664] Multiple choice question: "What is the definition of a variable?"

[1665] A. Memory area

[1666] B. Temporary storage of data

[1667] C. Graphical User Interface

[1668] True or False: "The for loop is used to repeat a fixed number of times. True or False?"

[1669] Short answer question: "Explain the basic usage of a while loop."

[1670] Example prompts to input to the generative AI model

[1671] "Please analyze the following PDF file and extract the main points and keywords."

[1672] "Based on the analysis results below, please generate a quiz in the form of multiple choice, true / false, or written questions."

[1673] "Collect answers corresponding to the next quiz and analyze learners' understanding."

[1674] This completes the detailed description of one embodiment of the present invention. This system makes it easy to automatically generate quizzes and measure comprehension levels, and is expected to improve the learning efficiency of learners.

[1675] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1676] Step 1: Upload your materials

[1677] Users use their own devices to select learning material files, such as PDF files, and click the designated upload button to send them to the server.

[1678] Input: Learning material file selected by the user

[1679] Output: Teaching material files saved on the server

[1680] Specific operation: The user accesses the "Upload learning material" screen and clicks the "Select file" button to select the learning material file. When the user clicks the "Upload" button, the file is sent to the server, and the server displays the message "Learning material upload completed" on the user's device.

[1681] Step 2: Analysis of teaching materials

[1682] The server passes the saved teaching material file to the generative AI model and begins analysis.

[1683] Input: Teaching material file saved on the server

[1684] Output: Key points and keywords extracted by the generative AI model

[1685] Specific operation: The server creates a prompt to input the learning material file into the generative AI model (e.g., "Please extract the key points about the programming concept."). The generative AI model analyzes the learning material file and extracts important information such as "variable definitions," "loop types," and "conditional branching."

[1686] Step 3: Quiz Generation

[1687] Based on the analysis results, the server issues commands to the generative AI model to automatically generate quizzes.

[1688] Input: Key points and keywords extracted by the generative AI model

[1689] Output: Generated quiz (multiple choice, true / false, short answer)

[1690] Specific operation: The server instructs the generative AI model to "create a quiz based on the following points." The generative AI model creates quizzes such as "What is the definition of a variable?" and "A for loop is used to repeat a fixed number of times. Is it true or false?" and saves them on the server.

[1691] Step 4: Quiz Review

[1692] The user uses the terminal to check the generated quiz content and edit it as necessary.

[1693] Input: Generated quiz stored on the server

[1694] Output: Quiz confirmed and edited by the user

[1695] Specific operation: The user accesses the "Quiz Confirmation" screen, reviews the generated quiz, and makes edits such as "revise the question text" and "add options." The user clicks the "Confirm Quiz" button, and the confirmed quiz is sent to the server.

[1696] Step 5: Stream the quiz

[1697] The server checks the learner list to deliver the determined quiz to the target learners.

[1698] Input: Quiz confirmed and edited by user, learner list

[1699] Output: An email containing the quiz access link delivered to the learner.

[1700] Specific operation: The server starts the "quiz distribution" process and checks the learner list. The server automatically sends an email containing an access link to the quiz to each learner. The learner clicks the link in the email and accesses the quiz.

[1701] Step 6: Collect and analyze answers

[1702] The learner answers the quiz and sends the answers to the server.

[1703] The server uses an automatic scoring system to score the answers and tally the scores.

[1704] Input: Quiz answered by learner, scoring system

[1705] Output: Scoring results, comprehension analysis results

[1706] Specific operation: The learner answers the quiz and clicks the "Submit" button. The server passes the received answers to the automatic scoring system, which generates a scoring result. The generative AI model analyzes the scoring result and analyzes the learner's level of understanding.

[1707] Step 7: Generate reports

[1708] The server generates a report from the comprehension analysis results and provides it to the user.

[1709] Input: Comprehension analysis results

[1710] Output: User-viewable comprehension report

[1711] Specific operation: The server executes the "report generation" process and creates a report based on the analysis results. The user checks the "comprehension report" and selects a means to share it with the learner (e.g., send it by email).

[1712] By using the above processing steps, the present invention makes it possible to efficiently generate quizzes from learning materials and accurately evaluate the learner's level of understanding.

[1713] (Application example 1)

[1714] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1715] Conventional systems that create quizzes based on teaching materials and assess learners' comprehension have difficulty meeting the diverse needs of learners and on-site training that requires specific expertise. It has also been difficult to generate and evaluate quizzes for efficient training of new employees and skill development. In particular, there has been a demand for quiz creation and comprehension assessment that effectively utilizes procedures and manuals in training engineers and workers in factories.

[1716] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1717] In this invention, the server includes means for inputting learning materials, means for extracting features using a generative AI model that analyzes the input learning materials, means for automatically generating quizzes based on the extracted features, means for providing a user interface for checking and editing the generated quizzes, means for distributing the checked and edited quizzes to target learners, means for collecting and analyzing the quiz answers of the target learners, means for generating a comprehension report based on the analysis results, means for uploading learning procedures and manuals and generating quizzes to evaluate the comprehension of engineers and workers, and means for use as an application installed on factory robots for training engineers and workers. This makes it possible to efficiently and accurately generate quizzes based on the content of the learning materials and evaluate the comprehension.

[1718] "Teaching materials" are information and materials used for the purpose of learner understanding.

[1719] A "generative AI model" is a model that uses artificial intelligence technology to perform text analysis and information extraction.

[1720] "Features" are important points or keywords extracted from the teaching materials.

[1721] A "quiz" is a question or problem used to assess a learner's understanding.

[1722] A "user interface" is an interface through which a user interacts with a system or application.

[1723] An "engineer" is someone who has specific skills or expertise and works in a factory or on-site.

[1724] "Workers" are workers who perform manual labor or operate machinery in factories or on-site.

[1725] A "procedure" is a document that describes the steps and methods for performing a specific task or operation.

[1726] A "manual" is a document that contains instructions or guidance for a particular operation or procedure.

[1727] "Training" refers to educational activities aimed at improving the skills and knowledge of engineers and workers.

[1728] An "understanding report" is a report summarizing the results of an evaluation of the level of understanding and proficiency of a learner or worker.

[1729] A "factory robot" is a machine or device used to automate work within a factory.

[1730] The present invention relates to a system that automatically generates quizzes from learning materials and evaluates the learner's level of understanding based on the quizzes. This system is installed on factory robots and is effectively used for training engineers and workers.

[1731] The server first receives the learning materials uploaded by the user. These learning materials, such as learning procedures or manuals, are uploaded in PDF format. The PDF file is converted to text using a Python library called pdfplumber. The server then passes this converted text to the generative AI model.

[1732] The generative AI model uses generative AI technology to analyze the teaching materials. This analysis extracts key points and keywords. For example,

[1733] This information includes "What is a variable?", "What type of loop?", etc. In this step, the OpenAI API is used. An example of a prompt sentence output by the generative AI model is as follows:

[1734] Based on the text below, please extract the key points:

[1735] A variable is a symbol that stores specific data. For example, in Python, you can define it as x = 10. There are two types of loops: for loops and while loops. A for loop is used to repeat a fixed number of times, while a while loop repeats as long as a condition is met.

[1736] Based on the analysis results, the server again uses the OpenAI API to automatically generate quizzes. The quizzes generated range from multiple choice questions, true / false questions, and essay questions. For example, the following quiz may be generated:

[1737] Multiple choice question: "What is the definition of a variable?"

[1738] A. Memory area

[1739] B. Temporary storage of data

[1740] C. Graphical User Interface

[1741] True or False: "The for loop is used to repeat a fixed number of times. True or False?"

[1742] Short answer question: "Explain the basic usage of a while loop."

[1743] After the quiz is created, the user can view and edit the content through a dedicated user interface. The quiz is then distributed to the target learners via email or other means, with an access link to the quiz.

[1744] After a learner answers the quiz, their answers are sent to the server and graded by an automatic scoring system. Based on the results, a generative AI model analyzes each learner's level of understanding and generates a comprehension report. This report is provided to the user, who can then provide feedback based on it.

[1745] In one embodiment of the present invention, when used for training engineers and workers in a factory, for example, manuals on how to operate new machinery or safety procedures can be uploaded and analyzed, and quizzes can be generated, allowing for efficient assessment of the level of understanding of specific work tasks and enhancing the effectiveness of training.

[1746] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1747] Step 1:

[1748] The server receives the teaching material files uploaded by the user. Specifically, when the user uploads the teaching material in PDF format, the server saves it and displays a notification to the user that the upload is complete. The input is a PDF file, and the output is PDF data saved on the server.

[1749] Step 2:

[1750] The server uses the pdfplumber library to convert the uploaded PDF file to text data. Specifically, it uses pdfplumber to extract text from each page of the PDF and combine them into a single text data. The input is the saved PDF file, and the output is the extracted text data.

[1751] Step 3:

[1752] The server passes the extracted text data to the generative AI model for analysis. Specifically, it uses the OpenAI API to send the text data to the generative AI model as a prompt sentence, and extracts key points and keywords. The input is the text data, and the output is the analysis results (keywords and points) generated by the AI ​​model.

[1753] Step 4:

[1754] The server then uses the generative AI model again based on the analysis results to automatically generate quizzes. Specifically, the analysis results are used as prompts to generate multiple choice, true / false, and written questions via the OpenAI API. The input is the analysis results, and the output is the generated quiz questions.

[1755] Step 5:

[1756] The server provides the generated quiz questions to the user through a user interface. Specifically, the server displays the generated quiz content on the user interface for the user to review and edit. The input is the generated quiz questions, and the output is the quiz content displayed to the user.

[1757] Step 6:

[1758] The user can view and edit the quiz content using the provided user interface. Specifically, the user can modify the question text or add options. The input is the displayed quiz content, and the output is the final quiz content that the user has viewed and edited.

[1759] Step 7:

[1760] The server distributes the quiz that the user has confirmed to the target learners. Specifically, based on the learner list, it sends an email containing an access link to the quiz to each learner. The input is the confirmed quiz content and the learner list, and the output is the quiz link sent to the learner.

[1761] Step 8:

[1762] The learner uses their own device to access the quiz from the link they received and answers the questions. Specifically, the learner answers the quiz and sends the results to the server. The input is the learner's answer, and the output is the answer data sent to the server.

[1763] Step 9:

[1764] The server automatically grades the collected answer data and tally up the scores of each learner. Specifically, the server's automatic grading system grades the answers and calculates the scores. The input is the answer data, and the output is the graded results.

[1765] Step 10:

[1766] The server uses a generative AI model based on the scoring results to create a comprehension report and provides it to the user. Specifically, it evaluates each learner's level of comprehension based on the analyzed answers and scores, and compiles the results in a report format. The input is the scoring results, and the output is a comprehension report.

[1767] These are the specific processing steps of this system. By processing and calculating data at each step, it is possible to efficiently generate quizzes and evaluate the learner's level of understanding.

[1768] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1769] The present invention relates to a system that automatically generates quizzes from learning materials, evaluates the learner's level of understanding, and simultaneously recognizes the user's emotional state and reflects it in the system. The following is a detailed description of this system.

[1770] Program processing explanation

[1771] 1. Uploading teaching materials

[1772] The user logs in to the learning management system using a device. They enter their username and password, and if authentication is successful, they are able to access the administration screen.

[1773] When a user clicks the "Upload teaching materials" button on the administration screen, a file selection dialog box is displayed.

[1774] The user selects the teaching material file (e.g., a PDF file) to upload and clicks the "Open" button.

[1775] The device will send the selected teaching material file to the server. A message will appear indicating that the file has been successfully uploaded.

[1776] 2. Teaching material analysis

[1777] The server saves the uploaded learning material files in the specified storage area.

[1778] The server passes the saved teaching material files to the generative AI model.

[1779] The generative AI model analyzes the content of the learning material and extracts key points and keywords, such as variables, loops, and conditional branches.

[1780] 3. User Emotion Recognition

[1781] The device uses its built-in camera and microphone to capture the user's emotional data.

[1782] The server utilizes an emotion engine to analyze the acquired emotion data and determine the user's emotional state (eg, stress, interest, concentration, etc.).

[1783] 4. Quiz Generation

[1784] The server issues commands to the generative AI model based on the characteristics of the learning material and the user's emotional state that have been analyzed in advance, and automatically generates quizzes. Specifically, multiple-choice questions, true / false questions, and essay-style questions are generated based on key points.

[1785] The content and difficulty of the generated quiz are adjusted based on the user's emotional state. For example, if the user is feeling stressed, questions with lower difficulty will be given priority.

[1786] 5. Quiz confirmation

[1787] The server temporarily stores the generated quiz and provides an interface for displaying it to the user.

[1788] The user can check the generated quiz from the management screen, specifically, the question text, options, correct answers, etc.

[1789] Users can edit the content of each quiz, for example, by correcting incorrect answers or adding additional options.

[1790] When the user has finished editing, he clicks the "Confirm Quiz" button and the confirmed quiz is sent to the server.

[1791] 6. Quiz Streaming

[1792] The server acquires the learner list and generates an email for delivering the confirmed quiz to each learner.

[1793] The server sends the learner an email containing an access link to the quiz.

[1794] 7. Answer collection and analysis

[1795] Learners receive the email on their device and click on the link to access the quiz screen.

[1796] The learner answers the quiz and submits the answers, which are automatically sent to the server.

[1797] The server receives and stores the learner's answers.

[1798] The server uses an automated scoring system to score each answer and tally the scores.

[1799] Based on the scoring results, the generative AI model analyzes each learner's level of understanding.

[1800] 8. Report Generation

[1801] The server generates a comprehension report from the analysis results and provides it to the user.

[1802] The user checks the report and provides feedback to the learner if necessary.

[1803] Specific examples

[1804] For example, if a learning material for learning "basic programming concepts" is uploaded, the generative AI model analyzes the material and identifies key points such as "variable definitions," "types of loops," and "conditional branching." Furthermore, if the user's emotion engine determines that their "stress level is high," the difficulty level of the generated quiz will be set low:

[1805] Multiple choice question: "What is the definition of a variable?"

[1806] A. Memory area

[1807] B. Temporary storage of data

[1808] C. Graphical User Interface

[1809] True or False: "The for loop is used to repeat a fixed number of times. True or False?"

[1810] Short answer question: "Explain the basic usage of a while loop."

[1811] By distributing these quizzes to learners, collecting and analyzing their answers, and generating comprehension reports, we can efficiently and effectively manage learning and provide an optimal learning experience according to the user's emotional state.

[1812] This concludes the detailed description of one embodiment of the present invention. This system facilitates the automatic generation of quizzes and the measurement of comprehension, and is expected to provide an optimal learning environment for both users and learners.

[1813] The processing flow will be explained below.

[1814] Step 1:

[1815] The user logs in to the learning management system using a device. They enter their username and password, and if authentication is successful, they are able to access the administration screen.

[1816] Step 2:

[1817] The user clicks the "Upload teaching materials" button on the administration screen, which displays a file selection dialog.

[1818] Step 3:

[1819] The user selects the teaching material file (e.g., a PDF file) to upload and clicks the "Open" button.

[1820] Step 4:

[1821] The device will send the selected teaching material file to the server. A message will appear indicating that the file has been successfully uploaded.

[1822] Step 5:

[1823] The server saves the uploaded learning material files in the specified storage area.

[1824] Step 6:

[1825] The server passes the saved teaching material files to the generative AI model.

[1826] Step 7:

[1827] The generative AI model analyzes the content of the learning material and extracts key points and keywords, such as variables, loops, and conditional branching.

[1828] Step 8:

[1829] The device uses its built-in camera and microphone to capture the user's emotional data.

[1830] Step 9:

[1831] The server utilizes an emotion engine to analyze the acquired emotion data and determine the user's emotional state (eg, stress, interest, concentration, etc.).

[1832] Step 10:

[1833] The server issues commands to the generative AI model based on the characteristics of the learning material and the user's emotional state that have been analyzed in advance, and automatically generates quizzes. Specifically, multiple-choice questions, true / false questions, and essay-style questions are generated based on key points.

[1834] Step 11:

[1835] The content and difficulty of the generated quiz are adjusted based on the user's emotional state. For example, if the user is feeling stressed, questions with lower difficulty will be given priority.

[1836] Step 12:

[1837] The server temporarily stores the generated quiz and provides an interface for displaying it to the user.

[1838] Step 13:

[1839] The user can check the generated quiz from the management screen, specifically, the question text, options, correct answers, etc.

[1840] Step 14:

[1841] Users can edit the content of each quiz, for example, by correcting incorrect answers or adding additional options.

[1842] Step 15:

[1843] When the user has finished editing, they click the "Confirm Quiz" button, and the confirmed quiz is sent to the server.

[1844] Step 16:

[1845] The server acquires the learner list and generates an email for delivering the confirmed quiz to each learner.

[1846] Step 17:

[1847] The server sends the learner an email containing an access link to the quiz.

[1848] Step 18:

[1849] Learners receive the email on their device and click on the link to access the quiz screen.

[1850] Step 19:

[1851] The learner answers the quiz and submits the answers, which are automatically sent to the server.

[1852] Step 20:

[1853] The server receives and stores the learner's answers.

[1854] Step 21:

[1855] The server runs an automated scoring system to score each answer, with the scoring criteria set in advance.

[1856] Step 22:

[1857] The server compiles the scoring results and analyzes the level of understanding, such as calculating each student's score and percentage of correct answers.

[1858] Step 23:

[1859] The server generates a comprehension report based on the analysis results, which includes each learner's grades and learning progress.

[1860] Step 24:

[1861] The server provides the generated comprehension report to the user and makes it viewable on the management screen.

[1862] Step 25:

[1863] The user can review the comprehension report and provide feedback to the learner as needed, for example by providing explanations for specific questions or additional practice questions.

[1864] Example 2

[1865] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1866] While remote learning and online education have become increasingly important in recent years, accurately assessing learners' comprehension and providing an optimal learning experience that takes into account their emotional state remains a challenging task. Traditional systems require manual analysis of learning materials and quiz generation, placing a heavy burden on human resources and making them inefficient. Furthermore, they lack the ability to reflect learners' emotional state in real time and maintain their motivation.

[1867] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for inputting learning materials, means for extracting features using a generation AI model that analyzes the input learning materials, means for automatically generating quizzes based on the extracted features, means for providing a user interface for confirming and editing the generated quizzes, means for delivering the confirmed and edited quizzes to target learners, means for collecting and analyzing quiz answers from target learners, means for evaluating the target learners' understanding levels based on the collected data and generating comprehension reports based on the analysis results, and means for acquiring user emotional data from the terminal, analyzing the emotional data, and reflecting the emotional data in quiz generation. This automates everything from learning material analysis to quiz generation, analysis of learners' answers, and even providing a learning experience that takes emotional states into account, enabling efficient and effective learning management.

[1868] "Instructional materials" refer to materials containing data and information used by learners for learning purposes.

[1869] A "generative AI model" is a program or algorithm that uses artificial intelligence to automatically perform specific tasks.

[1870] A "quiz" is a set of questions given to assess a learner's understanding.

[1871] A "user interface" is a part that provides a screen and operating means for a user to interact with a system.

[1872] A "server" is a computer that processes and manages data on a network.

[1873] "Terminal" means the device through which a user accesses the system.

[1874] "Emotion data" is information that represents the user's emotional state, and is data acquired through a camera or microphone.

[1875] "Analysis" refers to processing data or information to extract specific patterns or features.

[1876] "Comprehension report" refers to the evaluation results that show how well a learner has understood the learning content.

[1877] A "prompt" is a sentence that describes instructions or questions for a generative AI model.

[1878] MODE FOR CARRYING OUT THE INVENTION

[1879] The present invention relates to a system that automatically generates quizzes from learning materials, evaluates the learner's level of understanding, and simultaneously recognizes the user's emotional state and reflects it in the system. The following is a detailed description of this system.

[1880] First, the user logs in to the learning management system using a terminal. To log in, they enter their username and password, and if authentication is successful, they can access the administration screen. Next, the user clicks the "Upload Learning Materials" button on the administration screen and selects the learning material file (e.g., a PDF file) to upload from the file selection dialog. The terminal sends the selected learning material file to the server and displays a message indicating that the file has been successfully uploaded.

[1881] The server saves the uploaded learning material files in a designated storage area (e.g., AWS S3). The server then passes the saved learning material files to a generative AI model (e.g., OpenAI GPT-4), which analyzes the learning material content and extracts key points and keywords, such as "variables," "loops," and "conditional branching."

[1882] Meanwhile, to understand the user's emotional state, the device uses a built-in camera and microphone to capture the user's emotional data (e.g., facial expressions and voice tone). This emotional data is sent to the server in real time, and the server uses an emotion engine (e.g., Amazon Rekognition, Microsoft Azure Face API) to analyze the emotional state (e.g., stress, interest, concentration level, etc.). The analysis results are reflected in the generation of the next quiz.

[1883] The server issues commands to the generative AI model based on the analyzed characteristics of the learning material and the user's emotional state to automatically generate quizzes. Specifically, multiple choice questions, true / false questions, and written questions are generated based on important points. The difficulty and content of the quiz are adjusted according to the user's emotional state. For example, if the user is feeling stressed, easier questions will be given priority.

[1884] The generated quiz is temporarily saved and can be viewed on a management screen for display to the user. From this management screen, the user can check the questions and options of the generated quiz and edit them as necessary. Once editing is complete, the user clicks the "Confirm Quiz" button and the confirmed quiz is sent to the server.

[1885] Next, the server retrieves the list of learners and generates emails to deliver the finalized quiz to each learner. This is done using an email sending API (e.g., SendGrid, Amazon SES). Learners receive the email on their devices and click a link to access the quiz screen.

[1886] When a learner answers a quiz and submits their answers, the answer data is automatically sent to the server. The server receives this answer data and uses an automatic scoring system (e.g., a Google Sheets script or a Microsoft Excel macro) to score each answer and tally the scores. Finally, the server uses a generative AI model based on the tally to analyze each learner's level of comprehension and generate a comprehension report. This report is provided to the user, who can then provide the necessary feedback to the learner.

[1887] For example, if learning materials for learning "basic programming concepts" are uploaded, the generative AI model will analyze the materials and identify key points such as "variable definitions," "types of loops," and "conditional branching." Furthermore, if the emotion engine determines that the "stress level is high," the difficulty level of the generated quiz will be set low.

[1888] Below are some example prompts to input to a generative AI model:

[1889] "Analyze the content of the following learning materials and automatically generate quizzes for learners. The types of quizzes should be multiple choice, true / false, and written. Also, if the user's stress level is high, adjust the difficulty level to a lower level."

[1890] This concludes the detailed description of one embodiment of the present invention. This system makes it easy to automatically generate quizzes and measure comprehension levels, and is expected to provide an optimal learning environment for both users and learners.

[1891] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1892] Step 1:

[1893] A user logs in to a learning management system using a terminal. The user enters a username and password on the login screen, and the terminal sends this to the server. The server performs authentication, and if successful, provides the user with an administration screen. The input is the username and password, and the output is the authentication result. If the authentication result is successful, the administration screen is displayed.

[1894] Step 2:

[1895] The user clicks the "Upload teaching materials" button on the administration screen. The terminal displays a file selection dialog, and the user selects the teaching materials file (e.g., PDF file) to upload. When the "Open" button is clicked, the terminal sends the selected file to the server. The input is the teaching materials file, and the output is a message indicating that the upload has been completed successfully. A message is displayed indicating that the file has been uploaded successfully.

[1896] Step 3:

[1897] The server saves the uploaded learning material file in a storage area (e.g. AWS S3). The input is the uploaded learning material file, and the output is the result of saving it to storage. If saving is successful, the location where the file is saved is recorded.

[1898] Step 4:

[1899] The server passes the saved teaching material file to the generative AI model. It generates a prompt for the generative AI model and analyzes the content of the teaching material. The input is the saved teaching material file and the prompt, and the output is the main points and keywords of the teaching material. The generative AI model extracts key points such as "variables," "loops," and "conditional branching" from the content.

[1900] Step 5:

[1901] The device acquires the user's emotional data using the built-in camera and microphone. The data is sent to the server in real time. The input is the user's video and audio, and the output is emotional data. The acquired data is sent to the server.

[1902] Step 6:

[1903] The server uses an emotion engine to analyze the emotion data. The input is the transmitted emotion data, and the output is the judgment result of the emotion state (e.g., stress, interest, concentration). The emotion engine judges the emotion state and saves the result.

[1904] Step 7:

[1905] The server issues commands to the generative AI model based on the analyzed characteristics of the learning material and the user's emotional state to automatically generate a quiz. The prompt text reflects important points and the user's emotional state. The input is the characteristics of the learning material and the user's emotional state, and the output is the generated quiz (multiple choice questions, true / false questions, and written questions). The difficulty and content of the quiz are adjusted.

[1906] Step 8:

[1907] The server temporarily stores the generated quiz and provides an interface for user confirmation. A list of quizzes is displayed on the management screen. The input is the generated quiz, and the output is the interface for the management screen. The user can check the question text and options.

[1908] Step 9:

[1909] The user checks and edits the generated quiz on the management screen. Once the edits are complete, the user clicks the "Confirm Quiz" button. The input is the user's edits, and the output is the confirmed quiz data. Clicking the Confirm button sends the final data to the server.

[1910] Step 10:

[1911] The server retrieves the learner list and generates emails to distribute the confirmed quiz to each learner. It uses an email sending API. The input is the learner list and confirmed quiz data, and the output is email. An email is sent to each learner.

[1912] Step 11:

[1913] The learner receives the email using their device and clicks on the quiz link. They access the quiz screen, enter their answer, and submit it. The input is the learner's answer, and the output is the result of sending it to the server. The learner's answer is sent to the server.

[1914] Step 12:

[1915] The server receives the learner's answers and uses an automatic scoring system to tally up the scores. The input is the learner's answers and the output is the scoring results. The answers are scored by the automatic scoring system.

[1916] Step 13:

[1917] The server aggregates the results and the generative AI model analyzes each learner's level of understanding. The input is the scoring result, and the output is the comprehension analysis result. The generative AI model evaluates the level of understanding.

[1918] Step 14:

[1919] The server generates a comprehension report from the comprehension analysis results and provides it to the user. The input is the comprehension analysis results and the output is the comprehension report. The user can check the report and provide feedback.

[1920] (Application example 2)

[1921] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1922] Conventional online learning systems struggle to provide a personalized learning experience for each learner, resulting in insufficient learning effectiveness. Furthermore, they lack a means to provide learning content that takes into account the learner's emotional state, often resulting in learners feeling stressed or losing concentration. To solve these problems, a system is needed that recognizes the learner's emotional state in real time and adjusts the difficulty and content of the learning content accordingly.

[1923] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1924] In this invention, the server includes means for inputting learning materials, means for extracting features using a generative AI model that analyzes the input learning materials, means for automatically generating quizzes based on the extracted features, means for recognizing the emotional state of a user while watching, means for adjusting the difficulty of the quiz based on the recognized emotional state, means for providing a user interface for reviewing and editing the generated quiz, means for delivering the reviewed and edited quiz to a target learner, means for collecting and analyzing the quiz answers of the target learner, and means for generating a comprehension report based on the analysis results, thereby enabling the provision of personalized learning content that takes into account the learner's emotional state.

[1925] "Means for inputting teaching materials" refers to a function for uploading educational content (videos, PDFs, text, etc.) to a server from the device used by the learner.

[1926] A "generative AI model" is an artificial intelligence algorithm that analyzes the content of input teaching materials, extracts key points and keywords, and is used to automatically generate quizzes.

[1927] "Means for extracting features" refers to a function that uses a generative AI model to identify key points and important keywords from input learning materials.

[1928] The "means for automatically generating quizzes" is a function that automatically creates quizzes in various formats (multiple choice questions, true / false questions, written questions, etc.) based on the extracted features.

[1929] "Emotion recognition means" is a function that uses technology (such as facial recognition and voice analysis) to recognize the user's emotional state in real time while watching.

[1930] The "means for adjusting the difficulty of the quiz" is a function that dynamically changes the difficulty and content of the generated quiz based on the recognized emotional state of the user.

[1931] The "means for providing a user interface" is a function that provides a screen and operation means for the user to check and edit the generated quiz.

[1932] "Means for delivery to target learners" refers to the functionality for electronically transmitting the reviewed and edited quiz to the target learners.

[1933] "Means for collecting and analyzing quiz answers" is a function for collecting the results of quizzes answered by target learners and analyzing them to understand the learners' level of understanding and learning situation.

[1934] The "means for generating a comprehension report" is a function that creates a report summarizing each learner's level of comprehension and learning progress information based on the quiz answer results and analysis thereof.

[1935] MODE FOR CARRYING OUT THE INVENTION

[1936] The present invention relates to a system for automatically generating quizzes from video content in an online learning platform, and providing a personalized learning experience by recognizing a learner's emotional state in real time. A specific embodiment of the system and its operation are described below.

[1937] System Program

[1938] The system of the present invention consists of the following major components:

[1939] 1. How to input materials:

[1940] This function allows users to upload educational content (videos, PDFs, text, etc.) to the server from their devices. Devices include PCs and smartphones.

[1941] 2. How to extract features using generative AI models:

[1942] The server inputs the uploaded learning materials into a generative AI model to extract key points and keywords. For example, OpenAI GPT-3 is used as the generative AI model.

[1943] 3. Emotion recognition means to recognize the user's emotional state while watching:

[1944] This function uses the device's built-in camera and microphone to capture the user's facial expressions and voice, and analyzes their emotional state in real time. The emotion analysis engine used is, for example, the Microsoft Azure Emotion Analysis API.

[1945] 4. How to adjust the difficulty of the quiz:

[1946] This function allows the server to dynamically change the difficulty and content of the quiz based on the user's emotional state. For example, if the user is feeling stressed, easier questions will be prioritized.

[1947] 5. A means to provide a user interface for reviewing and editing the quiz:

[1948] This function allows the server to display the generated quiz to the user and provides an interface for checking and editing.

[1949] 6. Means of delivery to target learners:

[1950] This function allows the server to distribute the reviewed and edited quiz to the target learners via email or other means.

[1951] 7. How quiz answers will be collected and analyzed:

[1952] This function allows the server to collect and analyze learners' answers to distributed quizzes to assess their level of understanding.

[1953] 8. How to generate a comprehension report:

[1954] This function allows the server to generate a report showing each learner's level of understanding based on the analysis results and provide this to the user.

[1955] Program processing

[1956] The server uses a generative AI model (e.g., OpenAI GPT-3) to analyze the uploaded learning material. Once the learning material is uploaded, the server inputs it into the generative AI model to extract key points and keywords. For example, if a video on "SEO Strategy" is uploaded, the generative AI model will identify key points such as "keyword research" and "content marketing."

[1957] The device uses a built-in camera and microphone to recognize the user's emotional state in real time while watching, and performs analysis using an emotion analysis engine (for example, Microsoft Azure Emotion Analysis API), which determines the user's stress level and interest level.

[1958] The server adjusts the difficulty and content of the generated quiz based on the user's perceived emotional state: for example, it presents easier questions when the user is stressed and more detailed questions when the user is focused.

[1959] The user interface presents the generated quiz to the user and provides a screen for the user to review and edit, allowing the user to modify the question text and answer options.

[1960] The server then distributes the reviewed and edited quizzes to the target learners, who then answer the quizzes. The answers are sent to the server, where they are automatically compiled and analyzed. Finally, a report showing the learners' comprehension is generated and provided to the user.

[1961] Examples and prompts

[1962] For example, if a user is watching a video on "SEO Strategies," the following quiz will be auto-generated:

[1963] Multiple Choice Question: "What are the essential elements of SEO?"

[1964] A. Keyword Research

[1965] B. Visual Design

[1966] C. Database Management

[1967] True or False: "Meta descriptions affect SEO. True or False?"

[1968] Short answer question: "Explain the importance of on-page SEO."

[1969] Example prompt sentence:

[1970] Analyze the content of the following educational material and generate quiz questions. Please take into account the user's emotional state and set the difficulty level low if the user's stress level is high.

[1971] Material Content: SEO Strategies

[1972] This makes it possible to provide learners with personalized learning content and maximize learning effectiveness.

[1973] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1974] Step 1:

[1975] Uploading teaching materials

[1976] Subject: User

[1977] Description: A user uploads educational content (e.g., video, PDF, text file) from a device to a server. Specifically, the user logs in to a learning management system using a device, selects the educational material file, and clicks the "Upload" button. The input is the file path of the target educational material, and the output is the saving of the file to the server.

[1978] Step 2:

[1979] Teaching material analysis

[1980] Subject: Server

[1981] Description: The server inputs the uploaded learning material file into a generative AI model to analyze the content of the learning material. For example, the server uses a generative AI model (OpenAI GPT-3) to extract key points and keywords within the learning material. The input is the uploaded learning material file, and the output is a list of key points and keywords. Data processing includes text extraction and key point detection.

[1982] Step 3:

[1983] emotion recognition

[1984] Subject: Device

[1985] Description: The device uses the built-in camera and microphone to capture the user's facial expressions and voice. This data is sent in real time to an emotion analysis engine (Microsoft Azure Emotion Analysis API) to analyze the user's emotional state. The input is facial expression data and voice data, and the output is a judgment of the user's emotional state (e.g., stress level, interest level, etc.). Operations include facial recognition using the camera and voice analysis using the microphone.

[1986] Step 4:

[1987] Quiz Generation

[1988] Subject: Server

[1989] Description: The server issues commands to a generative AI model based on the extracted key points and emotional state data to automatically generate quizzes. The generated quizzes include a variety of formats, such as multiple choice, true / false, and written questions. The input is a list of key points and emotional state data, and the output is a set of generated quiz questions. Data calculation involves automatically generating quizzes using a generative AI model.

[1990] Step 5:

[1991] Quiz difficulty adjustment

[1992] Subject: Server

[1993] Description: The server dynamically adjusts the difficulty of the generated quiz based on emotional state data. For example, if the user is feeling stressed, easier questions will be prioritized. The input is the emotional state data and the generated quiz, and the output is the quiz questions with the adjusted difficulty. Actions include changing the question text and the format of the options.

[1994] Step 6:

[1995] Check and edit quiz

[1996] Subject: Server

[1997] Description: The server temporarily stores generated quizzes and provides an interface for users to review and edit them. Users can review quizzes from an admin screen and make corrections as needed. The input is the generated quiz set, and the output is the final quiz set edited by the user. Behavior includes providing a user interface and supporting quiz editing.

[1998] Step 7:

[1999] Quiz broadcast

[2000] Subject: Server

[2001] Description: The server generates and sends emails to distribute edited quizzes to target learners. The input is the final quiz set and a list of learners, and the output is an email containing an access link to the quiz. Actions include generating and sending emails.

[2002] Step 8:

[2003] Answer collection and analysis

[2004] Subject: Server

[2005] Description: The target learner answers a quiz on their device and sends the answers to the server. The server collects the answers and performs automatic scoring and comprehension analysis. The input is the learner's quiz answers, and the output is the scoring results and comprehension analysis results. Data calculations include comprehension analysis using an automatic scoring algorithm and a generative AI model.

[2006] Step 9:

[2007] Report Generation

[2008] Subject: Server

[2009] Description: The server generates a comprehension report for each learner based on the analysis results and provides it to the user. The input is the analysis result data, and the output is the comprehension report. The operations include generating the report and providing it to the user.

[2010] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[2011] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[2012] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[2013] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[2014] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[2015] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[2016] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[2017] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[2018] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[2019] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains em...

Claims

1. a means for inputting teaching materials; A means for extracting features using a generative AI model that analyzes input teaching materials; A means for automatically generating quizzes based on the extracted features; means for providing a user interface for reviewing and editing the generated quiz; a means for delivering the reviewed and edited quiz to the target learners; A means for collecting and analyzing quiz answers from target learners; The system includes means for generating a comprehension report based on the analysis results.

2. 2. The system of claim 1, further comprising means for a user to upload learning material files to the server.

3. 10. The system of claim 1, further comprising means for extracting key points from the educational content analyzed by the generative AI model.

Images