system

Abstract

Provide a system. [Solution] A means of collecting past questions from a database; A means for generating new questions based on collected past question data; means for transmitting the generated questions to a terminal; A means for a user to answer questions through a terminal; means for transmitting user response data to a server; A means for analyzing user response data and evaluating the level of understanding; a means for generating feedback based on comprehension; means for transmitting the generated feedback to the terminal; means for displaying feedback to the user through the terminal; A system including:

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Traditional education systems require high labor costs and many resources to maintain a certain level of educational quality. Furthermore, because the content of education depends on the individual abilities of the teacher, there is a problem of variation in educational quality depending on the location and environment. This makes it difficult for all students to receive an equally high-quality education. Furthermore, it is difficult for each teacher to provide individual feedback to every student, making it difficult to properly grasp students' levels of understanding and provide instruction based on that feedback. [Means for solving the problem]

[0005] The present invention provides a system that collects past question data and generates new questions based on that data. It also includes a means for transmitting the generated questions to a terminal, allowing the user to answer the questions through the terminal, and transmitting the answer data to a server. Furthermore, these problems are solved by providing a means for the server to analyze the user's answer data, evaluate their level of understanding, generate feedback based on that, send it to the terminal, and display it to the user. This makes it possible to provide a uniform quality of education regardless of location or environment while reducing labor costs. Furthermore, by using generation AI to efficiently evaluate the level of understanding and provide appropriate feedback individually, it is possible to improve the academic ability of each student.

[0006] A "database" is an electronic data collection system designed to efficiently store, search, and manage information.

[0007] "Means of collection" refers to the functions and methods for collecting specific information or data.

[0008] "Generative means" refers to the functions and methods for creating new information or problems based on specific conditions and data.

[0009] "Transmitting means" refers to the functionality or method for transferring specific data or information from one device to another.

[0010] "Terminal" refers to an electronic device such as a computer or tablet used by a user.

[0011] "Means for answering" refers to the functions and methods by which a user inputs answers to questions using a terminal.

[0012] "Means of analysis" refers to the functions and methods for processing collected data and information and understanding its content.

[0013] "Means of evaluation" refers to functions and methods for determining the user's level of understanding and performance using specific standards or criteria based on the analysis results.

[0014] "Feedback" is information such as evaluation and advice provided in response to a user's actions or answers.

[0015] "Means for displaying" refers to functions and methods that allow users to visually confirm information. [Brief explanation of the drawings]

[0016] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13]FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0017] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0018] First, the terms used in the following description will be explained.

[0019] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0020] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0021] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0022] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0023] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0024] [First embodiment]

[0025] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0026] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0027] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0028] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0029] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0030] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0031] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0032] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0033] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0034] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0035] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0036] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0037] This invention is a system for realizing efficient and uniform education by collecting past question data, generating new questions, and providing them to users. This system is composed of the following phases.

[0038] 1. Problem generation phase

[0039] The server collects past exam data from a database, for example, across multiple subjects such as math, English, and history.

[0040] The server generates new questions based on the collected question data. At this time, the difficulty and format of the questions can be specified using the generation AI.

[0041] 2. Problem Delivery Phase

[0042] The server packets the generated problem in XML or JSON format and sends it to the terminal.

[0043] The device displays the received questions on a user interface that is designed to make it easy for users to answer intuitively.

[0044] 3. Response Phase

[0045] The user answers questions using the device, such as multiple-choice questions in a quiz format or essay-style math questions.

[0046] The device converts the user's response data into XML or JSON format and sends it to the server.

[0047] 4. Understanding Assessment Phase

[0048] The server receives the response data sent by the user and passes it to the analysis module.

[0049] The server's AI analyzes the answer data, determines whether the answer is correct, and evaluates the level of understanding. In this process, the accuracy of the answer, the time it takes, and the answering method are used as evaluation criteria.

[0050] 5. Feedback Phase

[0051] The server generates feedback for the user based on the analysis results, including the percentage of correct answers, explanations for incorrect answers, and suggestions for additional study.

[0052] The server sends the generated feedback to the terminal, which displays the feedback on a user interface.

[0053] Users can check the feedback and use it to improve their learning progress.

[0054] Specific examples

[0055] Problem generation phase:

[0056] For example, the server collects equation problems from the past 10 years from a database of mathematical problems.

[0057] The server generates new equation problems based on the collected data, for example, by setting the difficulty level to beginner, intermediate, or advanced and creating different types of problems.

[0058] Problem Delivery Phase:

[0059] The server generates equation problems and packets them to the user's tablet.

[0060] The device receives these questions and displays them on the tablet screen, laid out to make it easy for users to see and answer them.

[0061] Response Phase:

[0062] The user answers the displayed equation questions using a tablet, for example, by writing the equations on an answer sheet.

[0063] The terminal converts the answers written by the user into digital data and sends it to the server.

[0064] Comprehension assessment phase:

[0065] The server analyzes the received answer data, and the AI ​​determines whether the answer is correct or not. For example, it takes into account not only the accuracy of the answer but also the time it took to answer.

[0066] The server compares the user's answer data with the correct answer data to evaluate the level of understanding.

[0067] Feedback Phase:

[0068] Based on the evaluation results, the server generates feedback such as, "This part was not answered correctly, so please read the explanation below and try again."

[0069] The server sends the generated feedback to the tablet, which displays the feedback on its user interface.

[0070] The user reviews the feedback and re-studies the question to gain a better understanding of the incorrect answers.

[0071] In this way, the present invention can standardize the quality of education and reduce labor costs. Furthermore, by providing feedback according to each user's level of understanding, it can effectively support the improvement of each user's academic ability.

[0072] The processing flow will be explained below.

[0073] Step 1:

[0074] The server connects to the database and collects past exam data. For example, it can collect exam data for a specific subject (such as mathematics or English).

[0075] Step 2:

[0076] New problems are generated based on past problem data collected by the server. At this time, a generation AI is used to specify the difficulty and format of the problem. For example, equation problems with beginner, intermediate, and advanced difficulty levels can be created.

[0077] Step 3:

[0078] The server converts the generated questions into XML or JSON format and creates a data packet for distribution.

[0079] Step 4:

[0080] The server sends the data packets to the terminal via the Internet.

[0081] Step 5:

[0082] The device analyzes the received question data and displays the questions on the user interface. The layout is adjusted to make it easier for the user to answer intuitively.

[0083] Step 6:

[0084] The user uses the device to answer the displayed questions, for example by entering mathematical expressions using the touchscreen or by selecting answers from multiple options.

[0085] Step 7:

[0086] The device converts the user's response data into XML or JSON format and sends it to the server.

[0087] Step 8:

[0088] The server receives the response data sent by the user and passes it to the analysis module.

[0089] Step 9:

[0090] The server's generated AI analyzes the answer data and judges whether it is correct or not, for example, by evaluating the accuracy of the formula, the answer method, and the answer time.

[0091] Step 10:

[0092] The server evaluates the user's level of understanding and generates feedback based on that evaluation, including, for example, explanations for incorrect answers and additional practice questions.

[0093] Step 11:

[0094] The server generates feedback and sends it to the device.

[0095] Step 12:

[0096] The device analyzes the received feedback and displays it in the user interface, providing visual explanations and additional questions to help users understand the process.

[0097] Step 13:

[0098] Users can review the feedback and use it to improve their learning, retrying the questions if necessary to further their understanding.

[0099] Through the above steps, the present invention can provide education efficiently and effectively, and improve the academic ability of users.

[0100] Example 1

[0101] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0102] Conventional educational systems have the problem of providing uniform questions that cannot adequately address the individual learner's level of understanding or adaptability, resulting in reduced learning effectiveness. They also require a large amount of human resources, making it difficult to provide efficient educational support. Furthermore, they lack a mechanism for effectively utilizing past learning data, making it difficult to provide appropriate feedback based on the learner's progress. To solve these issues, a system is needed that can automatically generate new questions based on collected data and provide feedback tailored to each learner's individual level of understanding.

[0103] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0104] In this invention, the server includes a means for collecting past questions from a database, a means for inputting prompts to the AI ​​model to generate new questions, and a means for transmitting the generated questions to the terminal. This makes it possible to automatically generate new questions using past learning data and provide appropriate feedback according to the individual learner's level of understanding.

[0105] The "means for collecting past questions from a database" is a means having a function for acquiring past question data stored in a database.

[0106] A "means for inputting prompts to an AI model to generate new problems" is a means for inputting instructions to a generative AI model, thereby generating new problems.

[0107] "Means for transmitting generated problems to a terminal" means means for transmitting problems generated by an AI model to a terminal in a digital format (e.g., JSON or XML).

[0108] The "means for the user to answer questions via the terminal" refers to a means for the user to answer questions displayed via a user interface.

[0109] The "means for transmitting user response data to the server" refers to a means for transmitting the response data entered by the user to the server in digital form.

[0110] The "means for analyzing the user's response data and evaluating the level of understanding" is a means for analyzing the response data sent by the user and evaluating the user's level of understanding based on the results.

[0111] The "means for generating feedback based on comprehension" is a means for generating feedback for the user based on the results of the assessed comprehension.

[0112] The "means for transmitting the generated feedback to the terminal" refers to means for transmitting the generated feedback to the terminal and displaying it to the user.

[0113] "Means for generating new problems using an AI model" refers to means for automatically generating new problems based on past problem data using an artificial intelligence model.

[0114] A "prompt" is an input sentence that provides instructions to an AI model and includes detailed conditions such as the type and difficulty of the problem to be generated.

[0115] The present invention is a system that realizes efficient and uniform education by collecting past question data, generating new questions, and providing them to users. This system is realized using the following hardware and software.

[0116] Hardware used

[0117] 1. Server: A computer system that connects to a database and collects, analyzes, and transmits data.

[0118] 2. Device: The device (e.g., tablet, smartphone, computer) on which the user answers the questions and receives feedback.

[0119] Software used

[0120] 1. Database management system: Database software such as MySQL (registered trademark), MongoDB, etc.

[0121] 2. Generative AI models: For example, generative AI models such as GPT-4 (registered trademark).

[0122] 3. Programming languages ​​and frameworks: Python, Flask, Django, etc.

[0123] System Operation

[0124] The server first collects past problem data from a database, using a method such as an SQL query to retrieve data that matches specific conditions. For example, the following query is used:

[0125] SELECT FROM question_data WHERE subject = 'Mathematics' AND year >= 2013

[0126] Next, the server uses the generative AI model to generate new questions. The server then inputs prompts like the following to the generative AI model to specify the type and difficulty of the questions desired:

[0127] Please set the difficulty level of this problem to beginner. Please generate other problems as well.

[0128] The generated new question is converted into XML or JSON format by the server and sent to the terminal. For example, the following JSON format data is sent:

[0129] {"Question": "Solve the equation", "Difficulty": "Beginner"}

[0130] The terminal displays the questions received from the server on the user interface. The user answers the questions using a keyboard or touch screen. The terminal converts the user's answers into digital data and sends them back to the server.

[0131] The server analyzes the response data received from the user and evaluates the level of understanding. The analysis uses a generative AI model and an analysis module to evaluate the accuracy of the response, response time, response method, etc. The server then generates feedback based on the evaluation results and sends it to the user.

[0132] This feedback might look something like this:

[0133] This part was not answered correctly, so please read the explanation below and try again.

[0134] The device displays the received feedback on the user interface for the user to confirm, allowing the user to check their own understanding and engage in further learning.

[0135] This system makes it possible to generate new questions using past learning data and provide feedback based on individual levels of understanding, thereby providing efficient and effective educational support.

[0136] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0137] Step 1:

[0138] The server collects past problem data from a database.

[0139] Specifically, the server connects to a database management system (e.g., MySQL) and executes an SQL query such as the following:

[0140] Input: Database connection information, query conditions (e.g., subject "Mathematics", year "2013 or later").

[0141] Output: A set of historical question data.

[0142] SELECT FROM question_data WHERE subject = 'Mathematics' AND year >= 2013

[0143] The server records the number and contents of the data in a log to confirm that the acquired data is correct.

[0144] Step 2:

[0145] New questions are generated based on the problem data collected by the server.

[0146] Specifically, the server inputs a prompt to the generative AI model (e.g., GPT-4) and instructs it to generate a problem.

[0147] Input: Past question data, prompt text.

[0148] Output: The new problem that was generated.

[0149] Example prompt sentence:

[0150] Please set the difficulty level of this problem to beginner. Please generate other problems as well.

[0151] The server receives the generated problem, verifies its contents, and stores it in a log.

[0152] Step 3:

[0153] The server packets the generated problem in XML or JSON format and sends it to the terminal.

[0154] Specifically, the server converts the problem into JSON format as shown below and sends it to the terminal as an HTTP POST request.

[0155] Input: Generated problem data.

[0156] Output: Sending status to terminal.

[0157] {"Question": "Solve the equation", "Difficulty": "Beginner"}

[0158] The server checks the HTTP request status code (e.g., 200 OK) and logs the successful submission.

[0159] Step 4:

[0160] The problem received by the terminal is displayed on the user interface.

[0161] Specifically, the device parses the received JSON data and constructs a layout that is intuitive for the user.

[0162] Input: The problem data received from the server.

[0163] Output: The problem as displayed to the user.

[0164] The terminal generates a timestamp to record when the user begins to answer.

[0165] Step 5:

[0166] The user answers the questions using the device.

[0167] Specifically, the user inputs the answer using a keyboard or touch screen.

[0168] Input: The user's answer.

[0169] Output: The entered answer data.

[0170] The terminal saves the user's input and proceeds to the next step when the answer button is pressed.

[0171] Step 6:

[0172] The terminal transmits the user's response data to the server.

[0173] Specifically, the device converts the answer data into JSON format as shown below and sends it to the server as an HTTP POST request.

[0174] Input: User's answer data.

[0175] Output: Status of submission to server.

[0176] {"Answer": "x = 2", "User ID": "12345"}

[0177] The terminal receives a response from the server and confirms that the transmission was successful.

[0178] Step 7:

[0179] The server receives the response data sent by the user and passes it to the analysis module.

[0180] Specifically, the server reads the received data using a Python library (e.g., Pandas) and analyzes it.

[0181] Input: User response data.

[0182] Output: Analysis results.

[0183] The server logs the analysis results in a format similar to the following: {"Correct Answer": True, "Answer Time": "5 minutes"}.

[0184] Step 8:

[0185] The server evaluates the level of understanding based on the response data.

[0186] Specifically, the data is analyzed using an AI model and evaluated based on criteria such as correctness and response time.

[0187] Input: Parsed response data.

[0188] Output: Comprehension assessment results.

[0189] The server stores the evaluation results in a database.

[0190] Step 9:

[0191] The server generates feedback based on the results of the comprehension assessment and sends it to the terminal.

[0192] Specifically, the server uses an AI model to generate feedback and sends it to the device in JSON format as follows:

[0193] Input: Comprehension assessment results.

[0194] Output: The generated feedback.

[0195] Generate feedback similar to the following example:

[0196] This part was not answered correctly, so please read the explanation below and try again.

[0197] The server logs the feedback sending status.

[0198] Step 10:

[0199] The terminal displays the feedback on the user interface for the user to confirm.

[0200] Specifically, the terminal parses the received feedback and displays it to the user.

[0201] Input: Feedback received from the server.

[0202] Output: The feedback that is displayed to the user.

[0203] The terminal records the timestamp when the user confirms the feedback and confirms the content of the re-learning.

[0204] The above is the specific processing flow of this system.

[0205] (Application example 1)

[0206] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0207] In conventional factory robot training programs, operator skill evaluations and feedback are uniform, making it difficult to improve training according to individual levels of understanding.There is a need for a system that can efficiently generate technical problems actually required in the field, evaluate the operator's level of understanding based on those problems, and provide appropriate feedback.

[0208] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0209] In this invention, the server includes means for collecting past question data from a database, means for generating new questions using a generative AI model, means for transmitting the generated questions to a terminal, means for a user to answer the generated questions using a tablet and transmit the answer data to the server, means for analyzing the answer data and evaluating the level of understanding based on accuracy, answer time, and operation procedures, and means for generating feedback based on the analysis results and providing the user with an opportunity to re-study. This enables efficient evaluation and feedback tailored to individual levels of understanding in a factory robot operator training program.

[0210] "Past problems" are data on problems and training tasks that the user has previously tackled.

[0211] A "database" is an information system for managing and storing past questions and answer data.

[0212] "New problems" are new training problems generated using a generative AI model based on past problem data.

[0213] A "terminal" is a digital device through which a user accesses and responds to training programs and questions.

[0214] A "server" is a centralized computer system that manages problems and data, analyzes them, generates feedback, etc.

[0215] An "analysis module" is a program that runs on a server and analyzes the user's response data to evaluate their level of understanding.

[0216] A "generative AI model" is an artificial intelligence model that generates new problems based on past problem data.

[0217] "Feedback" is information that is generated based on the user's answers and includes an evaluation of the user's level of understanding and suggestions for relearning.

[0218] A "tablet" is a portable computing device that allows a user to answer training questions and receive feedback.

[0219] "Question difficulty" refers to the level of difficulty of the questions to be generated.

[0220] "Answer time" is the time it takes the user to answer the question.

[0221] An "operation procedure" is a process of a series of actions performed by a user when answering a question.

[0222] "Relearning" is the process by which a user re-learns based on feedback.

[0223] A "training program" is a series of educational courses that factory robot operators go through to acquire specific skills and knowledge.

[0224] The "evaluation criteria" are scales or indicators used to evaluate the user's response data.

[0225] The present invention is a system for evaluating a factory robot operator training program and providing feedback for improvement. This system collects past training question data, generates new questions, and provides them to users, thereby achieving efficient and uniform technical education. A specific embodiment of this system will be described below.

[0226] 1. Problem generation phase

[0227] The server collects past training problem data from a database. This database stores problems in various technical fields (e.g., welding, assembly, and maintenance). Based on the collected data, the server generates new problems using a generative AI model (e.g., OpenAI's GPT-3). The difficulty and format of the generated problems can be specified as needed.

[0228] Specific examples

[0229] The server collects data on past welding technology problems and inputs the following prompt into GPT-3: "Generate new welding technology problems in the following format: medium difficulty, solution time within 30 minutes, use the following materials..." to generate new problems.

[0230] 2. Problem Delivery Phase

[0231] The generated questions are packetized in JSON format and sent to the terminal. The terminal displays the received questions on a user interface, which is designed to make the questions easy for users to see and answer intuitively.

[0232] Specific examples

[0233] The generated welding problems are packetized in JSON format and sent to a tablet, where they are displayed and the user works on them.

[0234] 3. Response Phase

[0235] The user answers questions using a terminal. The answer data is converted into digital data by the terminal and sent to the server. The server analyzes the received answer data and evaluates the level of understanding.

[0236] 4. Understanding Assessment Phase

[0237] The server's analysis module analyzes the response data and evaluates the level of understanding based on accuracy, response time, and operation procedures. The evaluation results are generated by the analysis module.

[0238] Specific examples

[0239] The server analyzes the user's response data and generates feedback such as, "The heat distribution during welding is not uniform, resulting in an improper joint. Please re-learn the cause and how to improve it."

[0240] 5. Feedback Phase

[0241] Feedback is generated based on the analysis results and sent to the device, which then displays the feedback on the user interface, giving the user an opportunity to re-learn.

[0242] Hardware and Software

[0243] Hardware: Smartphones and tablets (e.g., iPad (registered trademark))

[0244] Software: MySQL (database), JSON (data format), generative AI model (GPT-3), front-end (React Native)

[0245] Prompt Sentence Examples

[0246] Create a new welding problem in the following format:

[0247] Difficulty: Intermediate

[0248] Answer time: within 30 minutes

[0249] Materials used: iron, aluminum

[0250] In this way, the system of the present invention can efficiently implement training programs for factory robot operators and provide evaluations and feedback according to each operator's level of understanding, thereby effectively supporting the improvement of operator technical skills.

[0251] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0252] Step 1:

[0253] The server collects past question data from a database. The database stores past question data for each training program, and the server accesses and collects this data. The input is a database query, and the output is past question data. Specifically, the server accesses the database using an SQL query to retrieve the required data.

[0254] Step 2:

[0255] The server generates new problems using a generative AI model based on the collected past problem data. The input is the collected past problem data and the specified problem difficulty and format, and the output is the generated new problem. Specifically, a prompt sentence is input to the generative AI model (e.g., GPT-3) to generate a new problem.

[0256] Step 3:

[0257] The server packets the generated new problem in JSON format and sends it to the terminal. The input is the generated new problem, and the output is the problem data converted to JSON format. Specifically, the problem data is serialized in JSON format and sent to the terminal over the network.

[0258] Step 4:

[0259] The terminal displays the received problem data on the user interface. The input is the received JSON format problem data, and the output is the problem displayed to the user. Specifically, it deserializes the JSON data and renders the problem content on the screen.

[0260] Step 5:

[0261] The user answers questions through the terminal. The input is the answer selected or written by the user, and the output is the user's answer data. Specifically, the user enters the answer using the user interface and presses the send button to send the answer to the terminal.

[0262] Step 6:

[0263] The terminal sends the user's answer data to the server. The input is the user's answer data, and the output is the data to be sent to the server. Specifically, the answer data is converted into JSON format and sent to the server.

[0264] Step 7:

[0265] The server analyzes the received answer data and evaluates the level of comprehension. The input is the user's answer data, and the output is the comprehension evaluation result. Specifically, the server uses the analysis module to evaluate the accuracy of the answer, the answer time, and the operation procedure.

[0266] Step 8:

[0267] The server generates feedback based on the analysis results and sends it to the device. The input is the comprehension assessment result, and the output is the generated feedback data. Specifically, the server uses a feedback generation algorithm to create feedback for the user, converts it into JSON format, and sends it to the device.

[0268] Step 9:

[0269] The terminal displays the received feedback on the user interface. The input is the received feedback data, and the output is the feedback content displayed to the user. Specifically, the terminal deserializes the feedback data and renders it on the screen.

[0270] Through the above process steps, the system of the present invention can support a user's technical training program and provide efficient and individualized assessment and feedback according to the user's level of understanding.

[0271] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0272] The present invention is a system that provides education that takes into account not only the user's level of understanding but also their emotional state by combining a conventional education system with an emotion engine. This system is composed of the following phases.

[0273] 1. Problem generation phase

[0274] The server connects to a database and collects past exam data, such as questions for subjects like math, English, and history.

[0275] New questions are generated based on the question data collected by the server. Using the generation AI, questions can be generated by specifying their difficulty and format.

[0276] 2. Problem Delivery Phase

[0277] The server converts the generated questions into XML or JSON format and creates a data packet for distribution.

[0278] The server sends the data packets to the terminal via the Internet.

[0279] 3. Response Phase

[0280] The device analyzes the received question data and displays the questions on the user interface, which is designed to be laid out in a way that makes it easy for users to answer intuitively.

[0281] The user answers the questions using the device, for example by entering mathematical expressions using the touchscreen of a tablet or by selecting multiple choices.

[0282] 4. Emotional Data Collection Phase

[0283] An emotion engine built into the device analyzes the user's facial expressions and tone of voice to collect emotion data. For example, a camera can capture facial expressions and a microphone can analyze tone of voice.

[0284] The emotion data collected by the device is converted into XML or JSON format and sent to the server.

[0285] 5. Understanding Assessment Phase

[0286] The server receives the response data and emotion data sent by the user and passes them to the analysis module.

[0287] The server's generated AI analyzes the answer data, determines whether the answer is correct, and evaluates the level of understanding. It also evaluates the accuracy of the answer, the time it took to answer, and the answering method.

[0288] Furthermore, data from the emotion engine is taken into consideration and the emotional state of the user when answering is taken into account in the evaluation.

[0289] 6. Feedback generation phase

[0290] The server generates feedback based on the analysis results, taking into account the user's level of understanding and emotional state. For example, if a user has a high level of understanding but is nervous, the server may generate feedback such as "Good job! Stay calm and try again next time."

[0291] The server generates feedback and sends it to the device.

[0292] 7. Feedback display phase

[0293] The device analyzes the received feedback and displays it in the user interface. The feedback may include additional questions, explanations, or psychological support messages.

[0294] Users can review the feedback and use it to improve their learning, for example by re-studying incorrect answers to deepen their understanding.

[0295] Specific examples

[0296] Problem generation phase:

[0297] The server collects calculus problems from the past 10 years from a mathematical problem database.

[0298] The server generates new problems based on this data, such as "Beginner: Simple Differential Problems," "Intermediate: Definite Integral Problems," and "Advanced: Complex Integral Problems."

[0299] Problem Delivery Phase:

[0300] The server generates a problem packet and sends it to the user's tablet.

[0301] The device receives the question and displays it on the tablet screen.

[0302] Response Phase:

[0303] The user answers the displayed calculus problems using a tablet, for example, by writing mathematical formulas using a stylus pen.

[0304] The terminal converts the user's answers into digital data and sends it to the server.

[0305] Sentiment data collection phase:

[0306] The device's built-in camera captures the user's facial expressions, while the microphone analyzes the tone of the voice, detecting, for example, facial expressions that indicate impatience or a tense voice.

[0307] The terminal transmits this emotion data to the server.

[0308] Comprehension assessment phase:

[0309] The server analyzes the received answer data and evaluates whether the answer was correct or incorrect and the answer time.

[0310] Furthermore, emotional data is analyzed and the user's emotional state while answering is taken into account in the evaluation.

[0311] Feedback generation phase:

[0312] The server generates feedback based on the evaluation, including a message such as "Well done, but you seem a little nervous, so please relax and try again."

[0313] The server generates feedback and sends it to the user's tablet.

[0314] Feedback display phase:

[0315] The device displays the received feedback, which can include explanations based on comprehension and relaxation advice based on emotional data.

[0316] Users can review the feedback and use it to improve their learning experience next time.

[0317] In this way, the present invention can provide more comprehensive and effective learning support by providing education while taking into consideration the user's emotions.

[0318] The processing flow will be explained below.

[0319] Step 1:

[0320] The server connects to the database and collects past exam data, such as questions for each subject, such as mathematics, English, and history.

[0321] Step 2:

[0322] Based on past question data collected by the server, new questions are generated using a generation AI. For example, questions can be created by specifying a specific level of difficulty (beginner, intermediate, advanced) or format (multiple choice, written).

[0323] Step 3:

[0324] The server converts the generated questions into XML or JSON format and creates a data packet for distribution.

[0325] Step 4:

[0326] The server sends the data packets to the terminal via the Internet.

[0327] Step 5:

[0328] The device analyzes the received question data and displays the questions on the user interface. For example, the questions are laid out visually on a tablet screen.

[0329] Step 6:

[0330] The user uses the device to answer the displayed questions, for example by entering a mathematical formula or selecting multiple choices using the touchscreen.

[0331] Step 7:

[0332] The device converts the user's response data into XML or JSON format and sends it to the server.

[0333] Step 8:

[0334] The device's built-in emotion engine captures the user's facial expressions and tone of voice to collect emotion data. For example, the camera detects facial expressions and the microphone records tone of voice.

[0335] Step 9:

[0336] The emotion data collected by the device is converted into XML or JSON format and sent to the server.

[0337] Step 10:

[0338] The server receives the response data and emotion data sent by the user and passes them to the analysis module.

[0339] Step 11:

[0340] The server's generated AI analyzes the answer data and determines whether the answer is correct or incorrect and evaluates the level of understanding. For example, the accuracy of the answer, the answering time, and the answering method are used as evaluation criteria.

[0341] Step 12:

[0342] The server then performs additional analysis of the emotional data to assess the user's emotional state while answering the questions, such as analyzing facial expressions and tone of voice that indicate impatience or nervousness.

[0343] Step 13:

[0344] The server generates feedback based on the level of understanding and emotional state, such as a message like "Good job, but you seem a little nervous, so please relax and try again."

[0345] Step 14:

[0346] The server generates feedback and sends it to the device.

[0347] Step 15:

[0348] The device analyzes the received feedback and displays it in the user interface. For example, the feedback may include specific explanations based on the user's level of understanding or relaxation advice based on the user's emotional state.

[0349] Step 16:

[0350] Users can check the feedback and use it to improve their learning the next time. For example, they can re-study the incorrect answers and put the advice from the emotion engine into practice, improving their learning effectiveness.

[0351] Through the above steps, the present invention can comprehensively evaluate the user's understanding and emotions and provide optimal feedback.

[0352] Example 2

[0353] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0354] Conventional educational systems are limited to assessing users' learning progress and level of understanding, and lack support that takes into account the user's emotional state. This makes it difficult to improve learning efficiency and maintain motivation, potentially affecting long-term learning outcomes. Furthermore, the quality and variety of questions generated are limited, making it difficult to meet the diverse learning needs of users. It was necessary to solve these issues and create a system that provides more comprehensive and effective learning support.

[0355] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for collecting past data from a database, means for generating new questions based on the collected past data, and means for transmitting the generated questions to the terminal. This enables comprehensive learning support that takes into account not only the user's learning progress but also their emotional state. Specifically, new questions can be generated by specifying the difficulty level and format of the questions using a generative AI model, thereby meeting the diverse learning needs of users. Furthermore, by analyzing the user's response data and adding emotional data to the analysis, not only the user's level of understanding but also their emotional state during learning can be reflected in the evaluation. This allows for more personalized feedback to the user, improving learning efficiency and maintaining motivation.

[0356] I understand. Below are definitions of important terms included in the claims.

[0357] "Past data" refers to previously collected information related to the educational subject, such as question and answer data stored in a database.

[0358] A "database" is an information system used to efficiently store, manage, and search large amounts of data.

[0359] "Means for generating new questions" refers to the technology and algorithms used to generate new questions for users to answer based on collected past data.

[0360] A "generative AI model" refers to a machine learning model that uses artificial intelligence technology to generate new problems.

[0361] "Device" refers to an electronic device, such as a computer, tablet, or smartphone, that allows a user to access and answer questions.

[0362] "User" refers to an individual who uses the educational system to learn.

[0363] "Server" refers to the central system that collects data from the database, generates questions, and evaluates answers.

[0364] "Answer data" is digital information of the answer to the question that the user inputs through the terminal.

[0365] "Emotion data" is data that indicates the emotional state of the user analyzed from facial expressions, tone of voice, and the like.

[0366] "Evaluation" refers to the process of analyzing the user's response data and emotional data to determine the user's level of understanding and emotional state.

[0367] "Feedback" refers to study advice and instructions on next steps provided to the user based on the evaluation results.

[0368] "Analysis" refers to the process of extracting meaningful information from collected data and determining the user's level of understanding and emotional state.

[0369] The above are definitions of important terms contained in the claims.

[0370] The present invention is a system that provides education that takes into account not only the user's level of understanding but also their emotional state. This system aims to improve the efficiency and effectiveness of learning by combining a conventional education system with an emotional engine. The following describes how to specifically implement the invention.

[0371] System Overview

[0372] This system consists of a server, terminals, and users. The roles of each are as follows:

[0373] Server: Collects data from the database, generates new questions, and sends them to the device. It also analyzes the user's response data and emotional data, evaluates their understanding and emotional state, and generates feedback.

[0374] Terminal: Provides an interface for users to answer questions, collects answer data and emotion data, and sends them to the server. Receives feedback and displays it to the user.

[0375] User: Answers questions and receives feedback via the device.

[0376] Hardware and Software Configuration

[0377] The server is configured using, for example, a Python environment and a MySQL database, and uses a generative AI model such as OpenAI's GPT-3 to generate problems.

[0378] The devices are electronic devices such as tablets, smartphones, and PCs, and the user interface is built using JavaScript (registered trademark), HTML, and CSS. Emotion data is collected using libraries such as OpenCV and TENSORFLOW (registered trademark).

[0379] Users use these terminals to carry out learning activities.

[0380] Processing flow

[0381] 1. Problem generation:

[0382] The server connects to a MySQL database and collects past problem data using SQL queries.

[0383] Based on the collected data, a generative AI model (e.g., GPT-3) is used to generate new problems, with prompts being entered to generate the problems.

[0384] The generated questions are converted into JSON format and sent to the terminal.

[0385] Example prompt sentence:

[0386] "Generate beginner calculus problems"

[0387] "The user has a high level of understanding but is nervous, so generate feedback with advice to relax."

[0388] 2. Question delivery and answers:

[0389] The terminal analyzes the received problem data and displays it on the user interface.

[0390] The user enters mathematical expressions and answers using the tablet's touchscreen.

[0391] The terminal converts the user's answers into digital data and sends it to the server.

[0392] 3. Emotional Data Collection:

[0393] The device's built-in camera and microphone capture the user's facial expressions and tone of voice to collect emotional data.

[0394] Using OpenCV and TensorFlow, we analyze this data to determine the user's emotional state.

[0395] The collected emotion data is converted into JSON format and sent to the server.

[0396] 4. Comprehension assessment and feedback generation:

[0397] The server analyzes the response data and emotion data received using the Python pandas library to evaluate comprehension and emotional state.

[0398] Generative AI models are used to generate feedback based on the evaluation results, such as "Good job, but you seem a little nervous, so please relax and try again."

[0399] The generated feedback is sent to the device.

[0400] 5. Feedback display:

[0401] The terminal analyzes the received feedback and displays it on the user interface.

[0402] Users can review the feedback and use it to improve their learning experience next time.

[0403] In this way, the system of the present invention can simultaneously consider the user's level of understanding and emotional state, and provide more comprehensive and effective learning support.

[0404] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0405] Processing Steps

[0406] Step 1:

[0407] This is the procedure by which the server collects past data. Specifically, the server connects to a MySQL database and collects past problem data using SQL queries.

[0408] Input: MySQL database connection information and SQL query

[0409] Output: Problem data (past problem data)

[0410] Specific operation: The server uses Python's mysql-connector library to connect to a database named my_database, then executes the SQL query "SELECT FROM math_problems WHERE subject='Calculus' AND year BETWEEN® 2011 AND 2021;" to retrieve past problem data.

[0411] Step 2:

[0412] This is the procedure where the server generates new problems using a generative AI model. Based on past problem data collected by the server, new problems are generated using a generative AI model (e.g., GPT-3).

[0413] Input: Past problem data and prompts for the generative AI model

[0414] Output: New problem data

[0415] Specific operation: The server sends a prompt such as "Generate beginner calculus problems" to the OpenAI API based on the collected past problem data. The API responds with new problem data.

[0416] Step 3:

[0417] This is the procedure for sending questions generated by the server to the terminal. The generated question data is converted into JSON format and sent to the terminal via a RESTful API.

[0418] Input: Newly generated question data

[0419] Output: JSON data ready to be sent to the terminal

[0420] How it works: The server converts the generated question data into JSON format using Python's json library, and then sends the question data to the terminal using a RESTful API created using Flask.

[0421] Step 4:

[0422] This is the procedure in which the terminal analyzes the received question data and displays it on the user interface.

[0423] Input: Question data in JSON format sent from the server

[0424] Output: The problem as displayed in the user interface

[0425] Specific behavior: The device parses the JSON data using JavaScript and displays the problem in the user interface using HTML and CSS. Example: document.getElementById('problem').innerText = json_data['problem_text'];

[0426] Step 5:

[0427] This is the procedure by which the user answers the questions.

[0428] Input: The problem displayed on the terminal

[0429] Output: User response data

[0430] Specific actions: The user answers the displayed questions using the tablet's touchscreen, for example, by entering mathematical expressions using a stylus.

[0431] Step 6:

[0432] This is the procedure in which the terminal converts the user's answer into digital data and sends it to the server.

[0433] Input: User's handwritten answer or input

[0434] Output: Response data sent to the server

[0435] Specific behavior: The device captures the answer data, converts it to JSON format (e.g. answer_data = JSON.stringify(captured_answer)), and then sends it to the server using the fetch API.

[0436] Step 7:

[0437] This is a procedure in which the emotion engine built into the device collects the user's emotion data.

[0438] Input: User's facial expression and tone of voice

[0439] Output: Parsed emotion data

[0440] Specific operation: The device uses OpenCV to capture camera images and analyze facial features. It also uses a microphone to record voice tone and uses TensorFlow's voice analysis model to determine emotions.

[0441] Step 8:

[0442] This is the procedure by which the terminal transmits emotion data to the server.

[0443] Input: Collected emotion data

[0444] Output: Emotion data sent to the server

[0445] Specific operation: The device converts the emotion data into JSON format and sends it to the server using the fetch API.

[0446] Step 9:

[0447] This is a procedure for the server to analyze the received response data and emotion data.

[0448] Input: Answer data and emotion data

[0449] Output: Analysis results

[0450] Specific operation: The server uses Python's pandas library to convert the data into a data frame and evaluates comprehension and emotional state.

[0451] Step 10:

[0452] This is a procedure in which the server generates feedback based on the evaluation results.

[0453] Input: Analysis results and evaluation criteria

[0454] Output: Feedback message

[0455] What it does: Uses a generative AI model to generate feedback based on the user's comprehension and emotional state. For example, "Good job, but you seem a little nervous. Relax and try again."

[0456] Step 11:

[0457] This is a procedure for the server to send generated feedback to the terminal.

[0458] Input: Feedback message

[0459] Output: Feedback data ready to be sent to the device

[0460] Specific operation: The server converts the generated feedback message into JSON format and sends it to the terminal using a RESTful API.

[0461] Step 12:

[0462] This is a procedure for displaying the feedback received by the terminal on the user interface.

[0463] Input: Feedback data sent from the server

[0464] Output: Feedback message displayed in the user interface

[0465] What it does: It uses JavaScript to parse the feedback data and displays it in the user interface using HTML and CSS. Example: document.getElementById('feedback').innerText = json_data['feedback_text'];

[0466] The above is a detailed explanation of the specific processing steps of the program of this system.

[0467] (Application example 2)

[0468] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0469] Conventional electronic payment services do not optimize the payment process by taking into account the user's emotional state, which can lead to feelings of tension or impatience that can worsen the payment experience. Furthermore, because feedback based on the user's emotional state is not provided, it is difficult for users to receive advice or reassurance appropriate to their emotional state. The objective of the present invention is to solve these problems and enable users to make electronic payments in a more relaxed environment.

[0470] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0471] In this invention, the server includes means for collecting past data from a database, means for generating new data based on the collected past data, and means for transmitting the generated data to the terminal, thereby making it possible to provide feedback based on the user's level of understanding and emotional state.

[0472] "Historical Data" refers to information previously collected and stored by the System.

[0473] A "database" refers to a system that stores and manages structured information in an organized manner.

[0474] "Collection methods" refers to the methods and techniques by which data is obtained from databases and other sources.

[0475] "New data" refers to new information generated based on existing data.

[0476] "Means of generation" refers to methods and technologies for creating new data based on collected data.

[0477] "Terminal" refers to equipment or devices that a user operates.

[0478] "Transmission means" refers to the method or technology by which data or information is sent to another device or server.

[0479] "Operation data" refers to information that records the operations and inputs performed by a user using a terminal.

[0480] A "server" refers to a computer that processes data and provides services over a network.

[0481] "Analysis" refers to the process of analyzing collected data or information to arrive at a particular conclusion or result.

[0482] "Level of understanding" refers to an index that indicates how well the user understands the content.

[0483] "User emotion" refers to the emotional state a user has at a particular moment.

[0484] "Feedback" refers to responses or advice provided based on a user's actions or status.

[0485] "Camera" refers to a device that captures images and converts them into digital data.

[0486] "Facial expression capture" refers to the process of recording a user's facial expressions using a camera.

[0487] A "microphone" refers to a device that collects sound and converts it into digital data.

[0488] "Tone of voice analysis" refers to the process of analyzing voice data collected by a microphone to identify a user's emotional state.

[0489] "Relaxation advice" refers to instructions or advice to help the user relax.

[0490] The present invention is a system that optimizes the electronic payment process by taking into account the emotional state of the user. This system is composed of a server and a terminal, and is characterized by analyzing the user's emotions and dynamically adjusting the payment process and feedback based on the results.

[0491] The program for implementing the present invention operates as follows.

[0492] First, the server collects past data from a database. This data serves as the basis for generating new data. Next, the generated data is sent to the device. The device acts as a device that accepts user operations, using a camera and microphone to capture the user's facial expressions and tone of voice. This data is sent to the server along with the operation data.

[0493] The server analyzes the received data and evaluates the user's level of understanding and emotional state. The analysis module uses EmotionRecognizer (an emotion analysis library), which takes emotional state into account in addition to traditional methods for evaluating comprehension. The server generates feedback based on these evaluation results and sends it to the terminal. If the user is relaxed, a simple payment method (face recognition) is used, but if the user is nervous, additional confirmation steps (OTP, password entry) are applied. In this way, the payment process is dynamically adjusted according to the user's emotional state.

[0494] The feedback includes advice to encourage relaxation and is displayed to the user through the terminal. This feedback is generated by the FeedbackGenerator (feedback generation module).

[0495] For example, if a user is nervous about purchasing a high-value item, the system adds redundant confirmation steps. On the other hand, when making everyday small payments, facial recognition allows for quicker payments. This allows users to experience the optimal payment process based on their emotional state.

[0496] Use the following prompt for the generative AI model:

[0497] "Generate the following code: A Python program that analyzes emotions from face captures and dynamically determines the electronic payment method accordingly."

[0498] This invention provides a smoother and more secure electronic payment experience that takes into account the user's emotional state.

[0499] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0500] Step 1:

[0501] The server collects historical data from a database.

[0502] Input: Database query

[0503] Data manipulation: Extracting target data based on a query

[0504] Output: Collected historical data

[0505] Step 2:

[0506] The server generates new data based on the collected past data.

[0507] Input: Collected historical data

[0508] Data calculation: Applying new data generation logic using models

[0509] Output: Newly generated data

[0510] Step 3:

[0511] The server transmits the generated data to the terminal.

[0512] Input: Newly generated data

[0513] Data processing: Convert data into a format that can be sent (e.g., JSON format)

[0514] Output: Data packets sent

[0515] Step 4:

[0516] Based on the received data, the terminal displays an interface that the user can operate.

[0517] Input: Data packet sent

[0518] Data processing: Parsing incoming data and UI rendering

[0519] Output: A controllable interface

[0520] Step 5:

[0521] The user performs operations through the terminal, and the terminal collects the operation data.

[0522] Input: User actions (e.g. taps, text input)

[0523] Data processing: Convert user operations into log data

[0524] Output: Operation data

[0525] Step 6:

[0526] The device uses a camera and microphone to capture the user's facial expressions and tone of voice to collect emotional data.

[0527] Input: Camera video and audio input

[0528] Data Computing: Facial Expression and Voice Analysis

[0529] Output: Emotion data

[0530] Step 7:

[0531] The terminal transmits operation data and emotion data to the server.

[0532] Input: Operational data and emotional data

[0533] Data processing: Convert data into a format that can be sent (e.g., JSON format)

[0534] Output: Data packets sent

[0535] Step 8:

[0536] The server analyzes the received data and evaluates the user's comprehension and emotional state.

[0537] Input: Operational data and emotional data

[0538] Data Computation: Understanding and Emotion Assessment Using Data Analysis Models

[0539] Output: Comprehension and emotion evaluation results

[0540] Step 9:

[0541] The server generates feedback based on the evaluation results and transmits it to the terminal.

[0542] Input: Comprehension and emotion evaluation results

[0543] Data Calculation: Applying feedback generation logic

[0544] Output: Generated feedback data

[0545] Step 10:

[0546] The terminal displays the received feedback and provides it to the user.

[0547] Input: Feedback data

[0548] Data processing: Rendering received data to the UI

[0549] Output: Feedback display

[0550] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0551] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0552] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0553] [Second embodiment]

[0554] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0555] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0556] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0557] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0558] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0559] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0560] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0561] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0562] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0563] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0564] In the smart glasses 214, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0565] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0566] This invention is a system for realizing efficient and uniform education by collecting past question data, generating new questions, and providing them to users. This system is composed of the following phases.

[0567] 1. Problem generation phase

[0568] The server collects past exam data from a database, for example, across multiple subjects such as math, English, and history.

[0569] The server generates new questions based on the collected question data. At this time, the difficulty and format of the questions can be specified using the generation AI.

[0570] 2. Problem Delivery Phase

[0571] The server packets the generated problem in XML or JSON format and sends it to the terminal.

[0572] The device displays the received questions on a user interface that is designed to make it easy for users to answer intuitively.

[0573] 3. Response Phase

[0574] The user answers questions using the device, such as multiple-choice questions in a quiz format or essay-style math questions.

[0575] The device converts the user's response data into XML or JSON format and sends it to the server.

[0576] 4. Understanding Assessment Phase

[0577] The server receives the response data sent by the user and passes it to the analysis module.

[0578] The server's AI analyzes the answer data, determines whether the answer is correct, and evaluates the level of understanding. In this process, the accuracy of the answer, the time it takes, and the answering method are used as evaluation criteria.

[0579] 5. Feedback Phase

[0580] The server generates feedback for the user based on the analysis results, including the percentage of correct answers, explanations for incorrect answers, and suggestions for additional study.

[0581] The server sends the generated feedback to the terminal, which displays the feedback on a user interface.

[0582] Users can check the feedback and use it to improve their learning progress.

[0583] Specific examples

[0584] Problem generation phase:

[0585] For example, the server collects equation problems from the past 10 years from a database of mathematical problems.

[0586] The server generates new equation problems based on the collected data, for example, by setting the difficulty level to beginner, intermediate, or advanced and creating different types of problems.

[0587] Problem Delivery Phase:

[0588] The server generates equation problems and packets them to the user's tablet.

[0589] The device receives these questions and displays them on the tablet screen, laid out to make it easy for users to see and answer them.

[0590] Response Phase:

[0591] The user answers the displayed equation questions using a tablet, for example, by writing the equations on an answer sheet.

[0592] The terminal converts the answers written by the user into digital data and sends it to the server.

[0593] Comprehension assessment phase:

[0594] The server analyzes the received answer data, and the AI ​​determines whether the answer is correct or not. For example, it takes into account not only the accuracy of the answer but also the time it took to answer.

[0595] The server compares the user's answer data with the correct answer data to evaluate the level of understanding.

[0596] Feedback Phase:

[0597] Based on the evaluation results, the server generates feedback such as, "This part was not answered correctly, so please read the explanation below and try again."

[0598] The server sends the generated feedback to the tablet, which displays the feedback on its user interface.

[0599] The user reviews the feedback and re-studies the question to gain a better understanding of the incorrect answers.

[0600] In this way, the present invention can standardize the quality of education and reduce labor costs. Furthermore, by providing feedback according to each user's level of understanding, it can effectively support the improvement of each user's academic ability.

[0601] The processing flow will be explained below.

[0602] Step 1:

[0603] The server connects to the database and collects past exam data. For example, it can collect exam data for a specific subject (such as mathematics or English).

[0604] Step 2:

[0605] New problems are generated based on past problem data collected by the server. At this time, a generation AI is used to specify the difficulty and format of the problem. For example, equation problems with beginner, intermediate, and advanced difficulty levels can be created.

[0606] Step 3:

[0607] The server converts the generated questions into XML or JSON format and creates a data packet for distribution.

[0608] Step 4:

[0609] The server sends the data packets to the terminal via the Internet.

[0610] Step 5:

[0611] The device analyzes the received question data and displays the questions on the user interface. The layout is adjusted to make it easier for the user to answer intuitively.

[0612] Step 6:

[0613] The user uses the device to answer the displayed questions, for example by entering mathematical expressions using the touchscreen or by selecting answers from multiple options.

[0614] Step 7:

[0615] The device converts the user's response data into XML or JSON format and sends it to the server.

[0616] Step 8:

[0617] The server receives the response data sent by the user and passes it to the analysis module.

[0618] Step 9:

[0619] The server's generated AI analyzes the answer data and judges whether it is correct or not, for example, by evaluating the accuracy of the formula, the answer method, and the answer time.

[0620] Step 10:

[0621] The server evaluates the user's level of understanding and generates feedback based on that evaluation, including, for example, explanations for incorrect answers and additional practice questions.

[0622] Step 11:

[0623] The server generates feedback and sends it to the device.

[0624] Step 12:

[0625] The device analyzes the received feedback and displays it in the user interface, providing visual explanations and additional questions to help users understand the process.

[0626] Step 13:

[0627] Users can review the feedback and use it to improve their learning, retrying the questions if necessary to further their understanding.

[0628] Through the above steps, the present invention can provide education efficiently and effectively, and improve the academic ability of users.

[0629] Example 1

[0630] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0631] Conventional educational systems have the problem of providing uniform questions that cannot adequately address the individual learner's level of understanding or adaptability, resulting in reduced learning effectiveness. They also require a large amount of human resources, making it difficult to provide efficient educational support. Furthermore, they lack a mechanism for effectively utilizing past learning data, making it difficult to provide appropriate feedback based on the learner's progress. To solve these issues, a system is needed that can automatically generate new questions based on collected data and provide feedback tailored to each learner's individual level of understanding.

[0632] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0633] In this invention, the server includes a means for collecting past questions from a database, a means for inputting prompts to the AI ​​model to generate new questions, and a means for transmitting the generated questions to the terminal. This makes it possible to automatically generate new questions using past learning data and provide appropriate feedback according to the individual learner's level of understanding.

[0634] The "means for collecting past questions from a database" is a means having a function for acquiring past question data stored in a database.

[0635] A "means for inputting prompts to an AI model to generate new problems" is a means for inputting instructions to a generative AI model, thereby generating new problems.

[0636] "Means for transmitting generated problems to a terminal" means means for transmitting problems generated by an AI model to a terminal in a digital format (e.g., JSON or XML).

[0637] The "means for the user to answer questions via the terminal" refers to a means for the user to answer questions displayed via a user interface.

[0638] The "means for transmitting user response data to the server" refers to a means for transmitting the response data entered by the user to the server in digital form.

[0639] The "means for analyzing the user's response data and evaluating the level of understanding" is a means for analyzing the response data sent by the user and evaluating the user's level of understanding based on the results.

[0640] The "means for generating feedback based on comprehension" is a means for generating feedback for the user based on the results of the assessed comprehension.

[0641] The "means for transmitting the generated feedback to the terminal" refers to means for transmitting the generated feedback to the terminal and displaying it to the user.

[0642] "Means for generating new problems using an AI model" refers to means for automatically generating new problems based on past problem data using an artificial intelligence model.

[0643] A "prompt" is an input sentence that provides instructions to an AI model and includes detailed conditions such as the type and difficulty of the problem to be generated.

[0644] The present invention is a system that realizes efficient and uniform education by collecting past question data, generating new questions, and providing them to users. This system is realized using the following hardware and software.

[0645] Hardware used

[0646] 1. Server: A computer system that connects to a database and collects, analyzes, and transmits data.

[0647] 2. Device: The device (e.g., tablet, smartphone, computer) on which the user answers the questions and receives feedback.

[0648] Software used

[0649] 1. Database Management System: Database software such as MySQL, MongoDB, etc.

[0650] 2. Generative AI models: For example, generative AI models such as GPT-4.

[0651] 3. Programming languages ​​and frameworks: Python, Flask, Django, etc.

[0652] System Operation

[0653] The server first collects past problem data from a database, using a method such as an SQL query to retrieve data that matches specific conditions. For example, the following query is used:

[0654] SELECT FROM question_data WHERE subject = 'Mathematics' AND year >= 2013

[0655] Next, the server uses the generative AI model to generate new questions. The server then inputs prompts like the following to the generative AI model to specify the type and difficulty of the questions desired:

[0656] Please set the difficulty level of this problem to beginner. Please generate other problems as well.

[0657] The generated new question is converted into XML or JSON format by the server and sent to the terminal. For example, the following JSON format data is sent:

[0658] {"Question": "Solve the equation", "Difficulty": "Beginner"}

[0659] The terminal displays the questions received from the server on the user interface. The user answers the questions using a keyboard or touch screen. The terminal converts the user's answers into digital data and sends them back to the server.

[0660] The server analyzes the response data received from the user and evaluates the level of understanding. The analysis uses a generative AI model and an analysis module to evaluate the accuracy of the response, response time, response method, etc. The server then generates feedback based on the evaluation results and sends it to the user.

[0661] This feedback might look something like this:

[0662] This part was not answered correctly, so please read the explanation below and try again.

[0663] The device displays the received feedback on the user interface for the user to confirm, allowing the user to check their own understanding and engage in further learning.

[0664] This system makes it possible to generate new questions using past learning data and provide feedback based on individual levels of understanding, thereby providing efficient and effective educational support.

[0665] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0666] Step 1:

[0667] The server collects past problem data from a database.

[0668] Specifically, the server connects to a database management system (e.g., MySQL) and executes an SQL query such as the following:

[0669] Input: Database connection information, query conditions (e.g., subject "Mathematics", year "2013 or later").

[0670] Output: A set of historical question data.

[0671] SELECT FROM question_data WHERE subject = 'Mathematics' AND year >= 2013

[0672] The server records the number and contents of the data in a log to confirm that the acquired data is correct.

[0673] Step 2:

[0674] New questions are generated based on the problem data collected by the server.

[0675] Specifically, the server inputs a prompt to the generative AI model (e.g., GPT-4) and instructs it to generate a problem.

[0676] Input: Past question data, prompt text.

[0677] Output: The new problem that was generated.

[0678] Example prompt sentence:

[0679] Please set the difficulty level of this problem to beginner. Please generate other problems as well.

[0680] The server receives the generated problem, verifies its contents, and stores it in a log.

[0681] Step 3:

[0682] The server packets the generated problem in XML or JSON format and sends it to the terminal.

[0683] Specifically, the server converts the problem into JSON format as shown below and sends it to the terminal as an HTTP POST request.

[0684] Input: Generated problem data.

[0685] Output: Sending status to terminal.

[0686] {"Question": "Solve the equation", "Difficulty": "Beginner"}

[0687] The server checks the HTTP request status code (e.g., 200 OK) and logs the successful submission.

[0688] Step 4:

[0689] The problem received by the terminal is displayed on the user interface.

[0690] Specifically, the device parses the received JSON data and constructs a layout that is intuitive for the user.

[0691] Input: The problem data received from the server.

[0692] Output: The problem as displayed to the user.

[0693] The terminal generates a timestamp to record when the user begins to answer.

[0694] Step 5:

[0695] The user answers the questions using the device.

[0696] Specifically, the user inputs the answer using a keyboard or touch screen.

[0697] Input: The user's answer.

[0698] Output: The entered answer data.

[0699] The terminal saves the user's input and proceeds to the next step when the answer button is pressed.

[0700] Step 6:

[0701] The terminal transmits the user's response data to the server.

[0702] Specifically, the device converts the answer data into JSON format as shown below and sends it to the server as an HTTP POST request.

[0703] Input: User's answer data.

[0704] Output: Status of submission to server.

[0705] {"Answer": "x = 2", "User ID": "12345"}

[0706] The terminal receives a response from the server and confirms that the transmission was successful.

[0707] Step 7:

[0708] The server receives the response data sent by the user and passes it to the analysis module.

[0709] Specifically, the server reads the received data using a Python library (e.g., Pandas) and analyzes it.

[0710] Input: User response data.

[0711] Output: Analysis results.

[0712] The server logs the analysis results in a format similar to the following: {"Correct Answer": True, "Answer Time": "5 minutes"}.

[0713] Step 8:

[0714] The server evaluates the level of understanding based on the response data.

[0715] Specifically, the data is analyzed using an AI model and evaluated based on criteria such as correctness and response time.

[0716] Input: Parsed response data.

[0717] Output: Comprehension assessment results.

[0718] The server stores the evaluation results in a database.

[0719] Step 9:

[0720] The server generates feedback based on the results of the comprehension assessment and sends it to the terminal.

[0721] Specifically, the server uses an AI model to generate feedback and sends it to the device in JSON format as follows:

[0722] Input: Comprehension assessment results.

[0723] Output: The generated feedback.

[0724] Generate feedback similar to the following example:

[0725] This part was not answered correctly, so please read the explanation below and try again.

[0726] The server logs the feedback sending status.

[0727] Step 10:

[0728] The terminal displays the feedback on the user interface for the user to confirm.

[0729] Specifically, the terminal parses the received feedback and displays it to the user.

[0730] Input: Feedback received from the server.

[0731] Output: The feedback that is displayed to the user.

[0732] The terminal records the timestamp when the user confirms the feedback and confirms the content of the re-learning.

[0733] The above is the specific processing flow of this system.

[0734] (Application example 1)

[0735] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0736] In conventional factory robot training programs, operator skill evaluations and feedback are uniform, making it difficult to improve training according to individual levels of understanding.There is a need for a system that can efficiently generate technical problems actually required in the field, evaluate the operator's level of understanding based on those problems, and provide appropriate feedback.

[0737] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0738] In this invention, the server includes means for collecting past question data from a database, means for generating new questions using a generative AI model, means for transmitting the generated questions to a terminal, means for a user to answer the generated questions using a tablet and transmit the answer data to the server, means for analyzing the answer data and evaluating the level of understanding based on accuracy, answer time, and operation procedures, and means for generating feedback based on the analysis results and providing the user with an opportunity to re-study. This enables efficient evaluation and feedback tailored to individual levels of understanding in a factory robot operator training program.

[0739] "Past problems" are data on problems and training tasks that the user has previously tackled.

[0740] A "database" is an information system for managing and storing past questions and answer data.

[0741] "New problems" are new training problems generated using a generative AI model based on past problem data.

[0742] A "terminal" is a digital device through which a user accesses and responds to training programs and questions.

[0743] A "server" is a centralized computer system that manages problems and data, analyzes them, generates feedback, etc.

[0744] An "analysis module" is a program that runs on a server and analyzes the user's response data to evaluate their level of understanding.

[0745] A "generative AI model" is an artificial intelligence model that generates new problems based on past problem data.

[0746] "Feedback" is information that is generated based on the user's answers and includes an evaluation of the user's level of understanding and suggestions for relearning.

[0747] A "tablet" is a portable computing device that allows a user to answer training questions and receive feedback.

[0748] "Question difficulty" refers to the level of difficulty of the questions to be generated.

[0749] "Answer time" is the time it takes the user to answer the question.

[0750] An "operation procedure" is a process of a series of actions performed by a user when answering a question.

[0751] "Relearning" is the process by which a user re-learns based on feedback.

[0752] A "training program" is a series of educational courses that factory robot operators go through to acquire specific skills and knowledge.

[0753] The "evaluation criteria" are scales or indicators used to evaluate the user's response data.

[0754] The present invention is a system for evaluating a factory robot operator training program and providing feedback for improvement. This system collects past training question data, generates new questions, and provides them to users, thereby achieving efficient and uniform technical education. A specific embodiment of this system will be described below.

[0755] 1. Problem generation phase

[0756] The server collects past training problem data from a database containing problems from various technical fields (e.g., welding, assembly, and maintenance). Based on the collected data, the server generates new problems using a generative AI model (e.g., OpenAI's GPT-3). The difficulty and format of the generated problems can be specified as needed.

[0757] Specific examples

[0758] The server collects data on past welding technology problems and inputs the following prompt into GPT-3: "Generate new welding technology problems in the following format: medium difficulty, solution time within 30 minutes, use the following materials..." to generate new problems.

[0759] 2. Problem Delivery Phase

[0760] The generated questions are packetized in JSON format and sent to the terminal. The terminal displays the received questions on a user interface, which is designed to make the questions easy for users to see and answer intuitively.

[0761] Specific examples

[0762] The generated welding problems are packetized in JSON format and sent to a tablet, where they are displayed and the user works on them.

[0763] 3. Response Phase

[0764] The user answers questions using a terminal. The answer data is converted into digital data by the terminal and sent to the server. The server analyzes the received answer data and evaluates the level of understanding.

[0765] 4. Understanding Assessment Phase

[0766] The server's analysis module analyzes the response data and evaluates the level of understanding based on accuracy, response time, and operation procedures. The evaluation results are generated by the analysis module.

[0767] Specific examples

[0768] The server analyzes the user's response data and generates feedback such as, "The heat distribution during welding is not uniform, resulting in an improper joint. Please re-learn the cause and how to improve it."

[0769] 5. Feedback Phase

[0770] Feedback is generated based on the analysis results and sent to the device, which then displays the feedback on the user interface, giving the user an opportunity to re-learn.

[0771] Hardware and Software

[0772] Hardware: Smartphones and tablets (e.g. iPad)

[0773] Software: MySQL (database), JSON (data format), generative AI model (GPT-3), front-end (React Native)

[0774] Prompt Sentence Examples

[0775] Create a new welding problem in the following format:

[0776] Difficulty: Intermediate

[0777] Answer time: within 30 minutes

[0778] Materials used: iron, aluminum

[0779] In this way, the system of the present invention can efficiently implement training programs for factory robot operators and provide evaluations and feedback according to each operator's level of understanding, thereby effectively supporting the improvement of operator technical skills.

[0780] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0781] Step 1:

[0782] The server collects past question data from a database. The database stores past question data for each training program, and the server accesses and collects this data. The input is a database query, and the output is past question data. Specifically, the server accesses the database using an SQL query to retrieve the required data.

[0783] Step 2:

[0784] The server generates new problems using a generative AI model based on the collected past problem data. The input is the collected past problem data and the specified problem difficulty and format, and the output is the generated new problem. Specifically, a prompt sentence is input to the generative AI model (e.g., GPT-3) to generate a new problem.

[0785] Step 3:

[0786] The server packets the generated new problem in JSON format and sends it to the terminal. The input is the generated new problem, and the output is the problem data converted to JSON format. Specifically, the problem data is serialized in JSON format and sent to the terminal over the network.

[0787] Step 4:

[0788] The terminal displays the received problem data on the user interface. The input is the received JSON format problem data, and the output is the problem displayed to the user. Specifically, it deserializes the JSON data and renders the problem content on the screen.

[0789] Step 5:

[0790] The user answers questions through the terminal. The input is the answer selected or written by the user, and the output is the user's answer data. Specifically, the user enters the answer using the user interface and presses the send button to send the answer to the terminal.

[0791] Step 6:

[0792] The terminal sends the user's answer data to the server. The input is the user's answer data, and the output is the data to be sent to the server. Specifically, the answer data is converted into JSON format and sent to the server.

[0793] Step 7:

[0794] The server analyzes the received answer data and evaluates the level of comprehension. The input is the user's answer data, and the output is the comprehension evaluation result. Specifically, the server uses the analysis module to evaluate the accuracy of the answer, the answer time, and the operation procedure.

[0795] Step 8:

[0796] The server generates feedback based on the analysis results and sends it to the device. The input is the comprehension assessment result, and the output is the generated feedback data. Specifically, the server uses a feedback generation algorithm to create feedback for the user, converts it into JSON format, and sends it to the device.

[0797] Step 9:

[0798] The terminal displays the received feedback on the user interface. The input is the received feedback data, and the output is the feedback content displayed to the user. Specifically, the terminal deserializes the feedback data and renders it on the screen.

[0799] Through the above process steps, the system of the present invention can support a user's technical training program and provide efficient and individualized assessment and feedback according to the user's level of understanding.

[0800] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0801] The present invention is a system that provides education that takes into account not only the user's level of understanding but also their emotional state by combining a conventional education system with an emotion engine. This system is composed of the following phases.

[0802] 1. Problem generation phase

[0803] The server connects to a database and collects past exam data, such as questions for subjects like math, English, and history.

[0804] New questions are generated based on the question data collected by the server. Using the generation AI, questions can be generated by specifying their difficulty and format.

[0805] 2. Problem Delivery Phase

[0806] The server converts the generated questions into XML or JSON format and creates a data packet for distribution.

[0807] The server sends the data packets to the terminal via the Internet.

[0808] 3. Response Phase

[0809] The device analyzes the received question data and displays the questions on the user interface, which is designed to be laid out in a way that makes it easy for users to answer intuitively.

[0810] The user answers the questions using the device, for example by entering mathematical expressions using the touchscreen of a tablet or by selecting multiple choices.

[0811] 4. Emotional Data Collection Phase

[0812] An emotion engine built into the device analyzes the user's facial expressions and tone of voice to collect emotion data. For example, a camera can capture facial expressions and a microphone can analyze tone of voice.

[0813] The emotion data collected by the device is converted into XML or JSON format and sent to the server.

[0814] 5. Understanding Assessment Phase

[0815] The server receives the response data and emotion data sent by the user and passes them to the analysis module.

[0816] The server's generated AI analyzes the answer data, determines whether the answer is correct, and evaluates the level of understanding. It also evaluates the accuracy of the answer, the time it took to answer, and the answering method.

[0817] Furthermore, data from the emotion engine is taken into consideration and the emotional state of the user when answering is taken into account in the evaluation.

[0818] 6. Feedback generation phase

[0819] The server generates feedback based on the analysis results, taking into account the user's level of understanding and emotional state. For example, if a user has a high level of understanding but is nervous, the server may generate feedback such as "Good job! Stay calm and try again next time."

[0820] The server generates feedback and sends it to the device.

[0821] 7. Feedback display phase

[0822] The device analyzes the received feedback and displays it in the user interface. The feedback may include additional questions, explanations, or psychological support messages.

[0823] Users can review the feedback and use it to improve their learning, for example by re-studying incorrect answers to deepen their understanding.

[0824] Specific examples

[0825] Problem generation phase:

[0826] The server collects calculus problems from the past 10 years from a mathematical problem database.

[0827] The server generates new problems based on this data, such as "Beginner: Simple Differential Problems," "Intermediate: Definite Integral Problems," and "Advanced: Complex Integral Problems."

[0828] Problem Delivery Phase:

[0829] The server generates a problem packet and sends it to the user's tablet.

[0830] The device receives the question and displays it on the tablet screen.

[0831] Response Phase:

[0832] The user answers the displayed calculus problems using a tablet, for example, by writing mathematical formulas using a stylus pen.

[0833] The terminal converts the user's answers into digital data and sends it to the server.

[0834] Sentiment data collection phase:

[0835] The device's built-in camera captures the user's facial expressions, while the microphone analyzes the tone of the voice, detecting, for example, facial expressions that indicate impatience or a tense voice.

[0836] The terminal transmits this emotion data to the server.

[0837] Comprehension assessment phase:

[0838] The server analyzes the received answer data and evaluates whether the answer was correct or incorrect and the answer time.

[0839] Furthermore, emotional data is analyzed and the user's emotional state while answering is taken into account in the evaluation.

[0840] Feedback generation phase:

[0841] The server generates feedback based on the evaluation, including a message such as "Well done, but you seem a little nervous, so please relax and try again."

[0842] The server generates feedback and sends it to the user's tablet.

[0843] Feedback display phase:

[0844] The device displays the received feedback, which can include explanations based on comprehension and relaxation advice based on emotional data.

[0845] Users can review the feedback and use it to improve their learning experience next time.

[0846] In this way, the present invention can provide more comprehensive and effective learning support by providing education while taking into consideration the user's emotions.

[0847] The processing flow will be explained below.

[0848] Step 1:

[0849] The server connects to the database and collects past exam data, such as questions for each subject, such as mathematics, English, and history.

[0850] Step 2:

[0851] Based on past question data collected by the server, new questions are generated using a generation AI. For example, questions can be created by specifying a specific level of difficulty (beginner, intermediate, advanced) or format (multiple choice, written).

[0852] Step 3:

[0853] The server converts the generated questions into XML or JSON format and creates a data packet for distribution.

[0854] Step 4:

[0855] The server sends the data packets to the terminal via the Internet.

[0856] Step 5:

[0857] The device analyzes the received question data and displays the questions on the user interface. For example, the questions are laid out visually on a tablet screen.

[0858] Step 6:

[0859] The user uses the device to answer the displayed questions, for example by entering a mathematical formula or selecting multiple choices using the touchscreen.

[0860] Step 7:

[0861] The device converts the user's response data into XML or JSON format and sends it to the server.

[0862] Step 8:

[0863] The device's built-in emotion engine captures the user's facial expressions and tone of voice to collect emotion data. For example, the camera detects facial expressions and the microphone records tone of voice.

[0864] Step 9:

[0865] The emotion data collected by the device is converted into XML or JSON format and sent to the server.

[0866] Step 10:

[0867] The server receives the response data and emotion data sent by the user and passes them to the analysis module.

[0868] Step 11:

[0869] The server's generated AI analyzes the answer data and determines whether it is correct or incorrect and evaluates the level of understanding. For example, the accuracy of the answer, the answering time, and the answering method are used as evaluation criteria.

[0870] Step 12:

[0871] The server then performs additional analysis of the emotional data to assess the user's emotional state while answering the questions, such as analyzing facial expressions and tone of voice that indicate impatience or nervousness.

[0872] Step 13:

[0873] The server generates feedback based on the level of understanding and emotional state, such as a message like "Good job, but you seem a little nervous, so please relax and try again."

[0874] Step 14:

[0875] The server generates feedback and sends it to the device.

[0876] Step 15:

[0877] The device analyzes the received feedback and displays it in the user interface. For example, the feedback may include specific explanations based on the user's level of understanding or relaxation advice based on the user's emotional state.

[0878] Step 16:

[0879] Users can check the feedback and use it to improve their learning the next time. For example, they can re-study the incorrect answers and put the advice from the emotion engine into practice, improving their learning effectiveness.

[0880] Through the above steps, the present invention can comprehensively evaluate the user's understanding and emotions and provide optimal feedback.

[0881] Example 2

[0882] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0883] Conventional educational systems are limited to assessing users' learning progress and level of understanding, and lack support that takes into account the user's emotional state. This makes it difficult to improve learning efficiency and maintain motivation, potentially affecting long-term learning outcomes. Furthermore, the quality and variety of questions generated are limited, making it difficult to meet the diverse learning needs of users. It was necessary to solve these issues and create a system that provides more comprehensive and effective learning support.

[0884] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for collecting past data from a database, means for generating new questions based on the collected past data, and means for transmitting the generated questions to the terminal. This enables comprehensive learning support that takes into account not only the user's learning progress but also their emotional state. Specifically, new questions can be generated by specifying the difficulty level and format of the questions using a generative AI model, thereby meeting the diverse learning needs of users. Furthermore, by analyzing the user's response data and adding emotional data to the analysis, not only the user's level of understanding but also their emotional state during learning can be reflected in the evaluation. This allows for more personalized feedback to the user, improving learning efficiency and maintaining motivation.

[0885] I understand. Below are definitions of important terms included in the claims.

[0886] "Past data" refers to previously collected information related to the educational subject, such as question and answer data stored in a database.

[0887] A "database" is an information system used to efficiently store, manage, and search large amounts of data.

[0888] "Means for generating new questions" refers to the technology and algorithms used to generate new questions for users to answer based on collected past data.

[0889] A "generative AI model" refers to a machine learning model that uses artificial intelligence technology to generate new problems.

[0890] "Device" refers to an electronic device, such as a computer, tablet, or smartphone, that allows a user to access and answer questions.

[0891] "User" refers to an individual who uses the educational system to learn.

[0892] "Server" refers to the central system that collects data from the database, generates questions, and evaluates answers.

[0893] "Answer data" is digital information of the answer to the question that the user inputs through the terminal.

[0894] "Emotion data" is data that indicates the emotional state of the user analyzed from facial expressions, tone of voice, and the like.

[0895] "Evaluation" refers to the process of analyzing the user's response data and emotional data to determine the user's level of understanding and emotional state.

[0896] "Feedback" refers to study advice and instructions on next steps provided to the user based on the evaluation results.

[0897] "Analysis" refers to the process of extracting meaningful information from collected data and determining the user's level of understanding and emotional state.

[0898] The above are definitions of important terms contained in the claims.

[0899] The present invention is a system that provides education that takes into account not only the user's level of understanding but also their emotional state. This system aims to improve the efficiency and effectiveness of learning by combining a conventional education system with an emotional engine. The following describes how to specifically implement the invention.

[0900] System Overview

[0901] This system consists of a server, terminals, and users. The roles of each are as follows:

[0902] Server: Collects data from the database, generates new questions, and sends them to the device. It also analyzes the user's response data and emotional data, evaluates their understanding and emotional state, and generates feedback.

[0903] Terminal: Provides an interface for users to answer questions, collects answer data and emotion data, and sends them to the server. Receives feedback and displays it to the user.

[0904] User: Answers questions and receives feedback via the device.

[0905] Hardware and Software Configuration

[0906] The server is configured using, for example, a Python environment and a MySQL database, and uses a generative AI model such as OpenAI's GPT-3 to generate problems.

[0907] The devices are electronic devices such as tablets, smartphones, and PCs, and the user interface is built using JavaScript, HTML, and CSS. Emotion data is collected using libraries such as OpenCV and TensorFlow.

[0908] Users use these terminals to carry out learning activities.

[0909] Processing flow

[0910] 1. Problem generation:

[0911] The server connects to a MySQL database and collects past problem data using SQL queries.

[0912] Based on the collected data, a generative AI model (e.g., GPT-3) is used to generate new problems, with prompts being entered to generate the problems.

[0913] The generated questions are converted into JSON format and sent to the terminal.

[0914] Example prompt sentence:

[0915] "Generate beginner calculus problems"

[0916] "The user has a high level of understanding but is nervous, so generate feedback with advice to relax."

[0917] 2. Question delivery and answers:

[0918] The terminal analyzes the received problem data and displays it on the user interface.

[0919] The user enters mathematical expressions and answers using the tablet's touchscreen.

[0920] The terminal converts the user's answers into digital data and sends it to the server.

[0921] 3. Emotional Data Collection:

[0922] The device's built-in camera and microphone capture the user's facial expressions and tone of voice to collect emotional data.

[0923] Using OpenCV and TensorFlow, we analyze this data to determine the user's emotional state.

[0924] The collected emotion data is converted into JSON format and sent to the server.

[0925] 4. Comprehension assessment and feedback generation:

[0926] The server analyzes the response data and emotion data received using the Python pandas library to evaluate comprehension and emotional state.

[0927] Generative AI models are used to generate feedback based on the evaluation results, such as "Good job, but you seem a little nervous, so please relax and try again."

[0928] The generated feedback is sent to the device.

[0929] 5. Feedback display:

[0930] The terminal analyzes the received feedback and displays it on the user interface.

[0931] Users can review the feedback and use it to improve their learning experience next time.

[0932] In this way, the system of the present invention can simultaneously consider the user's level of understanding and emotional state, and provide more comprehensive and effective learning support.

[0933] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0934] Processing Steps

[0935] Step 1:

[0936] This is the procedure by which the server collects past data. Specifically, the server connects to a MySQL database and collects past problem data using SQL queries.

[0937] Input: MySQL database connection information and SQL query

[0938] Output: Problem data (past problem data)

[0939] What it does: The server uses Python's mysql-connector library to connect to a database named my_database, then executes the SQL query "SELECT FROM math_problems WHERE subject='calculus' AND year BETWEEN 2011 AND 2021;" to retrieve past problem data.

[0940] Step 2:

[0941] This is the procedure where the server generates new problems using a generative AI model. Based on past problem data collected by the server, new problems are generated using a generative AI model (e.g., GPT-3).

[0942] Input: Past problem data and prompts for the generative AI model

[0943] Output: New problem data

[0944] Specific operation: The server sends a prompt such as "Generate beginner calculus problems" to the OpenAI API based on the collected past problem data. The API responds with new problem data.

[0945] Step 3:

[0946] This is the procedure for sending questions generated by the server to the terminal. The generated question data is converted into JSON format and sent to the terminal via a RESTful API.

[0947] Input: Newly generated question data

[0948] Output: JSON data ready to be sent to the terminal

[0949] How it works: The server converts the generated question data into JSON format using Python's json library, and then sends the question data to the terminal using a RESTful API created using Flask.

[0950] Step 4:

[0951] This is the procedure in which the terminal analyzes the received question data and displays it on the user interface.

[0952] Input: Question data in JSON format sent from the server

[0953] Output: The problem as displayed in the user interface

[0954] Specific behavior: The device parses the JSON data using JavaScript and displays the problem in the user interface using HTML and CSS. Example: document.getElementById('problem').innerText = json_data['problem_text'];

[0955] Step 5:

[0956] This is the procedure by which the user answers the questions.

[0957] Input: The problem displayed on the terminal

[0958] Output: User response data

[0959] Specific actions: The user answers the displayed questions using the tablet's touchscreen, for example, by entering mathematical expressions using a stylus.

[0960] Step 6:

[0961] This is the procedure in which the terminal converts the user's answer into digital data and sends it to the server.

[0962] Input: User's handwritten answer or input

[0963] Output: Response data sent to the server

[0964] Specific behavior: The device captures the answer data, converts it to JSON format (e.g. answer_data = JSON.stringify(captured_answer)), and then sends it to the server using the fetch API.

[0965] Step 7:

[0966] This is a procedure in which the emotion engine built into the device collects the user's emotion data.

[0967] Input: User's facial expression and tone of voice

[0968] Output: Parsed emotion data

[0969] Specific operation: The device uses OpenCV to capture camera images and analyze facial features. It also uses a microphone to record voice tone and uses TensorFlow's voice analysis model to determine emotions.

[0970] Step 8:

[0971] This is the procedure by which the terminal transmits emotion data to the server.

[0972] Input: Collected emotion data

[0973] Output: Emotion data sent to the server

[0974] Specific operation: The device converts the emotion data into JSON format and sends it to the server using the fetch API.

[0975] Step 9:

[0976] This is a procedure for the server to analyze the received response data and emotion data.

[0977] Input: Answer data and emotion data

[0978] Output: Analysis results

[0979] Specific operation: The server uses Python's pandas library to convert the data into a data frame and evaluates comprehension and emotional state.

[0980] Step 10:

[0981] This is a procedure in which the server generates feedback based on the evaluation results.

[0982] Input: Analysis results and evaluation criteria

[0983] Output: Feedback message

[0984] What it does: Uses a generative AI model to generate feedback based on the user's comprehension and emotional state. For example, "Good job, but you seem a little nervous. Relax and try again."

[0985] Step 11:

[0986] This is a procedure for the server to send generated feedback to the terminal.

[0987] Input: Feedback message

[0988] Output: Feedback data ready to be sent to the device

[0989] Specific operation: The server converts the generated feedback message into JSON format and sends it to the terminal using a RESTful API.

[0990] Step 12:

[0991] This is a procedure for displaying the feedback received by the terminal on the user interface.

[0992] Input: Feedback data sent from the server

[0993] Output: Feedback message displayed in the user interface

[0994] What it does: It uses JavaScript to parse the feedback data and displays it in the user interface using HTML and CSS. Example: document.getElementById('feedback').innerText = json_data['feedback_text'];

[0995] The above is a detailed explanation of the specific processing steps of the program of this system.

[0996] (Application example 2)

[0997] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0998] Conventional electronic payment services do not optimize the payment process by taking into account the user's emotional state, which can lead to feelings of tension or impatience that can worsen the payment experience. Furthermore, because feedback based on the user's emotional state is not provided, it is difficult for users to receive advice or reassurance appropriate to their emotional state. The objective of the present invention is to solve these problems and enable users to make electronic payments in a more relaxed environment.

[0999] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1000] In this invention, the server includes means for collecting past data from a database, means for generating new data based on the collected past data, and means for transmitting the generated data to the terminal, thereby making it possible to provide feedback based on the user's level of understanding and emotional state.

[1001] "Historical Data" refers to information previously collected and stored by the System.

[1002] A "database" refers to a system that stores and manages structured information in an organized manner.

[1003] "Collection methods" refers to the methods and techniques by which data is obtained from databases and other sources.

[1004] "New data" refers to new information generated based on existing data.

[1005] "Means of generation" refers to methods and technologies for creating new data based on collected data.

[1006] "Terminal" refers to equipment or devices that a user operates.

[1007] "Transmission means" refers to the method or technology by which data or information is sent to another device or server.

[1008] "Operation data" refers to information that records the operations and inputs performed by a user using a terminal.

[1009] A "server" refers to a computer that processes data and provides services over a network.

[1010] "Analysis" refers to the process of analyzing collected data or information to arrive at a particular conclusion or result.

[1011] "Level of understanding" refers to an index that indicates how well the user understands the content.

[1012] "User emotion" refers to the emotional state a user has at a particular moment.

[1013] "Feedback" refers to responses or advice provided based on a user's actions or status.

[1014] "Camera" refers to a device that captures images and converts them into digital data.

[1015] "Facial expression capture" refers to the process of recording a user's facial expressions using a camera.

[1016] A "microphone" refers to a device that collects sound and converts it into digital data.

[1017] "Tone of voice analysis" refers to the process of analyzing voice data collected by a microphone to identify a user's emotional state.

[1018] "Relaxation advice" refers to instructions or advice to help the user relax.

[1019] The present invention is a system that optimizes the electronic payment process by taking into account the emotional state of the user. This system is composed of a server and a terminal, and is characterized by analyzing the user's emotions and dynamically adjusting the payment process and feedback based on the results.

[1020] The program for implementing the present invention operates as follows.

[1021] First, the server collects past data from a database. This data serves as the basis for generating new data. Next, the generated data is sent to the device. The device acts as a device that accepts user operations, using a camera and microphone to capture the user's facial expressions and tone of voice. This data is sent to the server along with the operation data.

[1022] The server analyzes the received data and evaluates the user's level of understanding and emotional state. The analysis module uses EmotionRecognizer (an emotion analysis library), which takes emotional state into account in addition to traditional methods for evaluating comprehension. The server generates feedback based on these evaluation results and sends it to the terminal. If the user is relaxed, a simple payment method (face recognition) is used, but if the user is nervous, additional confirmation steps (OTP, password entry) are applied. In this way, the payment process is dynamically adjusted according to the user's emotional state.

[1023] The feedback includes advice to encourage relaxation and is displayed to the user through the terminal. This feedback is generated by the FeedbackGenerator (feedback generation module).

[1024] For example, if a user is nervous about purchasing a high-value item, the system adds redundant confirmation steps. On the other hand, when making everyday small payments, facial recognition allows for quicker payments. This allows users to experience the optimal payment process based on their emotional state.

[1025] Use the following prompt for the generative AI model:

[1026] "Generate the following code: A Python program that analyzes emotions from face captures and dynamically determines the electronic payment method accordingly."

[1027] This invention provides a smoother and more secure electronic payment experience that takes into account the user's emotional state.

[1028] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1029] Step 1:

[1030] The server collects historical data from a database.

[1031] Input: Database query

[1032] Data manipulation: Extracting target data based on a query

[1033] Output: Collected historical data

[1034] Step 2:

[1035] The server generates new data based on the collected past data.

[1036] Input: Collected historical data

[1037] Data calculation: Applying new data generation logic using models

[1038] Output: Newly generated data

[1039] Step 3:

[1040] The server transmits the generated data to the terminal.

[1041] Input: Newly generated data

[1042] Data processing: Convert data into a format that can be sent (e.g., JSON format)

[1043] Output: Data packets sent

[1044] Step 4:

[1045] Based on the received data, the terminal displays an interface that the user can operate.

[1046] Input: Data packet sent

[1047] Data processing: Parsing incoming data and UI rendering

[1048] Output: A controllable interface

[1049] Step 5:

[1050] The user performs operations through the terminal, and the terminal collects the operation data.

[1051] Input: User actions (e.g. taps, text input)

[1052] Data processing: Convert user operations into log data

[1053] Output: Operation data

[1054] Step 6:

[1055] The device uses a camera and microphone to capture the user's facial expressions and tone of voice to collect emotional data.

[1056] Input: Camera video and audio input

[1057] Data Computing: Facial Expression and Voice Analysis

[1058] Output: Emotion data

[1059] Step 7:

[1060] The terminal transmits operation data and emotion data to the server.

[1061] Input: Operational data and emotional data

[1062] Data processing: Convert data into a format that can be sent (e.g., JSON format)

[1063] Output: Data packets sent

[1064] Step 8:

[1065] The server analyzes the received data and evaluates the user's comprehension and emotional state.

[1066] Input: Operational data and emotional data

[1067] Data Computation: Understanding and Emotion Assessment Using Data Analysis Models

[1068] Output: Comprehension and emotion evaluation results

[1069] Step 9:

[1070] The server generates feedback based on the evaluation results and transmits it to the terminal.

[1071] Input: Comprehension and emotion evaluation results

[1072] Data Calculation: Applying feedback generation logic

[1073] Output: Generated feedback data

[1074] Step 10:

[1075] The terminal displays the received feedback and provides it to the user.

[1076] Input: Feedback data

[1077] Data processing: Rendering received data to the UI

[1078] Output: Feedback display

[1079] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1080] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1081] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1082] [Third embodiment]

[1083] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1084] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1085] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1086] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1087] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1088] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1089] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1090] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1091] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1092] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1093] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1094] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1095] This invention is a system for realizing efficient and uniform education by collecting past question data, generating new questions, and providing them to users. This system is composed of the following phases.

[1096] 1. Problem generation phase

[1097] The server collects past exam data from a database, for example, across multiple subjects such as math, English, and history.

[1098] The server generates new questions based on the collected question data. At this time, the difficulty and format of the questions can be specified using the generation AI.

[1099] 2. Problem Delivery Phase

[1100] The server packets the generated problem in XML or JSON format and sends it to the terminal.

[1101] The device displays the received questions on a user interface that is designed to make it easy for users to answer intuitively.

[1102] 3. Response Phase

[1103] The user answers questions using the device, such as multiple-choice questions in a quiz format or essay-style math questions.

[1104] The device converts the user's response data into XML or JSON format and sends it to the server.

[1105] 4. Understanding Assessment Phase

[1106] The server receives the response data sent by the user and passes it to the analysis module.

[1107] The server's AI analyzes the answer data, determines whether the answer is correct, and evaluates the level of understanding. In this process, the accuracy of the answer, the time it takes, and the answering method are used as evaluation criteria.

[1108] 5. Feedback Phase

[1109] The server generates feedback for the user based on the analysis results, including the percentage of correct answers, explanations for incorrect answers, and suggestions for additional study.

[1110] The server sends the generated feedback to the terminal, which displays the feedback on a user interface.

[1111] Users can check the feedback and use it to improve their learning progress.

[1112] Specific examples

[1113] Problem generation phase:

[1114] For example, the server collects equation problems from the past 10 years from a database of mathematical problems.

[1115] The server generates new equation problems based on the collected data, for example, by setting the difficulty level to beginner, intermediate, or advanced and creating different types of problems.

[1116] Problem Delivery Phase:

[1117] The server generates equation problems and packets them to the user's tablet.

[1118] The device receives these questions and displays them on the tablet screen, laid out to make it easy for users to see and answer them.

[1119] Response Phase:

[1120] The user answers the displayed equation questions using a tablet, for example, by writing the equations on an answer sheet.

[1121] The terminal converts the answers written by the user into digital data and sends it to the server.

[1122] Comprehension assessment phase:

[1123] The server analyzes the received answer data, and the AI ​​determines whether the answer is correct or not. For example, it takes into account not only the accuracy of the answer but also the time it took to answer.

[1124] The server compares the user's answer data with the correct answer data to evaluate the level of understanding.

[1125] Feedback Phase:

[1126] Based on the evaluation results, the server generates feedback such as, "This part was not answered correctly, so please read the explanation below and try again."

[1127] The server sends the generated feedback to the tablet, which displays the feedback on its user interface.

[1128] The user reviews the feedback and re-studies the question to gain a better understanding of the incorrect answers.

[1129] In this way, the present invention can standardize the quality of education and reduce labor costs. Furthermore, by providing feedback according to each user's level of understanding, it can effectively support the improvement of each user's academic ability.

[1130] The processing flow will be explained below.

[1131] Step 1:

[1132] The server connects to the database and collects past exam data. For example, it can collect exam data for a specific subject (such as mathematics or English).

[1133] Step 2:

[1134] New problems are generated based on past problem data collected by the server. At this time, a generation AI is used to specify the difficulty and format of the problem. For example, equation problems with beginner, intermediate, and advanced difficulty levels can be created.

[1135] Step 3:

[1136] The server converts the generated questions into XML or JSON format and creates a data packet for distribution.

[1137] Step 4:

[1138] The server sends the data packets to the terminal via the Internet.

[1139] Step 5:

[1140] The device analyzes the received question data and displays the questions on the user interface. The layout is adjusted to make it easier for the user to answer intuitively.

[1141] Step 6:

[1142] The user uses the device to answer the displayed questions, for example by entering mathematical expressions using the touchscreen or by selecting answers from multiple options.

[1143] Step 7:

[1144] The device converts the user's response data into XML or JSON format and sends it to the server.

[1145] Step 8:

[1146] The server receives the response data sent by the user and passes it to the analysis module.

[1147] Step 9:

[1148] The server's generated AI analyzes the answer data and judges whether it is correct or not, for example, by evaluating the accuracy of the formula, the answer method, and the answer time.

[1149] Step 10:

[1150] The server evaluates the user's level of understanding and generates feedback based on that evaluation, including, for example, explanations for incorrect answers and additional practice questions.

[1151] Step 11:

[1152] The server generates feedback and sends it to the device.

[1153] Step 12:

[1154] The device analyzes the received feedback and displays it in the user interface, providing visual explanations and additional questions to help users understand the process.

[1155] Step 13:

[1156] Users can review the feedback and use it to improve their learning, retrying the questions if necessary to further their understanding.

[1157] Through the above steps, the present invention can provide education efficiently and effectively, and improve the academic ability of users.

[1158] Example 1

[1159] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1160] Conventional educational systems have the problem of providing uniform questions that cannot adequately address the individual learner's level of understanding or adaptability, resulting in reduced learning effectiveness. They also require a large amount of human resources, making it difficult to provide efficient educational support. Furthermore, they lack a mechanism for effectively utilizing past learning data, making it difficult to provide appropriate feedback based on the learner's progress. To solve these issues, a system is needed that can automatically generate new questions based on collected data and provide feedback tailored to each learner's individual level of understanding.

[1161] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1162] In this invention, the server includes a means for collecting past questions from a database, a means for inputting prompts to the AI ​​model to generate new questions, and a means for transmitting the generated questions to the terminal. This makes it possible to automatically generate new questions using past learning data and provide appropriate feedback according to the individual learner's level of understanding.

[1163] The "means for collecting past questions from a database" is a means having a function for acquiring past question data stored in a database.

[1164] A "means for inputting prompts to an AI model to generate new problems" is a means for inputting instructions to a generative AI model, thereby generating new problems.

[1165] "Means for transmitting generated problems to a terminal" means means for transmitting problems generated by an AI model to a terminal in a digital format (e.g., JSON or XML).

[1166] The "means for the user to answer questions via the terminal" refers to a means for the user to answer questions displayed via a user interface.

[1167] The "means for transmitting user response data to the server" refers to a means for transmitting the response data entered by the user to the server in digital form.

[1168] The "means for analyzing the user's response data and evaluating the level of understanding" is a means for analyzing the response data sent by the user and evaluating the user's level of understanding based on the results.

[1169] The "means for generating feedback based on comprehension" is a means for generating feedback for the user based on the results of the assessed comprehension.

[1170] The "means for transmitting the generated feedback to the terminal" refers to means for transmitting the generated feedback to the terminal and displaying it to the user.

[1171] "Means for generating new problems using an AI model" refers to means for automatically generating new problems based on past problem data using an artificial intelligence model.

[1172] A "prompt" is an input sentence that provides instructions to an AI model and includes detailed conditions such as the type and difficulty of the problem to be generated.

[1173] The present invention is a system that realizes efficient and uniform education by collecting past question data, generating new questions, and providing them to users. This system is realized using the following hardware and software.

[1174] Hardware used

[1175] 1. Server: A computer system that connects to a database and collects, analyzes, and transmits data.

[1176] 2. Device: The device (e.g., tablet, smartphone, computer) on which the user answers the questions and receives feedback.

[1177] Software used

[1178] 1. Database Management System: Database software such as MySQL, MongoDB, etc.

[1179] 2. Generative AI models: For example, generative AI models such as GPT-4.

[1180] 3. Programming languages ​​and frameworks: Python, Flask, Django, etc.

[1181] System Operation

[1182] The server first collects past problem data from a database, using a method such as an SQL query to retrieve data that matches specific conditions. For example, the following query is used:

[1183] SELECT FROM question_data WHERE subject = 'Mathematics' AND year >= 2013

[1184] Next, the server uses the generative AI model to generate new questions. The server then inputs prompts like the following to the generative AI model to specify the type and difficulty of the questions desired:

[1185] Please set the difficulty level of this problem to beginner. Please generate other problems as well.

[1186] The generated new question is converted into XML or JSON format by the server and sent to the terminal. For example, the following JSON format data is sent:

[1187] {"Question": "Solve the equation", "Difficulty": "Beginner"}

[1188] The terminal displays the questions received from the server on the user interface. The user answers the questions using a keyboard or touch screen. The terminal converts the user's answers into digital data and sends them back to the server.

[1189] The server analyzes the response data received from the user and evaluates the level of understanding. The analysis uses a generative AI model and an analysis module to evaluate the accuracy of the response, response time, response method, etc. The server then generates feedback based on the evaluation results and sends it to the user.

[1190] This feedback might look something like this:

[1191] This part was not answered correctly, so please read the explanation below and try again.

[1192] The device displays the received feedback on the user interface for the user to confirm, allowing the user to check their own understanding and engage in further learning.

[1193] This system makes it possible to generate new questions using past learning data and provide feedback based on individual levels of understanding, thereby providing efficient and effective educational support.

[1194] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1195] Step 1:

[1196] The server collects past problem data from a database.

[1197] Specifically, the server connects to a database management system (e.g., MySQL) and executes an SQL query such as the following:

[1198] Input: Database connection information, query conditions (e.g., subject "Mathematics", year "2013 or later").

[1199] Output: A set of historical question data.

[1200] SELECT FROM question_data WHERE subject = 'Mathematics' AND year >= 2013

[1201] The server records the number and contents of the data in a log to confirm that the acquired data is correct.

[1202] Step 2:

[1203] New questions are generated based on the problem data collected by the server.

[1204] Specifically, the server inputs a prompt to the generative AI model (e.g., GPT-4) and instructs it to generate a problem.

[1205] Input: Past question data, prompt text.

[1206] Output: The new problem that was generated.

[1207] Example prompt sentence:

[1208] Please set the difficulty level of this problem to beginner. Please generate other problems as well.

[1209] The server receives the generated problem, verifies its contents, and stores it in a log.

[1210] Step 3:

[1211] The server packets the generated problem in XML or JSON format and sends it to the terminal.

[1212] Specifically, the server converts the problem into JSON format as shown below and sends it to the terminal as an HTTP POST request.

[1213] Input: Generated problem data.

[1214] Output: Sending status to terminal.

[1215] {"Question": "Solve the equation", "Difficulty": "Beginner"}

[1216] The server checks the HTTP request status code (e.g., 200 OK) and logs the successful submission.

[1217] Step 4:

[1218] The problem received by the terminal is displayed on the user interface.

[1219] Specifically, the device parses the received JSON data and constructs a layout that is intuitive for the user.

[1220] Input: The problem data received from the server.

[1221] Output: The problem as displayed to the user.

[1222] The terminal generates a timestamp to record when the user begins to answer.

[1223] Step 5:

[1224] The user answers the questions using the device.

[1225] Specifically, the user inputs the answer using a keyboard or touch screen.

[1226] Input: The user's answer.

[1227] Output: The entered answer data.

[1228] The terminal saves the user's input and proceeds to the next step when the answer button is pressed.

[1229] Step 6:

[1230] The terminal transmits the user's response data to the server.

[1231] Specifically, the device converts the answer data into JSON format as shown below and sends it to the server as an HTTP POST request.

[1232] Input: User's answer data.

[1233] Output: Status of submission to server.

[1234] {"Answer": "x = 2", "User ID": "12345"}

[1235] The terminal receives a response from the server and confirms that the transmission was successful.

[1236] Step 7:

[1237] The server receives the response data sent by the user and passes it to the analysis module.

[1238] Specifically, the server reads the received data using a Python library (e.g., Pandas) and analyzes it.

[1239] Input: User response data.

[1240] Output: Analysis results.

[1241] The server logs the analysis results in a format similar to the following: {"Correct Answer": True, "Answer Time": "5 minutes"}.

[1242] Step 8:

[1243] The server evaluates the level of understanding based on the response data.

[1244] Specifically, the data is analyzed using an AI model and evaluated based on criteria such as correctness and response time.

[1245] Input: Parsed response data.

[1246] Output: Comprehension assessment results.

[1247] The server stores the evaluation results in a database.

[1248] Step 9:

[1249] The server generates feedback based on the results of the comprehension assessment and sends it to the terminal.

[1250] Specifically, the server uses an AI model to generate feedback and sends it to the device in JSON format as follows:

[1251] Input: Comprehension assessment results.

[1252] Output: The generated feedback.

[1253] Generate feedback similar to the following example:

[1254] This part was not answered correctly, so please read the explanation below and try again.

[1255] The server logs the feedback sending status.

[1256] Step 10:

[1257] The terminal displays the feedback on the user interface for the user to confirm.

[1258] Specifically, the terminal parses the received feedback and displays it to the user.

[1259] Input: Feedback received from the server.

[1260] Output: The feedback that is displayed to the user.

[1261] The terminal records the timestamp when the user confirms the feedback and confirms the content of the re-learning.

[1262] The above is the specific processing flow of this system.

[1263] (Application example 1)

[1264] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1265] In conventional factory robot training programs, operator skill evaluations and feedback are uniform, making it difficult to improve training according to individual levels of understanding.There is a need for a system that can efficiently generate technical problems actually required in the field, evaluate the operator's level of understanding based on those problems, and provide appropriate feedback.

[1266] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1267] In this invention, the server includes means for collecting past question data from a database, means for generating new questions using a generative AI model, means for transmitting the generated questions to a terminal, means for a user to answer the generated questions using a tablet and transmit the answer data to the server, means for analyzing the answer data and evaluating the level of understanding based on accuracy, answer time, and operation procedures, and means for generating feedback based on the analysis results and providing the user with an opportunity to re-study. This enables efficient evaluation and feedback tailored to individual levels of understanding in a factory robot operator training program.

[1268] "Past problems" are data on problems and training tasks that the user has previously tackled.

[1269] A "database" is an information system for managing and storing past questions and answer data.

[1270] "New problems" are new training problems generated using a generative AI model based on past problem data.

[1271] A "terminal" is a digital device through which a user accesses and responds to training programs and questions.

[1272] A "server" is a centralized computer system that manages problems and data, analyzes them, generates feedback, etc.

[1273] An "analysis module" is a program that runs on a server and analyzes the user's response data to evaluate their level of understanding.

[1274] A "generative AI model" is an artificial intelligence model that generates new problems based on past problem data.

[1275] "Feedback" is information that is generated based on the user's answers and includes an evaluation of the user's level of understanding and suggestions for relearning.

[1276] A "tablet" is a portable computing device that allows a user to answer training questions and receive feedback.

[1277] "Question difficulty" refers to the level of difficulty of the questions to be generated.

[1278] "Answer time" is the time it takes the user to answer the question.

[1279] An "operation procedure" is a process of a series of actions performed by a user when answering a question.

[1280] "Relearning" is the process by which a user re-learns based on feedback.

[1281] A "training program" is a series of educational courses that factory robot operators go through to acquire specific skills and knowledge.

[1282] The "evaluation criteria" are scales or indicators used to evaluate the user's response data.

[1283] The present invention is a system for evaluating a factory robot operator training program and providing feedback for improvement. This system collects past training question data, generates new questions, and provides them to users, thereby achieving efficient and uniform technical education. A specific embodiment of this system will be described below.

[1284] 1. Problem generation phase

[1285] The server collects past training problem data from a database containing problems from various technical fields (e.g., welding, assembly, and maintenance). Based on the collected data, the server generates new problems using a generative AI model (e.g., OpenAI's GPT-3). The difficulty and format of the generated problems can be specified as needed.

[1286] Specific examples

[1287] The server collects data on past welding technology problems and inputs the following prompt into GPT-3: "Generate new welding technology problems in the following format: medium difficulty, solution time within 30 minutes, use the following materials..." to generate new problems.

[1288] 2. Problem Delivery Phase

[1289] The generated questions are packetized in JSON format and sent to the terminal. The terminal displays the received questions on a user interface, which is designed to make the questions easy for users to see and answer intuitively.

[1290] Specific examples

[1291] The generated welding problems are packetized in JSON format and sent to a tablet, where they are displayed and the user works on them.

[1292] 3. Response Phase

[1293] The user answers questions using a terminal. The answer data is converted into digital data by the terminal and sent to the server. The server analyzes the received answer data and evaluates the level of understanding.

[1294] 4. Understanding Assessment Phase

[1295] The server's analysis module analyzes the response data and evaluates the level of understanding based on accuracy, response time, and operation procedures. The evaluation results are generated by the analysis module.

[1296] Specific examples

[1297] The server analyzes the user's response data and generates feedback such as, "The heat distribution during welding is not uniform, resulting in an improper joint. Please re-learn the cause and how to improve it."

[1298] 5. Feedback Phase

[1299] Feedback is generated based on the analysis results and sent to the device, which then displays the feedback on the user interface, giving the user an opportunity to re-learn.

[1300] Hardware and Software

[1301] Hardware: Smartphones and tablets (e.g. iPad)

[1302] Software: MySQL (database), JSON (data format), generative AI model (GPT-3), front-end (React Native)

[1303] Prompt Sentence Examples

[1304] Create a new welding problem in the following format:

[1305] Difficulty: Intermediate

[1306] Answer time: within 30 minutes

[1307] Materials used: iron, aluminum

[1308] In this way, the system of the present invention can efficiently implement training programs for factory robot operators and provide evaluations and feedback according to each operator's level of understanding, thereby effectively supporting the improvement of operator technical skills.

[1309] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1310] Step 1:

[1311] The server collects past question data from a database. The database stores past question data for each training program, and the server accesses and collects this data. The input is a database query, and the output is past question data. Specifically, the server accesses the database using an SQL query to retrieve the required data.

[1312] Step 2:

[1313] The server generates new problems using a generative AI model based on the collected past problem data. The input is the collected past problem data and the specified problem difficulty and format, and the output is the generated new problem. Specifically, a prompt sentence is input to the generative AI model (e.g., GPT-3) to generate a new problem.

[1314] Step 3:

[1315] The server packets the generated new problem in JSON format and sends it to the terminal. The input is the generated new problem, and the output is the problem data converted to JSON format. Specifically, the problem data is serialized in JSON format and sent to the terminal over the network.

[1316] Step 4:

[1317] The terminal displays the received problem data on the user interface. The input is the received JSON format problem data, and the output is the problem displayed to the user. Specifically, it deserializes the JSON data and renders the problem content on the screen.

[1318] Step 5:

[1319] The user answers questions through the terminal. The input is the answer selected or written by the user, and the output is the user's answer data. Specifically, the user enters the answer using the user interface and presses the send button to send the answer to the terminal.

[1320] Step 6:

[1321] The terminal sends the user's answer data to the server. The input is the user's answer data, and the output is the data to be sent to the server. Specifically, the answer data is converted into JSON format and sent to the server.

[1322] Step 7:

[1323] The server analyzes the received answer data and evaluates the level of comprehension. The input is the user's answer data, and the output is the comprehension evaluation result. Specifically, the server uses the analysis module to evaluate the accuracy of the answer, the answer time, and the operation procedure.

[1324] Step 8:

[1325] The server generates feedback based on the analysis results and sends it to the device. The input is the comprehension assessment result, and the output is the generated feedback data. Specifically, the server uses a feedback generation algorithm to create feedback for the user, converts it into JSON format, and sends it to the device.

[1326] Step 9:

[1327] The terminal displays the received feedback on the user interface. The input is the received feedback data, and the output is the feedback content displayed to the user. Specifically, the terminal deserializes the feedback data and renders it on the screen.

[1328] Through the above process steps, the system of the present invention can support a user's technical training program and provide efficient and individualized assessment and feedback according to the user's level of understanding.

[1329] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1330] The present invention is a system that provides education that takes into account not only the user's level of understanding but also their emotional state by combining a conventional education system with an emotion engine. This system is composed of the following phases.

[1331] 1. Problem generation phase

[1332] The server connects to a database and collects past exam data, such as questions for subjects like math, English, and history.

[1333] New questions are generated based on the question data collected by the server. Using the generation AI, questions can be generated by specifying their difficulty and format.

[1334] 2. Problem Delivery Phase

[1335] The server converts the generated questions into XML or JSON format and creates a data packet for distribution.

[1336] The server sends the data packets to the terminal via the Internet.

[1337] 3. Response Phase

[1338] The device analyzes the received question data and displays the questions on the user interface, which is designed to be laid out in a way that makes it easy for users to answer intuitively.

[1339] The user answers the questions using the device, for example by entering mathematical expressions using the touchscreen of a tablet or by selecting multiple choices.

[1340] 4. Emotional Data Collection Phase

[1341] An emotion engine built into the device analyzes the user's facial expressions and tone of voice to collect emotion data. For example, a camera can capture facial expressions and a microphone can analyze tone of voice.

[1342] The emotion data collected by the device is converted into XML or JSON format and sent to the server.

[1343] 5. Understanding Assessment Phase

[1344] The server receives the response data and emotion data sent by the user and passes them to the analysis module.

[1345] The server's generated AI analyzes the answer data, determines whether the answer is correct, and evaluates the level of understanding. It also evaluates the accuracy of the answer, the time it took to answer, and the answering method.

[1346] Furthermore, data from the emotion engine is taken into consideration and the emotional state of the user when answering is taken into account in the evaluation.

[1347] 6. Feedback generation phase

[1348] The server generates feedback based on the analysis results, taking into account the user's level of understanding and emotional state. For example, if a user has a high level of understanding but is nervous, the server may generate feedback such as "Good job! Stay calm and try again next time."

[1349] The server generates feedback and sends it to the device.

[1350] 7. Feedback display phase

[1351] The device analyzes the received feedback and displays it in the user interface. The feedback may include additional questions, explanations, or psychological support messages.

[1352] Users can review the feedback and use it to improve their learning, for example by re-studying incorrect answers to deepen their understanding.

[1353] Specific examples

[1354] Problem generation phase:

[1355] The server collects calculus problems from the past 10 years from a mathematical problem database.

[1356] The server generates new problems based on this data, such as "Beginner: Simple Differential Problems," "Intermediate: Definite Integral Problems," and "Advanced: Complex Integral Problems."

[1357] Problem Delivery Phase:

[1358] The server generates a problem packet and sends it to the user's tablet.

[1359] The device receives the question and displays it on the tablet screen.

[1360] Response Phase:

[1361] The user answers the displayed calculus problems using a tablet, for example, by writing mathematical formulas using a stylus pen.

[1362] The terminal converts the user's answers into digital data and sends it to the server.

[1363] Sentiment data collection phase:

[1364] The device's built-in camera captures the user's facial expressions, while the microphone analyzes the tone of the voice, detecting, for example, facial expressions that indicate impatience or a tense voice.

[1365] The terminal transmits this emotion data to the server.

[1366] Comprehension assessment phase:

[1367] The server analyzes the received answer data and evaluates whether the answer was correct or incorrect and the answer time.

[1368] Furthermore, emotional data is analyzed and the user's emotional state while answering is taken into account in the evaluation.

[1369] Feedback generation phase:

[1370] The server generates feedback based on the evaluation, including a message such as "Well done, but you seem a little nervous, so please relax and try again."

[1371] The server generates feedback and sends it to the user's tablet.

[1372] Feedback display phase:

[1373] The device displays the received feedback, which can include explanations based on comprehension and relaxation advice based on emotional data.

[1374] Users can review the feedback and use it to improve their learning experience next time.

[1375] In this way, the present invention can provide more comprehensive and effective learning support by providing education while taking into consideration the user's emotions.

[1376] The processing flow will be explained below.

[1377] Step 1:

[1378] The server connects to the database and collects past exam data, such as questions for each subject, such as mathematics, English, and history.

[1379] Step 2:

[1380] Based on past question data collected by the server, new questions are generated using a generation AI. For example, questions can be created by specifying a specific level of difficulty (beginner, intermediate, advanced) or format (multiple choice, written).

[1381] Step 3:

[1382] The server converts the generated questions into XML or JSON format and creates a data packet for distribution.

[1383] Step 4:

[1384] The server sends the data packets to the terminal via the Internet.

[1385] Step 5:

[1386] The device analyzes the received question data and displays the questions on the user interface. For example, the questions are laid out visually on a tablet screen.

[1387] Step 6:

[1388] The user uses the device to answer the displayed questions, for example by entering a mathematical formula or selecting multiple choices using the touchscreen.

[1389] Step 7:

[1390] The device converts the user's response data into XML or JSON format and sends it to the server.

[1391] Step 8:

[1392] The device's built-in emotion engine captures the user's facial expressions and tone of voice to collect emotion data. For example, the camera detects facial expressions and the microphone records tone of voice.

[1393] Step 9:

[1394] The emotion data collected by the device is converted into XML or JSON format and sent to the server.

[1395] Step 10:

[1396] The server receives the response data and emotion data sent by the user and passes them to the analysis module.

[1397] Step 11:

[1398] The server's generated AI analyzes the answer data and determines whether it is correct or incorrect and evaluates the level of understanding. For example, the accuracy of the answer, the answering time, and the answering method are used as evaluation criteria.

[1399] Step 12:

[1400] The server then performs additional analysis of the emotional data to assess the user's emotional state while answering the questions, such as analyzing facial expressions and tone of voice that indicate impatience or nervousness.

[1401] Step 13:

[1402] The server generates feedback based on the level of understanding and emotional state, such as a message like "Good job, but you seem a little nervous, so please relax and try again."

[1403] Step 14:

[1404] The server generates feedback and sends it to the device.

[1405] Step 15:

[1406] The device analyzes the received feedback and displays it in the user interface. For example, the feedback may include specific explanations based on the user's level of understanding or relaxation advice based on the user's emotional state.

[1407] Step 16:

[1408] Users can check the feedback and use it to improve their learning the next time. For example, they can re-study the incorrect answers and put the advice from the emotion engine into practice, improving their learning effectiveness.

[1409] Through the above steps, the present invention can comprehensively evaluate the user's understanding and emotions and provide optimal feedback.

[1410] Example 2

[1411] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1412] Conventional educational systems are limited to assessing users' learning progress and level of understanding, and lack support that takes into account the user's emotional state. This makes it difficult to improve learning efficiency and maintain motivation, potentially affecting long-term learning outcomes. Furthermore, the quality and variety of questions generated are limited, making it difficult to meet the diverse learning needs of users. It was necessary to solve these issues and create a system that provides more comprehensive and effective learning support.

[1413] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for collecting past data from a database, means for generating new questions based on the collected past data, and means for transmitting the generated questions to the terminal. This enables comprehensive learning support that takes into account not only the user's learning progress but also their emotional state. Specifically, new questions can be generated by specifying the difficulty level and format of the questions using a generative AI model, thereby meeting the diverse learning needs of users. Furthermore, by analyzing the user's response data and adding emotional data to the analysis, not only the user's level of understanding but also their emotional state during learning can be reflected in the evaluation. This allows for more personalized feedback to the user, improving learning efficiency and maintaining motivation.

[1414] I understand. Below are definitions of important terms included in the claims.

[1415] "Past data" refers to previously collected information related to the educational subject, such as question and answer data stored in a database.

[1416] A "database" is an information system used to efficiently store, manage, and search large amounts of data.

[1417] "Means for generating new questions" refers to the technology and algorithms used to generate new questions for users to answer based on collected past data.

[1418] A "generative AI model" refers to a machine learning model that uses artificial intelligence technology to generate new problems.

[1419] "Device" refers to an electronic device, such as a computer, tablet, or smartphone, that allows a user to access and answer questions.

[1420] "User" refers to an individual who uses the educational system to learn.

[1421] "Server" refers to the central system that collects data from the database, generates questions, and evaluates answers.

[1422] "Answer data" is digital information of the answer to the question that the user inputs through the terminal.

[1423] "Emotion data" is data that indicates the emotional state of the user analyzed from facial expressions, tone of voice, and the like.

[1424] "Evaluation" refers to the process of analyzing the user's response data and emotional data to determine the user's level of understanding and emotional state.

[1425] "Feedback" refers to study advice and instructions on next steps provided to the user based on the evaluation results.

[1426] "Analysis" refers to the process of extracting meaningful information from collected data and determining the user's level of understanding and emotional state.

[1427] The above are definitions of important terms contained in the claims.

[1428] The present invention is a system that provides education that takes into account not only the user's level of understanding but also their emotional state. This system aims to improve the efficiency and effectiveness of learning by combining a conventional education system with an emotional engine. The following describes how to specifically implement the invention.

[1429] System Overview

[1430] This system consists of a server, terminals, and users. The roles of each are as follows:

[1431] Server: Collects data from the database, generates new questions, and sends them to the device. It also analyzes the user's response data and emotional data, evaluates their understanding and emotional state, and generates feedback.

[1432] Terminal: Provides an interface for users to answer questions, collects answer data and emotion data, and sends them to the server. Receives feedback and displays it to the user.

[1433] User: Answers questions and receives feedback via the device.

[1434] Hardware and Software Configuration

[1435] The server is configured using, for example, a Python environment and a MySQL database, and uses a generative AI model such as OpenAI's GPT-3 to generate problems.

[1436] The devices are electronic devices such as tablets, smartphones, and PCs, and the user interface is built using JavaScript, HTML, and CSS. Emotion data is collected using libraries such as OpenCV and TensorFlow.

[1437] Users use these terminals to carry out learning activities.

[1438] Processing flow

[1439] 1. Problem generation:

[1440] The server connects to a MySQL database and collects past problem data using SQL queries.

[1441] Based on the collected data, a generative AI model (e.g., GPT-3) is used to generate new problems, with prompts being entered to generate the problems.

[1442] The generated questions are converted into JSON format and sent to the terminal.

[1443] Example prompt sentence:

[1444] "Generate beginner calculus problems"

[1445] "The user has a high level of understanding but is nervous, so generate feedback with advice to relax."

[1446] 2. Question delivery and answers:

[1447] The terminal analyzes the received problem data and displays it on the user interface.

[1448] The user enters mathematical expressions and answers using the tablet's touchscreen.

[1449] The terminal converts the user's answers into digital data and sends it to the server.

[1450] 3. Emotional Data Collection:

[1451] The device's built-in camera and microphone capture the user's facial expressions and tone of voice to collect emotional data.

[1452] Using OpenCV and TensorFlow, we analyze this data to determine the user's emotional state.

[1453] The collected emotion data is converted into JSON format and sent to the server.

[1454] 4. Comprehension assessment and feedback generation:

[1455] The server analyzes the response data and emotion data received using the Python pandas library to evaluate comprehension and emotional state.

[1456] Generative AI models are used to generate feedback based on the evaluation results, such as "Good job, but you seem a little nervous, so please relax and try again."

[1457] The generated feedback is sent to the device.

[1458] 5. Feedback display:

[1459] The terminal analyzes the received feedback and displays it on the user interface.

[1460] Users can review the feedback and use it to improve their learning experience next time.

[1461] In this way, the system of the present invention can simultaneously consider the user's level of understanding and emotional state, and provide more comprehensive and effective learning support.

[1462] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1463] Processing Steps

[1464] Step 1:

[1465] This is the procedure by which the server collects past data. Specifically, the server connects to a MySQL database and collects past problem data using SQL queries.

[1466] Input: MySQL database connection information and SQL query

[1467] Output: Problem data (past problem data)

[1468] What it does: The server uses Python's mysql-connector library to connect to a database named my_database, then executes the SQL query "SELECT FROM math_problems WHERE subject='calculus' AND year BETWEEN 2011 AND 2021;" to retrieve past problem data.

[1469] Step 2:

[1470] This is the procedure where the server generates new problems using a generative AI model. Based on past problem data collected by the server, new problems are generated using a generative AI model (e.g., GPT-3).

[1471] Input: Past problem data and prompts for the generative AI model

[1472] Output: New problem data

[1473] Specific operation: The server sends a prompt such as "Generate beginner calculus problems" to the OpenAI API based on the collected past problem data. The API responds with new problem data.

[1474] Step 3:

[1475] This is the procedure for sending questions generated by the server to the terminal. The generated question data is converted into JSON format and sent to the terminal via a RESTful API.

[1476] Input: Newly generated question data

[1477] Output: JSON data ready to be sent to the terminal

[1478] How it works: The server converts the generated question data into JSON format using Python's json library, and then sends the question data to the terminal using a RESTful API created using Flask.

[1479] Step 4:

[1480] This is the procedure in which the terminal analyzes the received question data and displays it on the user interface.

[1481] Input: Question data in JSON format sent from the server

[1482] Output: The problem as displayed in the user interface

[1483] Specific behavior: The device parses the JSON data using JavaScript and displays the problem in the user interface using HTML and CSS. Example: document.getElementById('problem').innerText = json_data['problem_text'];

[1484] Step 5:

[1485] This is the procedure by which the user answers the questions.

[1486] Input: The problem displayed on the terminal

[1487] Output: User response data

[1488] Specific actions: The user answers the displayed questions using the tablet's touchscreen, for example, by entering mathematical expressions using a stylus.

[1489] Step 6:

[1490] This is the procedure in which the terminal converts the user's answer into digital data and sends it to the server.

[1491] Input: User's handwritten answer or input

[1492] Output: Response data sent to the server

[1493] Specific behavior: The device captures the answer data, converts it to JSON format (e.g. answer_data = JSON.stringify(captured_answer)), and then sends it to the server using the fetch API.

[1494] Step 7:

[1495] This is a procedure in which the emotion engine built into the device collects the user's emotion data.

[1496] Input: User's facial expression and tone of voice

[1497] Output: Parsed emotion data

[1498] Specific operation: The device uses OpenCV to capture camera images and analyze facial features. It also uses a microphone to record voice tone and uses TensorFlow's voice analysis model to determine emotions.

[1499] Step 8:

[1500] This is the procedure by which the terminal transmits emotion data to the server.

[1501] Input: Collected emotion data

[1502] Output: Emotion data sent to the server

[1503] Specific operation: The device converts the emotion data into JSON format and sends it to the server using the fetch API.

[1504] Step 9:

[1505] This is a procedure for the server to analyze the received response data and emotion data.

[1506] Input: Answer data and emotion data

[1507] Output: Analysis results

[1508] Specific operation: The server uses Python's pandas library to convert the data into a data frame and evaluates comprehension and emotional state.

[1509] Step 10:

[1510] This is a procedure in which the server generates feedback based on the evaluation results.

[1511] Input: Analysis results and evaluation criteria

[1512] Output: Feedback message

[1513] What it does: Uses a generative AI model to generate feedback based on the user's comprehension and emotional state. For example, "Good job, but you seem a little nervous. Relax and try again."

[1514] Step 11:

[1515] This is a procedure for the server to send generated feedback to the terminal.

[1516] Input: Feedback message

[1517] Output: Feedback data ready to be sent to the device

[1518] Specific operation: The server converts the generated feedback message into JSON format and sends it to the terminal using a RESTful API.

[1519] Step 12:

[1520] This is a procedure for displaying the feedback received by the terminal on the user interface.

[1521] Input: Feedback data sent from the server

[1522] Output: Feedback message displayed in the user interface

[1523] What it does: It uses JavaScript to parse the feedback data and displays it in the user interface using HTML and CSS. Example: document.getElementById('feedback').innerText = json_data['feedback_text'];

[1524] The above is a detailed explanation of the specific processing steps of the program of this system.

[1525] (Application example 2)

[1526] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1527] Conventional electronic payment services do not optimize the payment process by taking into account the user's emotional state, which can lead to feelings of tension or impatience that can worsen the payment experience. Furthermore, because feedback based on the user's emotional state is not provided, it is difficult for users to receive advice or reassurance appropriate to their emotional state. The objective of the present invention is to solve these problems and enable users to make electronic payments in a more relaxed environment.

[1528] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1529] In this invention, the server includes means for collecting past data from a database, means for generating new data based on the collected past data, and means for transmitting the generated data to the terminal, thereby making it possible to provide feedback based on the user's level of understanding and emotional state.

[1530] "Historical Data" refers to information previously collected and stored by the System.

[1531] A "database" refers to a system that stores and manages structured information in an organized manner.

[1532] "Collection methods" refers to the methods and techniques by which data is obtained from databases and other sources.

[1533] "New data" refers to new information generated based on existing data.

[1534] "Means of generation" refers to methods and technologies for creating new data based on collected data.

[1535] "Terminal" refers to equipment or devices that a user operates.

[1536] "Transmission means" refers to the method or technology by which data or information is sent to another device or server.

[1537] "Operation data" refers to information that records the operations and inputs performed by a user using a terminal.

[1538] A "server" refers to a computer that processes data and provides services over a network.

[1539] "Analysis" refers to the process of analyzing collected data or information to arrive at a particular conclusion or result.

[1540] "Level of understanding" refers to an index that indicates how well the user understands the content.

[1541] "User emotion" refers to the emotional state a user has at a particular moment.

[1542] "Feedback" refers to responses or advice provided based on a user's actions or status.

[1543] "Camera" refers to a device that captures images and converts them into digital data.

[1544] "Facial expression capture" refers to the process of recording a user's facial expressions using a camera.

[1545] A "microphone" refers to a device that collects sound and converts it into digital data.

[1546] "Tone of voice analysis" refers to the process of analyzing voice data collected by a microphone to identify a user's emotional state.

[1547] "Relaxation advice" refers to instructions or advice to help the user relax.

[1548] The present invention is a system that optimizes the electronic payment process by taking into account the emotional state of the user. This system is composed of a server and a terminal, and is characterized by analyzing the user's emotions and dynamically adjusting the payment process and feedback based on the results.

[1549] The program for implementing the present invention operates as follows.

[1550] First, the server collects past data from a database. This data serves as the basis for generating new data. Next, the generated data is sent to the device. The device acts as a device that accepts user operations, using a camera and microphone to capture the user's facial expressions and tone of voice. This data is sent to the server along with the operation data.

[1551] The server analyzes the received data and evaluates the user's level of understanding and emotional state. The analysis module uses EmotionRecognizer (an emotion analysis library), which takes emotional state into account in addition to traditional methods for evaluating comprehension. The server generates feedback based on these evaluation results and sends it to the terminal. If the user is relaxed, a simple payment method (face recognition) is used, but if the user is nervous, additional confirmation steps (OTP, password entry) are applied. In this way, the payment process is dynamically adjusted according to the user's emotional state.

[1552] The feedback includes advice to encourage relaxation and is displayed to the user through the terminal. This feedback is generated by the FeedbackGenerator (feedback generation module).

[1553] For example, if a user is nervous about purchasing a high-value item, the system adds redundant confirmation steps. On the other hand, when making everyday small payments, facial recognition allows for quicker payments. This allows users to experience the optimal payment process based on their emotional state.

[1554] Use the following prompt for the generative AI model:

[1555] "Generate the following code: A Python program that analyzes emotions from face captures and dynamically determines the electronic payment method accordingly."

[1556] This invention provides a smoother and more secure electronic payment experience that takes into account the user's emotional state.

[1557] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1558] Step 1:

[1559] The server collects historical data from a database.

[1560] Input: Database query

[1561] Data manipulation: Extracting target data based on a query

[1562] Output: Collected historical data

[1563] Step 2:

[1564] The server generates new data based on the collected past data.

[1565] Input: Collected historical data

[1566] Data calculation: Applying new data generation logic using models

[1567] Output: Newly generated data

[1568] Step 3:

[1569] The server transmits the generated data to the terminal.

[1570] Input: Newly generated data

[1571] Data processing: Convert data into a format that can be sent (e.g., JSON format)

[1572] Output: Data packets sent

[1573] Step 4:

[1574] Based on the received data, the terminal displays an interface that the user can operate.

[1575] Input: Data packet sent

[1576] Data processing: Parsing incoming data and UI rendering

[1577] Output: A controllable interface

[1578] Step 5:

[1579] The user performs operations through the terminal, and the terminal collects the operation data.

[1580] Input: User actions (e.g. taps, text input)

[1581] Data processing: Convert user operations into log data

[1582] Output: Operation data

[1583] Step 6:

[1584] The device uses a camera and microphone to capture the user's facial expressions and tone of voice to collect emotional data.

[1585] Input: Camera video and audio input

[1586] Data Computing: Facial Expression and Voice Analysis

[1587] Output: Emotion data

[1588] Step 7:

[1589] The terminal transmits operation data and emotion data to the server.

[1590] Input: Operational data and emotional data

[1591] Data processing: Convert data into a format that can be sent (e.g., JSON format)

[1592] Output: Data packets sent

[1593] Step 8:

[1594] The server analyzes the received data and evaluates the user's comprehension and emotional state.

[1595] Input: Operational data and emotional data

[1596] Data Computation: Understanding and Emotion Assessment Using Data Analysis Models

[1597] Output: Comprehension and emotion evaluation results

[1598] Step 9:

[1599] The server generates feedback based on the evaluation results and transmits it to the terminal.

[1600] Input: Comprehension and emotion evaluation results

[1601] Data Calculation: Applying feedback generation logic

[1602] Output: Generated feedback data

[1603] Step 10:

[1604] The terminal displays the received feedback and provides it to the user.

[1605] Input: Feedback data

[1606] Data processing: Rendering received data to the UI

[1607] Output: Feedback display

[1608] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1609] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1610] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1611] [Fourth embodiment]

[1612] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1613] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1614] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1615] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1616] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1617] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1618] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1619] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1620] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1621] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1622] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1623] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1624] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1625] This invention is a system for realizing efficient and uniform education by collecting past question data, generating new questions, and providing them to users. This system is composed of the following phases.

[1626] 1. Problem generation phase

[1627] The server collects past exam data from a database, for example, across multiple subjects such as math, English, and history.

[1628] The server generates new questions based on the collected question data. At this time, the difficulty and format of the questions can be specified using the generation AI.

[1629] 2. Problem Delivery Phase

[1630] The server packets the generated problem in XML or JSON format and sends it to the terminal.

[1631] The device displays the received questions on a user interface that is designed to make it easy for users to answer intuitively.

[1632] 3. Response Phase

[1633] The user answers questions using the device, such as multiple-choice questions in a quiz format or essay-style math questions.

[1634] The device converts the user's response data into XML or JSON format and sends it to the server.

[1635] 4. Understanding Assessment Phase

[1636] The server receives the response data sent by the user and passes it to the analysis module.

[1637] The server's AI analyzes the answer data, determines whether the answer is correct, and evaluates the level of understanding. In this process, the accuracy of the answer, the time it takes, and the answering method are used as evaluation criteria.

[1638] 5. Feedback Phase

[1639] The server generates feedback for the user based on the analysis results, including the percentage of correct answers, explanations for incorrect answers, and suggestions for additional study.

[1640] The server sends the generated feedback to the terminal, which displays the feedback on a user interface.

[1641] Users can check the feedback and use it to improve their learning progress.

[1642] Specific examples

[1643] Problem generation phase:

[1644] For example, the server collects equation problems from the past 10 years from a database of mathematical problems.

[1645] The server generates new equation problems based on the collected data, for example, by setting the difficulty level to beginner, intermediate, or advanced and creating different types of problems.

[1646] Problem Delivery Phase:

[1647] The server generates equation problems and packets them to the user's tablet.

[1648] The device receives these questions and displays them on the tablet screen, laid out to make it easy for users to see and answer them.

[1649] Response Phase:

[1650] The user answers the displayed equation questions using a tablet, for example, by writing the equations on an answer sheet.

[1651] The terminal converts the answers written by the user into digital data and sends it to the server.

[1652] Comprehension assessment phase:

[1653] The server analyzes the received answer data, and the AI ​​determines whether the answer is correct or not. For example, it takes into account not only the accuracy of the answer but also the time it took to answer.

[1654] The server compares the user's answer data with the correct answer data to evaluate the level of understanding.

[1655] Feedback Phase:

[1656] Based on the evaluation results, the server generates feedback such as, "This part was not answered correctly, so please read the explanation below and try again."

[1657] The server sends the generated feedback to the tablet, which displays the feedback on its user interface.

[1658] The user reviews the feedback and re-studies the question to gain a better understanding of the incorrect answers.

[1659] In this way, the present invention can standardize the quality of education and reduce labor costs. Furthermore, by providing feedback according to each user's level of understanding, it can effectively support the improvement of each user's academic ability.

[1660] The processing flow will be explained below.

[1661] Step 1:

[1662] The server connects to the database and collects past exam data. For example, it can collect exam data for a specific subject (such as mathematics or English).

[1663] Step 2:

[1664] New problems are generated based on past problem data collected by the server. At this time, a generation AI is used to specify the difficulty and format of the problem. For example, equation problems with beginner, intermediate, and advanced difficulty levels can be created.

[1665] Step 3:

[1666] The server converts the generated questions into XML or JSON format and creates a data packet for distribution.

[1667] Step 4:

[1668] The server sends the data packets to the terminal via the Internet.

[1669] Step 5:

[1670] The device analyzes the received question data and displays the questions on the user interface. The layout is adjusted to make it easier for the user to answer intuitively.

[1671] Step 6:

[1672] The user uses the device to answer the displayed questions, for example by entering mathematical expressions using the touchscreen or by selecting answers from multiple options.

[1673] Step 7:

[1674] The device converts the user's response data into XML or JSON format and sends it to the server.

[1675] Step 8:

[1676] The server receives the response data sent by the user and passes it to the analysis module.

[1677] Step 9:

[1678] The server's generated AI analyzes the answer data and judges whether it is correct or not, for example, by evaluating the accuracy of the formula, the answer method, and the answer time.

[1679] Step 10:

[1680] The server evaluates the user's level of understanding and generates feedback based on that evaluation, including, for example, explanations for incorrect answers and additional practice questions.

[1681] Step 11:

[1682] The server generates feedback and sends it to the device.

[1683] Step 12:

[1684] The device analyzes the received feedback and displays it in the user interface, providing visual explanations and additional questions to help users understand the process.

[1685] Step 13:

[1686] Users can review the feedback and use it to improve their learning, retrying the questions if necessary to further their understanding.

[1687] Through the above steps, the present invention can provide education efficiently and effectively, and improve the academic ability of users.

[1688] Example 1

[1689] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1690] Conventional educational systems have the problem of providing uniform questions that cannot adequately address the individual learner's level of understanding or adaptability, resulting in reduced learning effectiveness. They also require a large amount of human resources, making it difficult to provide efficient educational support. Furthermore, they lack a mechanism for effectively utilizing past learning data, making it difficult to provide appropriate feedback based on the learner's progress. To solve these issues, a system is needed that can automatically generate new questions based on collected data and provide feedback tailored to each learner's individual level of understanding.

[1691] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1692] In this invention, the server includes a means for collecting past questions from a database, a means for inputting prompts to the AI ​​model to generate new questions, and a means for transmitting the generated questions to the terminal. This makes it possible to automatically generate new questions using past learning data and provide appropriate feedback according to the individual learner's level of understanding.

[1693] The "means for collecting past questions from a database" is a means having a function for acquiring past question data stored in a database.

[1694] A "means for inputting prompts to an AI model to generate new problems" is a means for inputting instructions to a generative AI model, thereby generating new problems.

[1695] "Means for transmitting generated problems to a terminal" means means for transmitting problems generated by an AI model to a terminal in a digital format (e.g., JSON or XML).

[1696] The "means for the user to answer questions via the terminal" refers to a means for the user to answer questions displayed via a user interface.

[1697] The "means for transmitting user response data to the server" refers to a means for transmitting the response data entered by the user to the server in digital form.

[1698] The "means for analyzing the user's response data and evaluating the level of understanding" is a means for analyzing the response data sent by the user and evaluating the user's level of understanding based on the results.

[1699] The "means for generating feedback based on comprehension" is a means for generating feedback for the user based on the results of the assessed comprehension.

[1700] The "means for transmitting the generated feedback to the terminal" refers to means for transmitting the generated feedback to the terminal and displaying it to the user.

[1701] "Means for generating new problems using an AI model" refers to means for automatically generating new problems based on past problem data using an artificial intelligence model.

[1702] A "prompt" is an input sentence that provides instructions to an AI model and includes detailed conditions such as the type and difficulty of the problem to be generated.

[1703] The present invention is a system that realizes efficient and uniform education by collecting past question data, generating new questions, and providing them to users. This system is realized using the following hardware and software.

[1704] Hardware used

[1705] 1. Server: A computer system that connects to a database and collects, analyzes, and transmits data.

[1706] 2. Device: The device (e.g., tablet, smartphone, computer) on which the user answers the questions and receives feedback.

[1707] Software used

[1708] 1. Database Management System: Database software such as MySQL, MongoDB, etc.

[1709] 2. Generative AI models: For example, generative AI models such as GPT-4.

[1710] 3. Programming languages ​​and frameworks: Python, Flask, Django, etc.

[1711] System Operation

[1712] The server first collects past problem data from a database, using a method such as an SQL query to retrieve data that matches specific conditions. For example, the following query is used:

[1713] SELECT FROM question_data WHERE subject = 'Mathematics' AND year >= 2013

[1714] Next, the server uses the generative AI model to generate new questions. The server then inputs prompts like the following to the generative AI model to specify the type and difficulty of the questions desired:

[1715] Please set the difficulty level of this problem to beginner. Please generate other problems as well.

[1716] The generated new question is converted into XML or JSON format by the server and sent to the terminal. For example, the following JSON format data is sent:

[1717] {"Question": "Solve the equation", "Difficulty": "Beginner"}

[1718] The terminal displays the questions received from the server on the user interface. The user answers the questions using a keyboard or touch screen. The terminal converts the user's answers into digital data and sends them back to the server.

[1719] The server analyzes the response data received from the user and evaluates the level of understanding. The analysis uses a generative AI model and an analysis module to evaluate the accuracy of the response, response time, response method, etc. The server then generates feedback based on the evaluation results and sends it to the user.

[1720] This feedback might look something like this:

[1721] This part was not answered correctly, so please read the explanation below and try again.

[1722] The device displays the received feedback on the user interface for the user to confirm, allowing the user to check their own understanding and engage in further learning.

[1723] This system makes it possible to generate new questions using past learning data and provide feedback based on individual levels of understanding, thereby providing efficient and effective educational support.

[1724] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1725] Step 1:

[1726] The server collects past problem data from a database.

[1727] Specifically, the server connects to a database management system (e.g., MySQL) and executes an SQL query such as the following:

[1728] Input: Database connection information, query conditions (e.g., subject "Mathematics", year "2013 or later").

[1729] Output: A set of historical question data.

[1730] SELECT FROM question_data WHERE subject = 'Mathematics' AND year >= 2013

[1731] The server records the number and contents of the data in a log to confirm that the acquired data is correct.

[1732] Step 2:

[1733] New questions are generated based on the problem data collected by the server.

[1734] Specifically, the server inputs a prompt to the generative AI model (e.g., GPT-4) and instructs it to generate a problem.

[1735] Input: Past question data, prompt text.

[1736] Output: The new problem that was generated.

[1737] Example prompt sentence:

[1738] Please set the difficulty level of this problem to beginner. Please generate other problems as well.

[1739] The server receives the generated problem, verifies its contents, and stores it in a log.

[1740] Step 3:

[1741] The server packets the generated problem in XML or JSON format and sends it to the terminal.

[1742] Specifically, the server converts the problem into JSON format as shown below and sends it to the terminal as an HTTP POST request.

[1743] Input: Generated problem data.

[1744] Output: Sending status to terminal.

[1745] {"Question": "Solve the equation", "Difficulty": "Beginner"}

[1746] The server checks the HTTP request status code (e.g., 200 OK) and logs the successful submission.

[1747] Step 4:

[1748] The problem received by the terminal is displayed on the user interface.

[1749] Specifically, the device parses the received JSON data and constructs a layout that is intuitive for the user.

[1750] Input: The problem data received from the server.

[1751] Output: The problem as displayed to the user.

[1752] The terminal generates a timestamp to record when the user begins to answer.

[1753] Step 5:

[1754] The user answers the questions using the device.

[1755] Specifically, the user inputs the answer using a keyboard or touch screen.

[1756] Input: The user's answer.

[1757] Output: The entered answer data.

[1758] The terminal saves the user's input and proceeds to the next step when the answer button is pressed.

[1759] Step 6:

[1760] The terminal transmits the user's response data to the server.

[1761] Specifically, the device converts the answer data into JSON format as shown below and sends it to the server as an HTTP POST request.

[1762] Input: User's answer data.

[1763] Output: Status of submission to server.

[1764] {"Answer": "x = 2", "User ID": "12345"}

[1765] The terminal receives a response from the server and confirms that the transmission was successful.

[1766] Step 7:

[1767] The server receives the response data sent by the user and passes it to the analysis module.

[1768] Specifically, the server reads the received data using a Python library (e.g., Pandas) and analyzes it.

[1769] Input: User response data.

[1770] Output: Analysis results.

[1771] The server logs the analysis results in a format similar to the following: {"Correct Answer": True, "Answer Time": "5 minutes"}.

[1772] Step 8:

[1773] The server evaluates the level of understanding based on the response data.

[1774] Specifically, the data is analyzed using an AI model and evaluated based on criteria such as correctness and response time.

[1775] Input: Parsed response data.

[1776] Output: Comprehension assessment results.

[1777] The server stores the evaluation results in a database.

[1778] Step 9:

[1779] The server generates feedback based on the results of the comprehension assessment and sends it to the terminal.

[1780] Specifically, the server uses an AI model to generate feedback and sends it to the device in JSON format as follows:

[1781] Input: Comprehension assessment results.

[1782] Output: The generated feedback.

[1783] Generate feedback similar to the following example:

[1784] This part was not answered correctly, so please read the explanation below and try again.

[1785] The server logs the feedback sending status.

[1786] Step 10:

[1787] The terminal displays the feedback on the user interface for the user to confirm.

[1788] Specifically, the terminal parses the received feedback and displays it to the user.

[1789] Input: Feedback received from the server.

[1790] Output: The feedback that is displayed to the user.

[1791] The terminal records the timestamp when the user confirms the feedback and confirms the content of the re-learning.

[1792] The above is the specific processing flow of this system.

[1793] (Application example 1)

[1794] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1795] In conventional factory robot training programs, operator skill evaluations and feedback are uniform, making it difficult to improve training according to individual levels of understanding.There is a need for a system that can efficiently generate technical problems actually required in the field, evaluate the operator's level of understanding based on those problems, and provide appropriate feedback.

[1796] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1797] In this invention, the server includes means for collecting past question data from a database, means for generating new questions using a generative AI model, means for transmitting the generated questions to a terminal, means for a user to answer the generated questions using a tablet and transmit the answer data to the server, means for analyzing the answer data and evaluating the level of understanding based on accuracy, answer time, and operation procedures, and means for generating feedback based on the analysis results and providing the user with an opportunity to re-study. This enables efficient evaluation and feedback tailored to individual levels of understanding in a factory robot operator training program.

[1798] "Past problems" are data on problems and training tasks that the user has previously tackled.

[1799] A "database" is an information system for managing and storing past questions and answer data.

[1800] "New problems" are new training problems generated using a generative AI model based on past problem data.

[1801] A "terminal" is a digital device through which a user accesses and responds to training programs and questions.

[1802] A "server" is a centralized computer system that manages problems and data, analyzes them, generates feedback, etc.

[1803] An "analysis module" is a program that runs on a server and analyzes the user's response data to evaluate their level of understanding.

[1804] A "generative AI model" is an artificial intelligence model that generates new problems based on past problem data.

[1805] "Feedback" is information that is generated based on the user's answers and includes an evaluation of the user's level of understanding and suggestions for relearning.

[1806] A "tablet" is a portable computing device that allows a user to answer training questions and receive feedback.

[1807] "Question difficulty" refers to the level of difficulty of the questions to be generated.

[1808] "Answer time" is the time it takes the user to answer the question.

[1809] An "operation procedure" is a process of a series of actions performed by a user when answering a question.

[1810] "Relearning" is the process by which a user re-learns based on feedback.

[1811] A "training program" is a series of educational courses that factory robot operators go through to acquire specific skills and knowledge.

[1812] The "evaluation criteria" are scales or indicators used to evaluate the user's response data.

[1813] The present invention is a system for evaluating a factory robot operator training program and providing feedback for improvement. This system collects past training question data, generates new questions, and provides them to users, thereby achieving efficient and uniform technical education. A specific embodiment of this system will be described below.

[1814] 1. Problem generation phase

[1815] The server collects past training problem data from a database containing problems from various technical fields (e.g., welding, assembly, and maintenance). Based on the collected data, the server generates new problems using a generative AI model (e.g., OpenAI's GPT-3). The difficulty and format of the generated problems can be specified as needed.

[1816] Specific examples

[1817] The server collects data on past welding technology problems and inputs the following prompt into GPT-3: "Generate new welding technology problems in the following format: medium difficulty, solution time within 30 minutes, use the following materials..." to generate new problems.

[1818] 2. Problem Delivery Phase

[1819] The generated questions are packetized in JSON format and sent to the terminal. The terminal displays the received questions on a user interface, which is designed to make the questions easy for users to see and answer intuitively.

[1820] Specific examples

[1821] The generated welding problems are packetized in JSON format and sent to a tablet, where they are displayed and the user works on them.

[1822] 3. Response Phase

[1823] The user answers questions using a terminal. The answer data is converted into digital data by the terminal and sent to the server. The server analyzes the received answer data and evaluates the level of understanding.

[1824] 4. Understanding Assessment Phase

[1825] The server's analysis module analyzes the response data and evaluates the level of understanding based on accuracy, response time, and operation procedures. The evaluation results are generated by the analysis module.

[1826] Specific examples

[1827] The server analyzes the user's response data and generates feedback such as, "The heat distribution during welding is not uniform, resulting in an improper joint. Please re-learn the cause and how to improve it."

[1828] 5. Feedback Phase

[1829] Feedback is generated based on the analysis results and sent to the device, which then displays the feedback on the user interface, giving the user an opportunity to re-learn.

[1830] Hardware and Software

[1831] Hardware: Smartphones and tablets (e.g. iPad)

[1832] Software: MySQL (database), JSON (data format), generative AI model (GPT-3), front-end (React Native)

[1833] Prompt Sentence Examples

[1834] Create a new welding problem in the following format:

[1835] Difficulty: Intermediate

[1836] Answer time: within 30 minutes

[1837] Materials used: iron, aluminum

[1838] In this way, the system of the present invention can efficiently implement training programs for factory robot operators and provide evaluations and feedback according to each operator's level of understanding, thereby effectively supporting the improvement of operator technical skills.

[1839] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1840] Step 1:

[1841] The server collects past question data from a database. The database stores past question data for each training program, and the server accesses and collects this data. The input is a database query, and the output is past question data. Specifically, the server accesses the database using an SQL query to retrieve the required data.

[1842] Step 2:

[1843] The server generates new problems using a generative AI model based on the collected past problem data. The input is the collected past problem data and the specified problem difficulty and format, and the output is the generated new problem. Specifically, a prompt sentence is input to the generative AI model (e.g., GPT-3) to generate a new problem.

[1844] Step 3:

[1845] The server packets the generated new problem in JSON format and sends it to the terminal. The input is the generated new problem, and the output is the problem data converted to JSON format. Specifically, the problem data is serialized in JSON format and sent to the terminal over the network.

[1846] Step 4:

[1847] The terminal displays the received problem data on the user interface. The input is the received JSON format problem data, and the output is the problem displayed to the user. Specifically, it deserializes the JSON data and renders the problem content on the screen.

[1848] Step 5:

[1849] The user answers questions through the terminal. The input is the answer selected or written by the user, and the output is the user's answer data. Specifically, the user enters the answer using the user interface and presses the send button to send the answer to the terminal.

[1850] Step 6:

[1851] The terminal sends the user's answer data to the server. The input is the user's answer data, and the output is the data to be sent to the server. Specifically, the answer data is converted into JSON format and sent to the server.

[1852] Step 7:

[1853] The server analyzes the received answer data and evaluates the level of comprehension. The input is the user's answer data, and the output is the comprehension evaluation result. Specifically, the server uses the analysis module to evaluate the accuracy of the answer, the answer time, and the operation procedure.

[1854] Step 8:

[1855] The server generates feedback based on the analysis results and sends it to the device. The input is the comprehension assessment result, and the output is the generated feedback data. Specifically, the server uses a feedback generation algorithm to create feedback for the user, converts it into JSON format, and sends it to the device.

[1856] Step 9:

[1857] The terminal displays the received feedback on the user interface. The input is the received feedback data, and the output is the feedback content displayed to the user. Specifically, the terminal deserializes the feedback data and renders it on the screen.

[1858] Through the above process steps, the system of the present invention can support a user's technical training program and provide efficient and individualized assessment and feedback according to the user's level of understanding.

[1859] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1860] The present invention is a system that provides education that takes into account not only the user's level of understanding but also their emotional state by combining a conventional education system with an emotion engine. This system is composed of the following phases.

[1861] 1. Problem generation phase

[1862] The server connects to a database and collects past exam data, such as questions for subjects like math, English, and history.

[1863] New questions are generated based on the question data collected by the server. Using the generation AI, questions can be generated by specifying their difficulty and format.

[1864] 2. Problem Delivery Phase

[1865] The server converts the generated questions into XML or JSON format and creates a data packet for distribution.

[1866] The server sends the data packets to the terminal via the Internet.

[1867] 3. Response Phase

[1868] The device analyzes the received question data and displays the questions on the user interface, which is designed to be laid out in a way that makes it easy for users to answer intuitively.

[1869] The user answers the questions using the device, for example by entering mathematical expressions using the touchscreen of a tablet or by selecting multiple choices.

[1870] 4. Emotional Data Collection Phase

[1871] An emotion engine built into the device analyzes the user's facial expressions and tone of voice to collect emotion data. For example, a camera can capture facial expressions and a microphone can analyze tone of voice.

[1872] The emotion data collected by the device is converted into XML or JSON format and sent to the server.

[1873] 5. Understanding Assessment Phase

[1874] The server receives the response data and emotion data sent by the user and passes them to the analysis module.

[1875] The server's generated AI analyzes the answer data, determines whether the answer is correct, and evaluates the level of understanding. It also evaluates the accuracy of the answer, the time it took to answer, and the answering method.

[1876] Furthermore, data from the emotion engine is taken into consideration and the emotional state of the user when answering is taken into account in the evaluation.

[1877] 6. Feedback generation phase

[1878] The server generates feedback based on the analysis results, taking into account the user's level of understanding and emotional state. For example, if a user has a high level of understanding but is nervous, the server may generate feedback such as "Good job! Stay calm and try again next time."

[1879] The server generates feedback and sends it to the device.

[1880] 7. Feedback display phase

[1881] The device analyzes the received feedback and displays it in the user interface. The feedback may include additional questions, explanations, or psychological support messages.

[1882] Users can review the feedback and use it to improve their learning, for example by re-studying incorrect answers to deepen their understanding.

[1883] Specific examples

[1884] Problem generation phase:

[1885] The server collects calculus problems from the past 10 years from a mathematical problem database.

[1886] The server generates new problems based on this data, such as "Beginner: Simple Differential Problems," "Intermediate: Definite Integral Problems," and "Advanced: Complex Integral Problems."

[1887] Problem Delivery Phase:

[1888] The server generates a problem packet and sends it to the user's tablet.

[1889] The device receives the question and displays it on the tablet screen.

[1890] Response Phase:

[1891] The user answers the displayed calculus problems using a tablet, for example, by writing mathematical formulas using a stylus pen.

[1892] The terminal converts the user's answers into digital data and sends it to the server.

[1893] Sentiment data collection phase:

[1894] The device's built-in camera captures the user's facial expressions, while the microphone analyzes the tone of the voice, detecting, for example, facial expressions that indicate impatience or a tense voice.

[1895] The terminal transmits this emotion data to the server.

[1896] Comprehension assessment phase:

[1897] The server analyzes the received answer data and evaluates whether the answer was correct or incorrect and the answer time.

[1898] Furthermore, emotional data is analyzed and the user's emotional state while answering is taken into account in the evaluation.

[1899] Feedback generation phase:

[1900] The server generates feedback based on the evaluation, including a message such as "Well done, but you seem a little nervous, so please relax and try again."

[1901] The server generates feedback and sends it to the user's tablet.

[1902] Feedback display phase:

[1903] The device displays the received feedback, which can include explanations based on comprehension and relaxation advice based on emotional data.

[1904] Users can review the feedback and use it to improve their learning experience next time.

[1905] In this way, the present invention can provide more comprehensive and effective learning support by providing education while taking into consideration the user's emotions.

[1906] The processing flow will be explained below.

[1907] Step 1:

[1908] The server connects to the database and collects past exam data, such as questions for each subject, such as mathematics, English, and history.

[1909] Step 2:

[1910] Based on past question data collected by the server, new questions are generated using a generation AI. For example, questions can be created by specifying a specific level of difficulty (beginner, intermediate, advanced) or format (multiple choice, written).

[1911] Step 3:

[1912] The server converts the generated questions into XML or JSON format and creates a data packet for distribution.

[1913] Step 4:

[1914] The server sends the data packets to the terminal via the Internet.

[1915] Step 5:

[1916] The device analyzes the received question data and displays the questions on the user interface. For example, the questions are laid out visually on a tablet screen.

[1917] Step 6:

[1918] The user uses the device to answer the displayed questions, for example by entering a mathematical formula or selecting multiple choices using the touchscreen.

[1919] Step 7:

[1920] The device converts the user's response data into XML or JSON format and sends it to the server.

[1921] Step 8:

[1922] The device's built-in emotion engine captures the user's facial expressions and tone of voice to collect emotion data. For example, the camera detects facial expressions and the microphone records tone of voice.

[1923] Step 9:

[1924] The emotion data collected by the device is converted into XML or JSON format and sent to the server.

[1925] Step 10:

[1926] The server receives the response data and emotion data sent by the user and passes them to the analysis module.

[1927] Step 11:

[1928] The server's generated AI analyzes the answer data and determines whether it is correct or incorrect and evaluates the level of understanding. For example, the accuracy of the answer, the answering time, and the answering method are used as evaluation criteria.

[1929] Step 12:

[1930] The server then performs additional analysis of the emotional data to assess the user's emotional state while answering the questions, such as analyzing facial expressions and tone of voice that indicate impatience or nervousness.

[1931] Step 13:

[1932] The server generates feedback based on the level of understanding and emotional state, such as a message like "Good job, but you seem a little nervous, so please relax and try again."

[1933] Step 14:

[1934] The server generates feedback and sends it to the device.

[1935] Step 15:

[1936] The device analyzes the received feedback and displays it in the user interface. For example, the feedback may include specific explanations based on the user's level of understanding or relaxation advice based on the user's emotional state.

[1937] Step 16:

[1938] Users can check the feedback and use it to improve their learning the next time. For example, they can re-study the incorrect answers and put the advice from the emotion engine into practice, improving their learning effectiveness.

[1939] Through the above steps, the present invention can comprehensively evaluate the user's understanding and emotions and provide optimal feedback.

[1940] Example 2

[1941] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1942] Conventional educational systems are limited to assessing users' learning progress and level of understanding, and lack support that takes into account the user's emotional state. This makes it difficult to improve learning efficiency and maintain motivation, potentially affecting long-term learning outcomes. Furthermore, the quality and variety of questions generated are limited, making it difficult to meet the diverse learning needs of users. It was necessary to solve these issues and create a system that provides more comprehensive and effective learning support.

[1943] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for collecting past data from a database, means for generating new questions based on the collected past data, and means for transmitting the generated questions to the terminal. This enables comprehensive learning support that takes into account not only the user's learning progress but also their emotional state. Specifically, new questions can be generated by specifying the difficulty level and format of the questions using a generative AI model, thereby meeting the diverse learning needs of users. Furthermore, by analyzing the user's response data and adding emotional data to the analysis, not only the user's level of understanding but also their emotional state during learning can be reflected in the evaluation. This allows for more personalized feedback to the user, improving learning efficiency and maintaining motivation.

[1944] I understand. Below are definitions of important terms included in the claims.

[1945] "Past data" refers to previously collected information related to the educational subject, such as question and answer data stored in a database.

[1946] A "database" is an information system used to efficiently store, manage, and search large amounts of data.

[1947] "Means for generating new questions" refers to the technology and algorithms used to generate new questions for users to answer based on collected past data.

[1948] A "generative AI model" refers to a machine learning model that uses artificial intelligence technology to generate new problems.

[1949] "Device" refers to an electronic device, such as a computer, tablet, or smartphone, that allows a user to access and answer questions.

[1950] "User" refers to an individual who uses the educational system to learn.

[1951] "Server" refers to the central system that collects data from the database, generates questions, and evaluates answers.

[1952] "Answer data" is digital information of the answer to the question that the user inputs through the terminal.

[1953] "Emotion data" is data that indicates the emotional state of the user analyzed from facial expressions, tone of voice, and the like.

[1954] "Evaluation" refers to the process of analyzing the user's response data and emotional data to determine the user's level of understanding and emotional state.

[1955] "Feedback" refers to study advice and instructions on next steps provided to the user based on the evaluation results.

[1956] "Analysis" refers to the process of extracting meaningful information from collected data and determining the user's level of understanding and emotional state.

[1957] The above are definitions of important terms contained in the claims.

[1958] The present invention is a system that provides education that takes into account not only the user's level of understanding but also their emotional state. This system aims to improve the efficiency and effectiveness of learning by combining a conventional education system with an emotional engine. The following describes how to specifically implement the invention.

[1959] System Overview

[1960] This system consists of a server, terminals, and users. The roles of each are as follows:

[1961] Server: Collects data from the database, generates new questions, and sends them to the device. It also analyzes the user's response data and emotional data, evaluates their understanding and emotional state, and generates feedback.

[1962] Terminal: Provides an interface for users to answer questions, collects answer data and emotion data, and sends them to the server. Receives feedback and displays it to the user.

[1963] User: Answers questions and receives feedback via the device.

[1964] Hardware and Software Configuration

[1965] The server is configured using, for example, a Python environment and a MySQL database, and uses a generative AI model such as OpenAI's GPT-3 to generate problems.

[1966] The devices are electronic devices such as tablets, smartphones, and PCs, and the user interface is built using JavaScript, HTML, and CSS. Emotion data is collected using libraries such as OpenCV and TensorFlow.

[1967] Users use these terminals to carry out learning activities.

[1968] Processing flow

[1969] 1. Problem generation:

[1970] The server connects to a MySQL database and collects past problem data using SQL queries.

[1971] Based on the collected data, a generative AI model (e.g., GPT-3) is used to generate new problems, with prompts being entered to generate the problems.

[1972] The generated questions are converted into JSON format and sent to the terminal.

[1973] Example prompt sentence:

[1974] "Generate beginner calculus problems"

[1975] "The user has a high level of understanding but is nervous, so generate feedback with advice to relax."

[1976] 2. Question delivery and answers:

[1977] The terminal analyzes the received problem data and displays it on the user interface.

[1978] The user enters mathematical expressions and answers using the tablet's touchscreen.

[1979] The terminal converts the user's answers into digital data and sends it to the server.

[1980] 3. Emotional Data Collection:

[1981] The device's built-in camera and microphone capture the user's facial expressions and tone of voice to collect emotional data.

[1982] Using OpenCV and TensorFlow, we analyze this data to determine the user's emotional state.

[1983] The collected emotion data is converted into JSON format and sent to the server.

[1984] 4. Comprehension assessment and feedback generation:

[1985] The server analyzes the response data and emotion data received using the Python pandas library to evaluate comprehension and emotional state.

[1986] Generative AI models are used to generate feedback based on the evaluation results, such as "Good job, but you seem a little nervous, so please relax and try again."

[1987] The generated feedback is sent to the device.

[1988] 5. Feedback display:

[1989] The terminal analyzes the received feedback and displays it on the user interface.

[1990] Users can review the feedback and use it to improve their learning experience next time.

[1991] In this way, the system of the present invention can simultaneously consider the user's level of understanding and emotional state, and provide more comprehensive and effective learning support.

[1992] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1993] Processing Steps

[1994] Step 1:

[1995] This is the procedure by which the server collects past data. Specifically, the server connects to a MySQL database and collects past problem data using SQL queries.

[1996] Input: MySQL database connection information and SQL query

[1997] Output: Problem data (past problem data)

[1998] What it does: The server uses Python's mysql-connector library to connect to a database named my_database, then executes the SQL query "SELECT FROM math_problems WHERE subject='calculus' AND year BETWEEN 2011 AND 2021;" to retrieve past problem data.

[1999] Step 2:

[2000] This is the procedure where the server generates new problems using a generative AI model. Based on past problem data collected by the server, new problems are generated using a generative AI model (e.g., GPT-3).

[2001] Input: Past problem data and prompts for the generative AI model

[2002] Output: New problem data

[2003] Specific operation: The server sends a prompt such as "Generate beginner calculus problems" to the OpenAI API based on the collected past problem data. The API responds with new problem data.

[2004] Step 3:

[2005] This is the procedure for sending questions generated by the server to the terminal. The generated question data is converted into JSON format and sent to the terminal via a RESTful API.

[2006] Input: Newly generated question data

[2007] Output: JSON data ready to be sent to the terminal

[2008] How it works: The server converts the generated question data into JSON format using Python's json library, and then sends the question data to the terminal using a RESTful API created using Flask.

[2009] Step 4:

[2010] This is the procedure in which the terminal analyzes the received question data and displays it on the user interface.

[2011] Input: Question data in JSON format sent from the server

[2012] Output: The problem as displayed in the user interface

[2013] Specific behavior: The device parses the JSON data using JavaScript and displays the problem in the user interface using HTML and CSS. Example: document.getElementById('problem').innerText = json_data['problem_text'];

[2014] Step 5:

[2015] This is the procedure by which the user answers the questions.

[2016] Input: The problem displayed on the terminal

[2017] Output: User response data

[2018] Specific actions: The user answers the displayed questions using the tablet's touchscreen, for example, by entering mathematical expressions using a stylus.

[2019] Step 6:

[2020] This is the procedure in which the terminal converts the user's answer into digital data and sends it to the server.

[2021] Input: User's handwritten answer or input

[2022] Output: Response data sent to the server

[2023] Specific behavior: The device captures the answer data, converts it to JSON format (e.g. answer_data = JSON.stringify(captured_answer)), and then sends it to the server using the fetch API.

[2024] Step 7:

[2025] This is a procedure in which the emotion engine built into the device collects the user's emotion data.

[2026] Input: User's facial expression and tone of voice

[2027] Output: Parsed emotion data

[2028] Specific operation: The device uses OpenCV to capture camera images and analyze facial features. It also uses a microphone to record voice tone and uses TensorFlow's voice analysis model to determine emotions.

[2029] Step 8:

[2030] This is the procedure by which the terminal transmits emotion data to the server.

[2031] Input: Collected emotion data

[2032] Output: Emotion data sent to the server

[2033] Specific operation: The device converts the emotion data into JSON format and sends it to the server using the fetch API.

[2034] Step 9:

[2035] This is a procedure for the server to analyze the received response data and emotion data.

[2036] Input: Answer data and emotion data

[2037] Output: Analysis results

[2038] Specific operation: The server uses Python's pandas library to convert the data into a data frame and evaluates comprehension and emotional state.

[2039] Step 10:

[2040] This is a procedure in which the server generates feedback based on the evaluation results.

[2041] Input: Analysis results and evaluation criteria

[2042] Output: Feedback message

[2043] What it does: Uses a generative AI model to generate feedback based on the user's comprehension and emotional state. For example, "Good job, but you seem a little nervous. Relax and try again."

[2044] Step 11:

[2045] This is a procedure for the server to send generated feedback to the terminal.

[2046] Input: Feedback message

[2047] Output: Feedback data ready to be sent to the device

[2048] Specific operation: The server converts the generated feedback message into JSON format and sends it to the terminal using a RESTful API.

[2049] Step 12:

[2050] This is a procedure for displaying the feedback received by the terminal on the user interface.

[2051] Input: Feedback data sent from the server

[2052] Output: Feedback message displayed in the user interface

[2053] What it does: It uses JavaScript to parse the feedback data and displays it in the user interface using HTML and CSS. Example: document.getElementById('feedback').innerText = json_data['feedback_text'];

[2054] The above is a detailed explanation of the specific processing steps of the program of this system.

[2055] (Application example 2)

[2056] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2057] Conventional electronic payment services do not optimize the payment process by taking into account the user's emotional state, which can lead to feelings of tension or impatience that can worsen the payment experience. Furthermore, because feedback based on the user's emotional state is not provided, it is difficult for users to receive advice or reassurance appropriate to their emotional state. The objective of the present invention is to solve these problems and enable users to make electronic payments in a more relaxed environment.

[2058] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[2059] In this invention, the server includes means for collecting past data from a database, means for generating new data based on the collected past data, and means for transmitting the generated data to the terminal, thereby making it possible to provide feedback based on the user's level of understanding and emotional state.

[2060] "Historical Data" refers to information previously collected and stored by the System.

[2061] A "database" refers to a system that stores and manages structured information in an organized manner.

[2062] "Collection methods" refers to the methods and techniques by which data is obtained from databases and other sources.

[2063] "New data" refers to new information generated based on existing data.

[2064] "Means of generation" refers to methods and technologies for creating new data based on collected data.

[2065] "Terminal" refers to equipment or devices that a user operates.

[2066] "Transmission means" refers to the method or technology by which data or information is sent to another device or server.

[2067] "Operation data" refers to information that records the operations and inputs performed by a user using a terminal.

[2068] A "server" refers to a computer that processes data and provides services over a network.

[2069] "Analysis" refers to the process of analyzing collected data or information to arrive at a particular conclusion or result.

[2070] "Level of understanding" refers to an index that indicates how well the user understands the content.

[2071] "User emotion" refers to the emotional state a user has at a particular moment.

[2072] "Feedback" refers to responses or advice provided based on a user's actions or status.

[2073] "Camera" refers to a device that captures images and converts them into digital data.

[2074] "Facial expression capture" refers to the process of recording a user's facial expressions using a camera.

[2075] A "microphone" refers to a device that collects sound and converts it into digital data.

[2076] "Tone of voice analysis" refers to the process of analyzing voice data collected by a microphone to identify a user's emotional state.

[2077] "Relaxation advice" refers to instructions or advice to help the user relax.

[2078] The present invention is a system that optimizes the electronic payment process by taking into account the emotional state of the user. This system is composed of a server and a terminal, and is characterized by analyzing the user's emotions and dynamically adjusting the payment process and feedback based on the results.

[2079] The program for implementing the present invention operates as follows.

[2080] First, the server collects past data from a database. This data serves as the basis for generating new data. Next, the generated data is sent to the device. The device acts as a device that accepts user operations, using a camera and microphone to capture the user's facial expressions and tone of voice. This data is sent to the server along with the operation data.

[2081] The server analyzes the received data and evaluates the user's level of understanding and emotional state. The analysis module uses EmotionRecognizer (an emotion analysis library), which takes emotional state into account in addition to traditional methods for evaluating comprehension. The server generates feedback based on these evaluation results and sends it to the terminal. If the user is relaxed, a simple payment method (face recognition) is used, but if the user is nervous, additional confirmation steps (OTP, password entry) are applied. In this way, the payment process is dynamically adjusted according to the user's emotional state.

[2082] The feedback includes advice to encourage relaxation and is displayed to the user through the terminal. This feedback is generated by the FeedbackGenerator (feedback generation module).

[2083] For example, if a user is nervous about purchasing a high-value item, the system adds redundant confirmation steps. On the other hand, when making everyday small payments, facial recognition allows for quicker payments. This allows users to experience the optimal payment process based on their emotional state.

[2084] Use the following prompt for the generative AI model:

[2085] "Generate the following code: A Python program that analyzes emotions from face captures and dynamically determines the electronic payment method accordingly."

[2086] This invention provides a smoother and more secure electronic payment experience that takes into account the user's emotional state.

[2087] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[2088] Step 1:

[2089] The server collects historical data from a database.

[2090] Input: Database query

[2091] Data manipulation: Extracting target data based on a query

[2092] Output: Collected historical data

[2093] Step 2:

[2094] The server generates new data based on the collected past data.

[2095] Input: Collected historical data

[2096] Data calculation: Applying new data generation logic using models

[2097] Output: Newly generated data

[2098] Step 3:

[2099] The server transmits the generated data to the terminal.

[2100] Input: Newly generated data

[2101] Data processing: Convert data into a format that can be sent (e.g., JSON format)

[2102] Output: Data packets sent

[2103] Step 4:

[2104] Based on the received data, the terminal displays an interface that the user can operate.

[2105] Input: Data packet sent

[2106] Data processing: Parsing incoming data and UI rendering

[2107] Output: A controllable interface

[2108] Step 5:

[2109] The user performs operations through the terminal, and the terminal collects the operation data.

[2110] Input: User actions (e.g. taps, text input)

[2111] Data processing: Convert user operations into log data

[2112] Output: Operation data

[2113] Step 6:

[2114] The device uses a camera and microphone to capture the user's facial expressions and tone of voice to collect emotional data.

[2115] Input: Camera video and audio input

[2116] Data Computing: Facial Expression and Voice Analysis

[2117] Output: Emotion data

[2118] Step 7:

[2119] The terminal transmits operation data and emotion data to the server.

[2120] Input: Operational data and emotional data

[2121] Data processing: Convert data into a format that can be sent (e.g., JSON format)

[2122] Output: Data packets sent

[2123] Step 8:

[2124] The server analyzes the received data and evaluates the user's comprehension and emotional state.

[2125] Input: Operational data and emotional data

[2126] Data Computation: Understanding and Emotion Assessment Using Data Analysis Models

[2127] Output: Comprehension and emotion evaluation results

[2128] Step 9:

[2129] The server generates feedback based on the evaluation results and transmits it to the terminal.

[2130] Input: Comprehension and emotion evaluation results

[2131] Data Calculation: Applying feedback generation logic

[2132] Output: Generated feedback data

[2133] Step 10:

[2134] The terminal displays the received feedback and provides it to the user.

[2135] Input: Feedback data

[2136] Data processing: Rendering received data to the UI

[2137] Output: Feedback display

[2138] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[2139] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[2140] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[2141] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[2142] FIG. 9 illustrates an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and behaviors arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[2143] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[2144] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[2145] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[2146] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[2147] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[2148] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[2149] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[2150] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[2151] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[2152] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[2153] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[2154] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[2155] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[2156] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[2157] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[2158] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[2159] The following is further disclosed regarding the above embodiment.

[2160] (Claim 1)

[2161] A means of collecting past questions from a database;

[2162] A means for generating new questions based on collected past question data;

[2163] means for transmitting the generated questions to a terminal;

[2164] A means for a user to answer questions through a terminal;

[2165] means for transmitting user response data to a server;

[2166] A means for analyzing user response data and evaluating the level of understanding;

[2167] a means for generating feedback based on comprehension;

[2168] means for transmitting the generated feedback to the terminal;

[2169] means for displaying feedback to the user through the terminal;

[2170] A system including:

[2171] (Claim 2)

[2172] 2. The system according to claim 1, wherein the means for generating new questions has a function for specifying the difficulty level and format of the questions.

[2173] (Claim 3)

[2174] 2. The system according to claim 1, wherein the means for evaluating comprehension has a function of analyzing the accuracy, time, and method of answering.

[2175] "Example 1"

[2176] (Claim 1)

[2177] A means of collecting past questions from a database;

[2178] A means for generating new questions based on collected past question data;

[2179] means for transmitting the generated questions to a terminal;

[2180] A means for a user to answer questions through a terminal;

[2181] means for transmitting user response data to a server;

[2182] A means for analyzing user response data and evaluating the level of understanding;

[2183] a means for generating feedback based on comprehension;

[2184] means for transmitting the generated feedback to the terminal;

[2185] means for displaying feedback to the user through the terminal;

[2186] A means of generating new problems using AI models; and

[2187] a means for inputting prompts to the AI ​​model to direct problem generation;

[2188] A system including:

[2189] (Claim 2)

[2190] 2. The system according to claim 1, wherein the means for generating new questions has a function for specifying the difficulty level and format of the questions.

[2191] (Claim 3)

[2192] 2. The system according to claim 1, wherein the means for evaluating comprehension has a function of analyzing the accuracy, time, and method of answering.

[2193] "Application Example 1"

[2194] (Claim 1)

[2195] A means of collecting past questions from a database;

[2196] A means for generating new questions based on collected past question data;

[2197] means for transmitting the generated questions to a terminal;

[2198] A means for a user to answer questions through a terminal;

[2199] means for transmitting user response data to a server;

[2200] A means for analyzing user response data and evaluating the level of understanding;

[2201] a means for generating feedback based on comprehension;

[2202] means for transmitting the generated feedback to the terminal;

[2203] means for displaying feedback to the user through the terminal;

[2204] Furthermore, when this system is applied to the evaluation of a training program for a factory robot, a problem generation means based on the training program is provided.

[2205] A means to generate new questions using generative AI models and specify the difficulty and format of the questions;

[2206] A means for a user to use a tablet to answer the generated questions, convert the answer data into a digital format, and transmit the digital data to a server;

[2207] An evaluation means for analyzing the answer data and using accuracy, answer time, and operation procedure as evaluation criteria;

[2208] means for generating feedback based on the analysis results and displaying the feedback for the user to restudy;

[2209] A system including:

[2210] (Claim 2)

[2211] 2. The system of claim 1, wherein the means for generating new questions has a function of specifying the difficulty and format of the questions using a generative AI model.

[2212] (Claim 3)

[2213] 2. The system according to claim 1, wherein the means for evaluating the level of understanding has a function of analyzing the accuracy of answers, time, and operation procedures.

[2214] "Example 2: Combining Emotion Engines"

[2215] (Claim 1)

[2216] a means of collecting historical data from a database;

[2217] A means of generating new questions based on collected past data;

[2218] means for transmitting the generated questions to a terminal;

[2219] A means for a user to answer questions through a terminal;

[2220] means for transmitting user response data to a server;

[2221] A means for analyzing user response data and evaluating the level of understanding;

[2222] a means for generating feedback based on comprehension;

[2223] means for transmitting the generated feedback to the terminal;

[2224] means for displaying feedback to the user through the terminal;

[2225] means for collecting user emotion data;

[2226] A means for analyzing emotion data and reflecting it in evaluations;

[2227] A system including:

[2228] (Claim 2)

[2229] 2. The system of claim 1, wherein the means for generating new questions has a function of specifying the difficulty and format of the questions using a generative AI model.

[2230] (Claim 3)

[2231] 2. The system according to claim 1, wherein the means for evaluating the level of understanding has a function of analyzing the accuracy, time, and method of the user's answer, and further adding emotional data to the evaluation.

[2232] "Application example 2 when combining emotion engines"

[2233] (Claim 1)

[2234] a means of collecting historical data from a database;

[2235] A means of generating new data based on collected historical data;

[2236] means for transmitting the generated data to a terminal;

[2237] A means for a user to perform operations on the data through the terminal;

[2238] means for transmitting user operation data to a server;

[2239] A means for analyzing user operation data and evaluating the level of understanding;

[2240] In the process, a means of analyzing user sentiment,

[2241] means for generating feedback based on comprehension and user sentiment;

[2242] means for transmitting the generated feedback to the terminal;

[2243] means for displaying feedback to the user through the terminal;

[2244] A system including:

[2245] (Claim 2)

[2246] 2. The system according to claim 1, wherein the means for generating new data has a function for specifying the level of difficulty and format of the data.

[2247] (Claim 3)

[2248] 2. The system according to claim 1, wherein the means for evaluating the level of understanding has a function of analyzing the accuracy, time, and method of operation.

[2249] (Claim 4)

[2250] 10. The system of claim 1, wherein the means for analyzing user emotions includes capturing facial expressions using a camera and analyzing tone of voice using a microphone.

[2251] (Claim 5)

[2252] 10. The system of claim 1, wherein the feedback based on comprehension and user emotion includes relaxation advice. [Explanation of symbols]

[2253] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. A means of collecting past questions from a database; A means for generating new questions based on collected past question data; means for transmitting the generated questions to a terminal; A means for a user to answer questions through a terminal; means for transmitting user response data to a server; A means for analyzing user response data and evaluating the level of understanding; a means for generating feedback based on comprehension; means for transmitting the generated feedback to the terminal; means for displaying feedback to the user through the terminal; A system including:

2. 2. The system according to claim 1, wherein the means for generating new questions has a function for specifying the difficulty level and format of the questions.

3. 2. The system according to claim 1, wherein the means for evaluating the level of understanding has a function of analyzing the accuracy, time, and method of answering.

Images